Method for acquiring network model, storage medium and electronic device

Abstract

The application discloses a method for obtaining a network model, a storage medium and an electronic device. The method comprises the following steps: obtaining a target data set; constructing a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model comprises a plurality of network substructures, the dynamic routing structure is used for selecting to skip or execute at least one network substructure in the plurality of network substructures, and the second network model is an initial dynamic inference network model based on path selection; training the second network model by using the target data set to obtain a third network model; and performing binary processing on the dynamic routing structure based on the third network model to obtain a fourth network model. The application solves the technical problem of low learning efficiency, low flexibility and large calculation consumption of the network model in the related art.

Description

Technical Field

[0001] This invention relates to the fields of computer technology and artificial intelligence, and more specifically, to a method, storage medium, and electronic device for acquiring network models. Background Technology

[0002] In the field of artificial intelligence, the number of network layers in deep learning models is constantly increasing to improve their accuracy. However, an excessive number of network layers does not necessarily improve model accuracy. While some images (such as those with very low resolution, indistinct features, or those difficult to distinguish with the naked eye) do require a large number of network layers to extract features and classify them, in reality, most images in application scenarios can be feature-extracted and classified without a very large number of network layers. Therefore, the methods for obtaining network models in related technologies have the following drawbacks: low learning efficiency and flexibility of the network models, and high computational cost.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a method, storage medium, and electronic device for acquiring network models, thereby at least addressing the technical problems of low learning efficiency, low flexibility, and high computational consumption in related technologies for acquiring network models.

[0005] According to one aspect of the present invention, a method for obtaining a network model is provided, comprising: obtaining a target dataset, wherein the target dataset is an image classification dataset; constructing a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model comprising: multiple network substructures, the dynamic routing structure being used to select and skip or execute at least one of the multiple network substructures, and the second network model being an initial dynamic inference network model based on path selection; training the second network model using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; and binarizing the dynamic routing structure based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model, and the fourth network model is used to classify the target image to be classified.

[0006] According to another aspect of the present invention, an image classification method is also provided, comprising: acquiring a target image to be classified; using a dynamic prediction network model to perform classification prediction on the target image to be classified, and obtaining a classification result; wherein, the dynamic prediction network model is determined by a target dataset, a first network model, and a dynamic routing structure, the target dataset is an image classification dataset, the first network model and the dynamic routing structure are used to construct a second network model, the first network model is an image classification network model, the first network model includes: multiple network substructures, the dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures, the second network model is an initial dynamic inference network model based on path selection, the target dataset is used to train the second network model to obtain a third network model, the third network model is a target dynamic inference network model with optimized network parameters, and the third network model is used to perform binarization processing on the dynamic routing structure to obtain a dynamic prediction network model.

[0007] According to another aspect of the present invention, a storage medium is also provided, the storage medium including a stored program, wherein, when the program is executed, the device where the storage medium is located executes any of the above-described methods for obtaining a network model.

[0008] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions for processing the following steps: acquiring a target dataset, wherein the target dataset is an image classification dataset; constructing a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model comprising: multiple network substructures, the dynamic routing structure being used to select to skip or execute at least one of the multiple network substructures, and the second network model being an initial dynamic inference network model based on path selection; training the second network model using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; and binarizing the dynamic routing structure based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model, and the fourth network model is used to classify the target image to be classified.

[0009] In this embodiment of the invention, a target dataset is first obtained, wherein the target dataset is an image classification dataset. A second network model is constructed based on a first network model and a dynamic routing structure. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select and skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The second network model is trained using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters. The dynamic routing structure is further binarized based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model. The fourth network model is used to classify the target image to be classified. This achieves the purpose of constructing a dynamic prediction network model based on the image classification network model and the dynamic routing structure to classify the target image, thereby improving the learning efficiency and flexibility of the network model and reducing the computational cost of image classification. This solves the technical problems of low learning efficiency, low flexibility, and high computational cost of network model acquisition methods in related technologies. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0011] Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for acquiring a network model is shown.

[0012] Figure 2 This is a flowchart of a method for obtaining a network model according to an embodiment of the present invention;

[0013] Figure 3 This is a schematic diagram of an optional image feature evolution according to an embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of the structure of an optional network model according to an embodiment of the present invention;

[0015] Figure 5 This is a flowchart of an image classification method according to an embodiment of the present invention;

[0016] Figure 6 This is a schematic diagram of the structure of an apparatus for obtaining a network model according to an embodiment of the present invention;

[0017] Figure 7This is a schematic diagram of the structure of another device for obtaining a network model according to an embodiment of the present invention;

[0018] Figure 8 This is a schematic diagram of the structure of another device for obtaining a network model according to an embodiment of the present invention;

[0019] Figure 9 This is a schematic diagram of the structure of another device for obtaining a network model according to an embodiment of the present invention;

[0020] Figure 10 This is a schematic diagram of the structure of another device for obtaining a network model according to an embodiment of the present invention;

[0021] Figure 11 This is a schematic diagram of the structure of another device for obtaining a network model according to an embodiment of the present invention;

[0022] Figure 12 This is a structural block diagram of another computer terminal according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] First, some nouns or terms that appear in the description of the embodiments of the present invention shall be interpreted as follows:

[0026] Dynamic inference networks are neural networks that use dynamic inference mechanisms to improve inference speed. In dynamic inference, certain computational operations can be skipped dynamically based on input samples according to specific strategies. Dynamic inference networks can balance accuracy and computational cost.

[0027] Dynamic routing structure: refers to a network structure that can automatically build corresponding routing tables and adjust accordingly based on actual changes in links and nodes.

[0028] Example 1

[0029] According to an embodiment of the present invention, a method embodiment for obtaining a network model is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] The method embodiment provided in Embodiment 1 of the present invention can be executed in a mobile terminal, computer terminal or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for acquiring a network model is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, a keyboard, a cursor control device (such as a mouse), an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0031] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of the present invention, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for obtaining a network model in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned method for obtaining a network model. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0033] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0034] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0035] It should be noted here that, in some optional embodiments, the above... Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1This is only one instance of a specific particular instance and is intended to illustrate the types of components that may exist in the aforementioned computer device (or mobile device).

[0036] Under the above operating environment, the present invention provides, as follows: Figure 2 This illustrates a method for obtaining a network model. Figure 2 This is a flowchart of a method for obtaining a network model according to an embodiment of the present invention, such as... Figure 2 As shown, the method for obtaining the network model includes:

[0037] Step S21: Obtain the target dataset, wherein the target dataset is an image classification dataset;

[0038] The target dataset mentioned above can be an image classification dataset used to train an image classification model (such as the dynamic prediction network model described below). The specific implementation method for obtaining the target dataset can be: pre-specified by technical personnel according to application scenario requirements, or updated in real-time based on test data involved in model training.

[0039] Step S22: Based on the first network model and the dynamic routing structure, construct a second network model. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection.

[0040] The aforementioned first network model can be an image classification network model. This image classification network model can be used to identify and classify images in fields such as power energy, industrial manufacturing, transportation and logistics, retail, finance, healthcare, and education. The neural network category corresponding to this image classification network model can be Convolutional Neural Networks (CNN), Recursive Neural Networks (RNN), and Generative Adversarial Networks (GAN), etc.

[0041] The aforementioned multiple network substructures included in the first network model can be multiple partial network structures obtained by dividing the first neural network model. For example, the first neural network can be divided according to partitioning criteria such as network layers and network blocks to obtain the aforementioned multiple network substructures.

[0042] The dynamic routing structure described above can be an additional network structure within the image classification network model. This dynamic routing structure can be placed before and associated with the aforementioned network substructures. It can be used to select whether to skip or execute at least one of the multiple network substructures. Skipping a network substructure indicates that the training path corresponding to the network model training process does not contain that substructure. Executing a network substructure indicates that the training path corresponding to the network model training process contains that substructure.

[0043] Based on the first network model and the dynamic routing structure described above, the second network model can be constructed. This second network model can be an initial dynamic inference network model based on path selection; that is, the network structure of this second network model is related to the selection result of skipping or executing at least one of the multiple network substructures chosen by the dynamic routing structure. This initial dynamic inference network model can perform dynamic inference during image classification, and it can be used (e.g., after network parameter optimization) to generate a target dynamic inference network model.

[0044] Specifically, the above-described construction of the second network model based on the first network model and dynamic routing structure also includes other methods and steps, which can be referred to in the further description of the embodiments of the present invention below, and will not be repeated here.

[0045] Step S23: Train the second network model using the target dataset to obtain the third network model, wherein the third network model is the target dynamic inference network model with optimized network parameters.

[0046] The third network model mentioned above can be a target dynamic reasoning network model. This target dynamic reasoning network model can be obtained by training the initial dynamic reasoning network model using the target dataset mentioned above.

[0047] The network parameters mentioned above can be the neural network parameters corresponding to the third network model. For example, when the third network model is a convolutional neural network structure, the network parameters can include: convolutional layer bias parameters, output layer weight parameters, output layer bias parameters, and output layer error parameters, etc.

[0048] The specific implementation process of training the second network model with the target dataset to obtain the third network model can be as follows: using the initial dynamic inference network model (equivalent to the second network model mentioned above), machine learning training is performed on the training data in the image classification dataset (equivalent to the target dataset mentioned above). During the machine learning training process, the network parameters are optimized to obtain the target dynamic inference network model (equivalent to the third network model mentioned above).

[0049] Specifically, the above-mentioned method of training the second network model with the target dataset to obtain the third network model also includes other methods and steps, which can be referred to in the further description of the embodiments of the present invention below, and will not be repeated here.

[0050] Step S24: Binarize the dynamic routing structure based on the third network model to obtain the fourth network model, where the fourth network model is a dynamic prediction network model, which is used to classify the target image to be classified.

[0051] The fourth network model mentioned above can be a dynamic prediction network model. This dynamic prediction network model can be used to classify target images. It can dynamically predict during the image classification process to flexibly adjust the machine learning training process.

[0052] In the process of binarizing the dynamic routing structure based on the third network model to obtain the fourth network model, the binarization process can be as follows: the selection result of at least one network substructure selected by the dynamic routing structure to skip or execute multiple network substructures is processed into a binarized result (e.g., 0 and 1). This binarized result can be used to represent skipping or executing the current network substructure (e.g., 0 represents skipping, 1 represents executing).

[0053] For example, binarizing the above dynamic routing structure can be done by converting the output result v of each dynamic routing structure of the third network model. n Binarization is performed to obtain the processed output result r. j Specifically, one method of binarization could be: when v j When <0.5, let r j =0; when v j When ≥0.5, let r j =1.

[0054] It should be noted that the specific implementation process of classifying the target image to be classified using the above-mentioned fourth network model can be referred to the further description of the embodiments of the present invention below, and will not be repeated here.

[0055] In this embodiment of the invention, a target dataset is first obtained, wherein the target dataset is an image classification dataset. A second network model is constructed based on a first network model and a dynamic routing structure. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select and skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The second network model is trained using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters. The dynamic routing structure is further binarized based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model. The fourth network model is used to classify the target image to be classified. This achieves the purpose of constructing a dynamic prediction network model based on the image classification network model and the dynamic routing structure to classify the target image, thereby improving the learning efficiency and flexibility of the network model and reducing the computational cost of image classification. This solves the technical problems of low learning efficiency, low flexibility, and high computational cost of network model acquisition methods in related technologies.

[0056] It is easy to understand that, compared with related technologies, the key point of the method provided by the embodiments of the present invention is: to provide a dynamic prediction network model, which can dynamically select the network path during the classification of the target image to be classified, so that the network model can be more fully utilized.

[0057] In summary, one of the beneficial effects of this invention is that, through a dynamic routing mechanism, during the image classification process, some or all of the network structures can be dynamically and selectively executed according to the characteristics of the image samples, which can improve the learning efficiency of the network model, reduce the computational cost of the algorithm, and effectively balance accuracy and computational cost.

[0058] The methods described in the embodiments of the present invention will be further described below.

[0059] In an optional embodiment, in step S22, a second network model is constructed based on the first network model and the dynamic routing structure, including the following method steps:

[0060] Step S221: Divide the first network model into multiple network substructures, using blocks as units;

[0061] Step S222: Add a dynamic routing structure to at least one of the multiple network substructures to construct a second network model. The output of the dynamic routing structure is determined by the output of the previous network substructure, and the output of the current network substructure with the dynamic routing structure is jointly determined by the output of the dynamic routing structure and the output of the previous network substructure.

[0062] The aforementioned blocks can be model blocks of a network model. A block can be a custom construct. Each block can describe a single network layer, a network component composed of multiple network layers, or the entire network model. In this embodiment of the invention, an example of a block describing a network component composed of multiple network layers will be used for illustration.

[0063] The specific implementation process of dividing the first network model into multiple network substructures based on blocks can be as follows: obtain a predefined unit Block; then divide the first network model into multiple network substructures according to the unit Block, wherein each of the multiple network substructures corresponds to a model Block (or a network component composed of multiple network layers).

[0064] The aforementioned dynamic routing structure can be a network structure pre-configured by technicians to implement the dynamic routing mechanism. This dynamic routing structure can be used to select to skip or execute at least one of the aforementioned network substructures.

[0065] Add a dynamic routing structure to at least one of the above network substructures. You can choose to skip or execute the network substructure based on the dynamic routing structure corresponding to the network substructure. Then, construct the second network model based on the selection result.

[0066] For example, in the second network model described above, taking two adjacent network substructures with dynamic routing structures as an example, the link connection relationship can be as follows: the output of the preceding dynamic routing structure can be used as an input to the preceding network substructure (here, the other input of the preceding network substructure can be the output of the previous network substructure of the preceding network substructure), the output of the preceding network substructure can be used as an input to the following dynamic routing structure, and the output of the following dynamic routing structure can be used as an input to the following network substructure (here, the other input of the following network substructure can be the output of the preceding network substructure).

[0067] In the second network model described above, the output of the dynamic routing structure can be determined by the output of the previous network substructure. That is, for any dynamic routing structure other than the first dynamic routing structure, its output can be determined by the output of the previous network substructure associated with it (which can be a network substructure with or without a dynamic routing structure); the output of the first dynamic routing structure can be determined by the intermediate features of the network model image corresponding to that first dynamic routing structure.

[0068] In the second network model described above, the output of the current network substructure with the dynamic routing structure is determined jointly by the output of the dynamic routing structure and the output of the previous network substructure. That is, for any network substructure (equivalent to the current network substructure) other than the first network substructure associated with the dynamic routing structure, the output of the current network substructure can be determined by the following two factors: the output of the dynamic routing structure associated with the current network substructure and the output of the previous network substructure. The output of the first network substructure (referring to the first network substructure after training begins) can be determined by the input of the first network substructure pre-specified by the technician.

[0069] For example, in the field of transportation and logistics, when obtaining a dynamic prediction network model M4 for image classification, the method provided by this invention can be used: obtain a convolutional neural network model M1 for image classification (equivalent to the first network model mentioned above); then, according to a pre-specified block, divide the model M1 into N network substructures, and denote these N network substructures as... Among them, F j (x j-1 ) represents the j-th network substructure, and the input x of the j-th network substructure. j-1 This is the output of the (j-1)th network substructure.

[0070] Taking the acquisition of a dynamic prediction network model M4 for image classification in the field of transportation and logistics as an example, the process of acquiring model M4 also includes: adding N dynamic routing structures before the N network substructures of model M1 to select whether to skip or execute the current network substructure. The network model M2 is obtained (equivalent to the second network model mentioned above).

[0071] Specifically, network substructure F j (x j-1 The corresponding dynamic routing structure R j The input is the network substructure F. j-1 (x j-2The output of ) and the dynamic routing structure R j The output r j It can be expressed as shown in the following formula (1):

[0072] r j =gumbel-softmax(fc(x) j-1 )) Formula (1)

[0073] In the above formula (1), fc represents the fully connected layer model, and gumbel-softmax represents a parameterized sampling method such that r j The value can be 0 or 1.

[0074] Specifically, the dynamic routing structure R j The corresponding j-th network substructure F j (x j-1 The output x j It can be expressed as shown in the following formula (2):

[0075] x j =r j F j (x j-1 )+(1-r j )x j-1 Formula (2)

[0076] It is easy to understand that the output of the j-th network substructure is determined by the output of the j-th dynamic routing structure and the output of the (j-1)-th network substructure.

[0077] In an optional embodiment, in step S23, the second network model is trained using the target dataset to obtain the third network model, including the following method steps:

[0078] Step S231: Divide the target dataset to obtain multiple batches of images;

[0079] Step S232: Input multiple batches of images into the second network model in batches for training to determine the direction of feature evolution;

[0080] Step S233: Optimize the feature evolution direction and determine the first loss;

[0081] Step S234: Adjust the network parameters of the second network model based at least on the first loss to obtain the third network model.

[0082] The target dataset mentioned above can be an image classification dataset used to train a dynamic prediction network model. The multiple batches of images mentioned above can be multiple batches of input images during the model training process.

[0083] The specific implementation process of dividing the target dataset to obtain multiple batches of images can be as follows: The image classification data contained in the target dataset is divided according to a pre-specified division strategy, thereby obtaining the aforementioned multiple batches of images. The batch number, batch size, etc., of the aforementioned multiple batches of images are related to the division strategy described above.

[0084] The aforementioned batches of images are input into a second network model for training. The machine learning process of this second network model can be dataflow learning. The aforementioned feature evolution direction can be the direction of feature space evolution of the aforementioned batches of images during the dataflow learning process of the second network model.

[0085] The aforementioned first loss can be the cosine similarity loss during the training process of the second network model. The specific implementation process for optimizing the feature evolution direction and determining the first loss can be found in the further description of the embodiments of the present invention below, and will not be repeated here.

[0086] The network parameters mentioned above can be the neural network parameters corresponding to the second network model. For example, when the second network model is a convolutional neural network structure, the network parameters can include: convolutional layer bias parameters, output layer weight parameters, output layer bias parameters, and output layer error parameters, etc.

[0087] By adjusting the network parameters of the second network model based on at least the first loss mentioned above, the third network model can be obtained. It is readily apparent that the prediction accuracy of the third network model can be further improved through this network parameter adjustment process.

[0088] Taking the acquisition of a dynamic prediction network model M4 for image classification in the transportation and logistics field as an example, the process of acquiring model M4 also includes: acquiring an image dataset (equivalent to the target dataset mentioned above) in the transportation and logistics field for training the target task. This image dataset includes image groups. Where I i Let represent the i-th image in the image group, and F represent the number of images in the image group; then, the image dataset is divided into multiple batches, each containing L images.

[0089] Furthermore, the image datasets from the aforementioned multiple batches are input into the aforementioned network model M2 in batches for training, thereby obtaining network model M3 (equivalent to the aforementioned third network model).

[0090] In an optional embodiment, in step S232, multiple batches of images are input into the second network model in batches for training to determine the feature evolution direction, including the following method steps:

[0091] Step S2321: Input multiple batches of images into the second network model for training, align the features before each downsampling layer in the second network model, and obtain multiple aligned features;

[0092] Step S2322: Calculate multiple evolution angles using multiple alignment features;

[0093] Step S2323: Determine the feature evolution direction through multiple evolution angles.

[0094] The second network model described above may include multiple downsampling layers. These downsampling layers can be used to adjust the image size to fit the display area size, and can also be used to generate thumbnails of the image.

[0095] The features before each downsampling layer in the second network model described above can be: the image features of the images before they pass through the downsampling layer during the training process of the multiple batches of images input into the second network model.

[0096] The aforementioned batches of images are input into the second network model for training in batches. The features before each downsampling layer in the second network model are aligned, resulting in multiple aligned features. These aligned features can be used to calculate the aforementioned multiple evolution angles, which are then used to determine the direction of feature evolution. These multiple evolution angles can be the angles between the trajectories of the feature space evolution of the aforementioned batches of images during the dataflow learning process of the second network model.

[0097] Taking the acquisition of a dynamic prediction network model M4 for image classification in the field of transportation and logistics as an example, the process of acquiring this model M4 also includes: during the training process of inputting the above-mentioned multiple batches of image datasets into the above-mentioned network model M2 in batches, for any input image, the image features (denoted as {x} before each downsampling layer (or downsampling layer) in the network model M2 are... l1 x l2 x l3 ...x lK Alignment is performed, and the feature evolution angles corresponding to the aligned features can be calculated, as shown in formulas (3) to (5) below:

[0098]

[0099]

[0100] …

[0101]

[0102] In the above formulas (3) to (5), Indicates feature x l1and feature x l2 The corresponding feature evolution angle, Indicates feature x l2 and feature x l3 The corresponding feature evolution angle, and so on; a1 represents feature x l1 The corresponding alignment network layer, a2 represents feature x l2 The corresponding alignment network layer, and so on, can be further divided into a pooling layer and a multilayer perceptron (MLP).

[0103] It should be noted here that, in the above feature {x l1 x l2 x l3 ...x lK In the table, the subscript 'l' represents the index of the downsampling layer in the network. For example, in the ResNet 110 residual network, feature x... l1 The subscript l1 is 18, and the feature x l2 The subscript l2 is 36, and the feature x l3 The subscript l3 is 53.

[0104] Figure 3 This is a schematic diagram of an optional image feature evolution according to an embodiment of the present invention, such as... Figure 3 As shown, in the network structure of network model M2, image features x lK (Equivalent to the input features of the network structure) evolves to x l(K+1) (equivalent to the characteristics of the middle layer of a network structure) further evolves to x l(K+2) In the process of (equivalent to the output features of the network structure), two evolution angles (such as...) can be involved. Figure 3 (Angles 1 and 2 shown in the figure). In this embodiment of the invention, the evolution angle can be represented by a vector determined by two features, and the larger the evolution angle, the greater the intensity of feature evolution.

[0105] It is easy to understand that, through the aforementioned multiple evolutionary angles... The direction of feature evolution corresponding to the above network model M2 can be determined.

[0106] In an optional embodiment, step S233 involves optimizing the feature evolution direction to determine a first loss, including the following steps:

[0107] Step S2331: Obtain the cosine similarity of each pair of adjacent evolution angles among multiple evolution angles;

[0108] Step S2332: By constraining the cosine similarity, the feature evolution direction is optimized to be the same, so as to determine the first loss.

[0109] The cosine similarity between the two evolution angles can be estimated by taking the cosine of the angle between the two feature evolution directions corresponding to the two evolution angles.

[0110] By constraining the cosine similarity, the feature evolution direction corresponding to the second network model can be optimized, thereby calculating the first loss. Specifically, the feature evolution direction corresponding to the second network model can be adjusted to the same direction.

[0111] Taking the acquisition of a dynamic prediction network model M4 for image classification in the field of transportation and logistics as an example, the process of acquiring model M4 also includes: calculating the cosine similarity between every two adjacent evolution angles among the above multiple evolution angles, and calculating the first loss L based on the cosine similarity. evo The calculation method can be shown in the following formula (6):

[0112]

[0113] In the above formula (6), Indicates the evolution angle and evolution angle The cosine similarity.

[0114] Furthermore, based on the aforementioned first loss L evo The network parameters of network model M2 (such as convolutional layer bias parameters, output layer weight parameters, output layer bias parameters, and output layer error parameters) can be adjusted to obtain the aforementioned network model M3.

[0115] In one optional embodiment, the method for obtaining the network model further includes the following steps:

[0116] Step S25: Based on the second network model, construct a fifth network model, wherein the network structure of the second network model is the same as that of the fifth network model, and the network parameters of the fifth network model are updated synchronously with the network parameters of the second network model.

[0117] The fifth network model described above can be a dynamic auxiliary model. The network structure of this fifth network model is the same as that of the second network model described above. When the second network model is a convolutional neural network model, the network parameters of this fifth network model may include: convolutional layer bias parameters, output layer weight parameters, output layer bias parameters, and output layer error parameters, etc.

[0118] Taking the acquisition of a dynamic prediction network model M4 for image classification in the field of transportation and logistics as an example, the process of acquiring this model M4 also includes: for the above network model M2, a dynamic auxiliary model M5 (equivalent to the above fifth network model) with the same structure as the network model M2 can be created. In particular, the network parameters of the dynamic auxiliary model M5 and the network parameters of the above network model M2 can be updated synchronously with momentum. The synchronous momentum update can be performed as shown in the following formula (7):

[0119] θ t =q×θ t +(1-q)×θ d Formula (7)

[0120] In the above formula (7), θ t θ represents the network parameters of the dynamic auxiliary model M5 described above. d Let represent the network parameters of the network model M2 above, and q represent the momentum coefficient corresponding to the momentum update.

[0121] Figure 4 This is a schematic diagram of the structure of an optional network model according to an embodiment of the present invention, such as... Figure 4 As shown, the dynamic inference network is equivalent to the network model M2 mentioned above. The dynamic auxiliary model (equivalent to the dynamic auxiliary model M5 mentioned above) established for the dynamic inference network has the same network structure as the dynamic inference network, and the network parameters of the dynamic auxiliary model are updated synchronously with the network parameters of the dynamic inference network.

[0122] Still as Figure 4 As shown, the first loss mentioned above (e.g., in the feature evolution direction of the dynamic inference network) can be calculated. Figure 4 The loss 1) in the process is used to optimize the dynamic inference network.

[0123] In one optional embodiment, the method for obtaining the network model further includes the following steps:

[0124] Step S261: Insert multiple classifiers into each downsampling layer in the second network model, and insert multiple classifiers into each downsampling layer in the fifth network model;

[0125] Step S262: Multiple classifiers are used to classify the features before each downsampling layer in the second network model to obtain the first classification result; and multiple classifiers are used to classify the features before each downsampling layer in the fifth network model to obtain the second classification result.

[0126] Step S263: Determine the second loss based on the first classification result and the second classification result.

[0127] The second network model described above can be an initial dynamic inference network model based on path selection. This second network model may include multiple downsampling layers. In each downsampling layer of this second network model, multiple classifiers are inserted, enabling the classification of image features preceding the multiple downsampling layers.

[0128] The fifth network model described above can be a dynamic auxiliary model. Similar in structure to the second network model, this fifth network model can also include multiple downsampling layers. In this fifth network model, multiple classifiers are inserted into each sampling layer, enabling the classification of image features preceding the multiple downsampling layers of the second network model.

[0129] The features before each sampling layer in the second network model are classified using the aforementioned multiple classifiers to obtain a first classification result. Similarly, the features before each sampling layer in the fifth network model are classified using the same multiple classifiers to obtain a second classification result. These first and second classification results can be used to constrain the dynamic auxiliary model to calculate the loss function.

[0130] The second loss is determined based on the first classification result and the second classification result described above. This second loss can be the regularization term (or regularization loss) of the loss function.

[0131] Taking the dynamic prediction network model M4 for image classification in the transportation and logistics field as an example, auxiliary classifiers are added before each downsampling layer of the aforementioned network model M2 and dynamic auxiliary model M5. That is, K auxiliary classifiers, denoted as c1, c2, c3...c, are inserted into network model M2 and dynamic auxiliary model M5 respectively. K .

[0132] Furthermore, the image features {x} prior to each of the above downsampling layers are... l1 x l2 x l3 ...x lK The image features x are input into the corresponding auxiliary classifier for classification. Specifically, the image features x are... l1 Input the auxiliary classifier c1, and process the image features x l2 Input the auxiliary classifier c2, and so on.

[0133] Furthermore, the classification results of the above K auxiliary classifiers can be represented by the following formula (8):

[0134] y1, y2, y3...y K =c1(x l1 c2(x) l2 c3(x) l3 ...c K (xlK ) Formula (8)

[0135] It should be noted here that the classification results of the K auxiliary classifiers in the above network model M2 (equivalent to the first classification result) can be denoted as... The classification results of the K auxiliary classifiers in the dynamic auxiliary model M5 above (equivalent to the second classification result above) are denoted as:

[0136] Furthermore, based on the above classification results and classification results The regularization loss L can be calculated using the following formula (9). logits (Equivalent to the second loss mentioned above):

[0137]

[0138] In the above formula (9), KL(|·|) represents the Kullback-Leibler distance, and K represents the total number of auxiliary classifiers.

[0139] Still as Figure 4 As shown, the second loss (e.g., the classification result corresponding to the classifier in the dynamic inference network (equivalent to the first classification result mentioned above) and the classification result corresponding to the classifier in the dynamic auxiliary model (equivalent to the second classification result mentioned above) can be calculated through a fully connected layer. Figure 4 The loss 2) in the model is used to optimize the dynamic inference network. In an optional embodiment, the method for obtaining the network model further includes the following steps:

[0140] Step S271: Select sample images from the target dataset;

[0141] Step S272: Input the sample image into the second network model to establish the first selection path, and input the sample image into the fifth network model to establish the second selection path;

[0142] Step S273: Determine the third loss based on the first selected path and the second selected path.

[0143] The target dataset mentioned above can be an image classification dataset used to train an image classification model (equivalent to the dynamic prediction network model mentioned above). The sample images selected from this target image set can be any image used to determine the third loss mentioned above.

[0144] By inputting the aforementioned sample images into the second network model, the first selection path can be established. By inputting the aforementioned sample images into the fifth network model, a second selection path can be established. Based on the first and second selection paths, the third loss can be determined. This third loss can be a path loss.

[0145] Taking the dynamic prediction network model M4 for image classification in the field of transportation and logistics as an example, from the image dataset... Select any sample image from the given list; input this sample image into the network model M2 described above, and you can obtain the selection path of the network model M2 (equivalent to the first selection path mentioned above, denoted as...). Inputting the sample image into the dynamic auxiliary model M5 above yields the selection path of the dynamic auxiliary model M5 (equivalent to the second selection path mentioned above, denoted as...). ), where N represents the number of dynamic routing structures in network model M2 (the network structure is the same as that of dynamic auxiliary model M5), r i This represents the output of the i-th dynamic routing structure.

[0146] Furthermore, based on the above path selection... and select path The third loss L can be obtained by distillation calculation according to the following formula (10). path :

[0147]

[0148] In the above formula (10), This represents the execution decision corresponding to the nth dynamic routing structure in network model M2. This represents the execution decision corresponding to the nth dynamic routing structure in the dynamic auxiliary model M5.

[0149] Still as Figure 4 As shown, the dynamic selection path (equivalent to the first selection path) established by the dynamic inference network (equivalent to the network model M2 above) and the dynamic selection path (equivalent to the second selection path above) established by the dynamic auxiliary model (equivalent to the dynamic auxiliary model M5 above) can calculate the third loss mentioned above (such as...). Figure 4 Loss 3) in the process is used to optimize the dynamic inference network.

[0150] It is readily understood that, compared with related technologies, the key point of the method provided by the above optional embodiments of the present invention is: to provide a dynamic auxiliary model (equivalent to the above fifth network model) to further optimize the dynamic prediction network model, and to achieve higher performance with lower computational cost when the network model structure is the same.

[0151] According to the above optional embodiments, one of the beneficial effects of the method for obtaining network models provided by the present invention is that: through the dynamic auxiliary model, auxiliary constraints can be applied to the execution mode of different unit blocks corresponding to the target image to be classified; in addition, by modeling the feature evolution process of each target image to be classified, the image sample learning path of the dynamic prediction network model can be further optimized, thereby effectively improving the accuracy of the dynamic prediction network model and the interpretability of the learning path.

[0152] In one optional embodiment, the method for obtaining the network model further includes the following steps:

[0153] Step S28: Determine the fourth loss based on the first classification result and the preset label.

[0154] The first classification result mentioned above can be obtained by classifying the features before each downsampling layer in the second network model using the aforementioned multiple classifiers. The preset labels mentioned above can be image labels pre-defined by technicians according to the application scenario requirements. Based on the first classification result and the preset labels, the fourth loss mentioned above can be determined. The fourth loss can be the cross-entropy loss.

[0155] Taking the dynamic prediction network model M4 for image classification in the field of transportation and logistics as an example, we obtain a label Y pre-set by technicians, and use this label Y to evaluate the classification results of the K auxiliary classifiers in the aforementioned network model M2. By applying cross-entropy constraints, the fourth loss L is obtained. cls The specific calculation process can be shown in the following formula (11):

[0156]

[0157] In the above formula (11), CE(|·|) represents the above cross-entropy constraint.

[0158] Still as Figure 4 As shown, the fourth loss (e.g., the classification result corresponding to the classifier in the dynamic inference network (equivalent to the first classification result) and the labels preset by the technician can be calculated. Figure 4 Loss 4) in the process is used to optimize the dynamic reasoning network.

[0159] In an optional embodiment, in step S234, the network parameters of the second network model are adjusted based at least on the first loss to obtain a third network model, including the following method steps:

[0160] Step S2341: Determine the target loss using the first loss, second loss, third loss, and fourth loss;

[0161] Step S2342: Adjust the network parameters of the second network model based on the target loss to obtain the third network model.

[0162] Using the first loss, second loss, third loss, and fourth loss described above, the target loss can be determined. This target loss can be used to adjust the network parameters of the initial dynamic inference network model based on path selection (equivalent to the second network model described above) to obtain the third network model described above.

[0163] Taking the acquisition of a dynamic prediction network model M4 for image classification in the field of transportation and logistics as an example, the process of acquiring this model M4 also includes: based on the first loss L mentioned above... evo The aforementioned second loss L logits The aforementioned third loss L path And the fourth loss L mentioned above cls The target loss L can be obtained.

[0164] Furthermore, by using the aforementioned target loss L to optimize and adjust the network parameters of the aforementioned network model M2, the aforementioned network parameters M3 can be obtained.

[0165] In one optional embodiment, the method for obtaining the network model further includes the following steps:

[0166] Step S291: Obtain the target image to be classified;

[0167] Step S292: Use the fourth network model to perform classification prediction on the target image to be classified, and obtain the classification result.

[0168] The target images to be classified can be images from fields such as power energy, industrial manufacturing, transportation and logistics, retail, finance, healthcare, and education. The specific methods for obtaining the target images to be classified can be: input by technicians according to the classification requirements of the application scenario, or real-time acquisition from a database (which can be a cloud database) or a server (which can be a cloud server).

[0169] The fourth network model mentioned above can be a dynamic prediction network model. By using this fourth network model to perform classification prediction on the target image to be classified, a classification result can be obtained.

[0170] It is easy to understand that the method provided by the above optional embodiments can obtain a dynamic prediction network model through a dynamic routing mechanism. The process of image classification using this dynamic prediction network model has low computational cost and the classification result has high accuracy.

[0171] Furthermore, in one possible application scenario, based on two widely used open-source datasets, Cifar10 / 100 and ImageNet, the method provided in this embodiment of the invention can be used: the target network model has a convolutional neural network structure, and a Residual Network (ResNet110) is used on the Cifar10 / 100 dataset, while a ResNet50 is used on the ImageNet dataset. Moreover, the loss function is optimized using stochastic gradient descent. Based on the application of the above method, comparing the prediction accuracy and computational cost of the target network model with those of the basic ResNet reveals that the method provided in this embodiment of the invention effectively improves the accuracy of the network model and flexibly adjusts the computational cost of the network model.

[0172] It should be noted that the method for obtaining a network model provided in the embodiments of the present invention, especially the improvement of the dynamic routing structure in the method for obtaining a network model, can be directly applied to existing network models, so that the network model can improve efficiency and reduce consumption without losing model accuracy.

[0173] It should be noted that, compared with the dynamic network learning methods in related technologies, the method for obtaining network models provided in this embodiment of the invention can significantly improve the running speed of network models based on models such as Residual Networks (ResNet) and Dense Networks (DenseNet), and has good application value.

[0174] Under the above operating environment, the present invention provides, as follows: Figure 5 This illustrates an image classification method. Figure 5 This is a flowchart of an image classification method according to an embodiment of the present invention, such as... Figure 5 As shown, this image classification method includes:

[0175] Step S51: Obtain the target image to be classified;

[0176] Step S52: Use a dynamic prediction network model to classify and predict the target image to be classified, and obtain the classification result;

[0177] The dynamic prediction network model is determined by the target dataset, the first network model, and the dynamic routing structure. The target dataset is an image classification dataset. The first network model and the dynamic routing structure are used to construct the second network model. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The target dataset is used to train the second network model to obtain the third network model. The third network model is the target dynamic inference network model with optimized network parameters. The third network model is used to binarize the dynamic routing structure to obtain the dynamic prediction network model.

[0178] The target images to be classified can be images from fields such as power energy, industrial manufacturing, transportation and logistics, retail, finance, healthcare, and education. The specific methods for obtaining the target images to be classified can be: input by technicians according to the classification requirements of the application scenario, or real-time acquisition from a database (which can be a cloud database) or a server (which can be a cloud server).

[0179] The aforementioned dynamic prediction network model for classifying target images can be obtained by employing part or all of the methods provided in steps S21 to S24 of the embodiments of the present invention, as well as the methods provided in the various possible embodiments corresponding to that method. Therefore, in various application scenarios, using this dynamic prediction network model to classify target images can, through a dynamic routing mechanism, dynamically and selectively execute part or all of the network structure according to the characteristics of the image samples, thereby improving the learning efficiency of the network model, reducing the computational cost of the algorithm, and effectively balancing accuracy and computational cost.

[0180] In addition, the above image classification method may involve other technical details, which can be found in the relevant descriptions in the embodiments of the above method for obtaining network models, and will not be repeated here.

[0181] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0182] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0183] Example 2

[0184] According to embodiments of the present invention, an apparatus for implementing the above-described method for obtaining a network model is also provided. Figure 6 This is a schematic diagram of the structure of an apparatus for obtaining a network model according to an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes: an acquisition module 61, a first construction module 62, a training module 63, and a processing module 64, wherein,

[0185] The acquisition module 61 is used to acquire the target dataset, wherein the target dataset is an image classification dataset; the first construction module 62 is used to construct a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model includes: multiple network substructures, the dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures, and the second network model is an initial dynamic inference network model based on path selection; the training module 63 is used to train the second network model using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; the processing module 64 is used to perform binarization processing on the dynamic routing structure based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model, and the fourth network model is used to classify the target image to be classified.

[0186] Optionally, the first building module 62 is further configured to: divide the first network model into multiple network substructures in blocks; add a dynamic routing structure to at least one of the multiple network substructures to build a second network model, wherein the output of the dynamic routing structure is determined by the output of the previous network substructure, and the output of the current network substructure with the dynamic routing structure is jointly determined by the output of the dynamic routing structure and the output of the previous network substructure.

[0187] Optionally, the training module 63 is further configured to: divide the target dataset to obtain multiple batches of images; input the multiple batches of images into the second network model for training in batches to determine the feature evolution direction; optimize the feature evolution direction to determine the first loss; and adjust the network parameters of the second network model based at least on the first loss to obtain the third network model.

[0188] Optionally, the training module 63 is further configured to: input multiple batches of images into the second network model for training, align the features before each downsampling layer in the second network model to obtain multiple aligned features; calculate multiple evolution angles using the multiple aligned features; and determine the feature evolution direction through the multiple evolution angles.

[0189] Optionally, the training module 63 is further configured to: obtain the cosine similarity of each pair of adjacent evolution angles among multiple evolution angles; optimize the feature evolution direction to the same direction by constraining the cosine similarity, so as to determine the first loss.

[0190] Optionally, Figure 7 This is a schematic diagram of another device for obtaining a network model according to an embodiment of the present invention, such as... Figure 7 As shown, the device includes, in addition to Figure 6 In addition to all the modules shown, it also includes: a second construction module 65, used to construct a fifth network model based on the second network model, wherein the network structure of the second network model is the same as that of the fifth network model, and the network parameters of the fifth network model are updated synchronously with the network parameters of the second network model.

[0191] Optionally, Figure 8 This is a schematic diagram of another device for obtaining a network model according to an embodiment of the present invention, such as... Figure 8 As shown, the device includes, in addition to Figure 7 In addition to all the modules shown, it also includes: a first classification module 66, used to insert multiple classifiers into each downsampling layer in the second network model and into each downsampling layer in the fifth network model; to classify the features before each downsampling layer in the second network model using multiple classifiers to obtain a first classification result, and to classify the features before each downsampling layer in the fifth network model using multiple classifiers to obtain a second classification result; and to determine a second loss based on the first classification result and the second classification result.

[0192] Optionally, Figure 9 This is a schematic diagram of another device for obtaining a network model according to an embodiment of the present invention, such as... Figure 9 As shown, the device includes, in addition to Figure 8In addition to all the modules shown, it also includes: a first loss module 67, used to select sample images from the target dataset; input the sample images into a second network model to establish a first selection path, and input the sample images into a fifth network model to establish a second selection path; and determine a third loss based on the first and second selection paths.

[0193] Optionally, Figure 10 This is a schematic diagram of another device for obtaining a network model according to an embodiment of the present invention, such as... Figure 10 As shown, the device includes, in addition to Figure 9 In addition to all the modules shown, it also includes: a second loss module 68, used to determine a fourth loss based on the first classification result and a preset label.

[0194] Optionally, the training module 63 is further configured to: determine the target loss using the first loss, the second loss, the third loss, and the fourth loss; and adjust the network parameters of the second network model based on the target loss to obtain the third network model.

[0195] Optionally, Figure 11 This is a schematic diagram of another device for obtaining a network model according to an embodiment of the present invention, such as... Figure 11 As shown, the device includes, in addition to Figure 10 In addition to all the modules shown, it also includes: a second classification module 69, used to acquire the target image to be classified; and a fourth network model is used to perform classification prediction on the target image to be classified to obtain the classification result.

[0196] It should be noted that the acquisition module 61, the first construction module 62, the training module 63, and the processing module 64 mentioned above correspond to steps S21 to S24 in Embodiment 1. The four modules and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the device, can run on the computer terminal 10 provided in Embodiment 1.

[0197] In this embodiment of the invention, a target dataset is first obtained, wherein the target dataset is an image classification dataset. A second network model is constructed based on a first network model and a dynamic routing structure. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select and skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The second network model is trained using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters. The dynamic routing structure is further binarized based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model. The fourth network model is used to classify the target image to be classified. This achieves the purpose of constructing a dynamic prediction network model based on the image classification network model and the dynamic routing structure to classify the target image, thereby improving the learning efficiency and flexibility of the network model and reducing the computational cost of image classification. This solves the technical problems of low learning efficiency, low flexibility, and high computational cost of network model acquisition methods in related technologies.

[0198] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0199] Example 3

[0200] According to an embodiment of the present invention, an embodiment of an electronic device is also provided, which may be any computing device in a group of computing devices. The electronic device includes: a processor and a memory, wherein:

[0201] A memory, connected to the processor, is used to provide the processor with instructions for processing the following steps: acquiring a target dataset, wherein the target dataset is an image classification dataset; constructing a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model including: multiple network substructures, the dynamic routing structure being used to select to skip or execute at least one of the multiple network substructures, and the second network model being an initial dynamic inference network model based on path selection; training the second network model using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; and binarizing the dynamic routing structure based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model, and the fourth network model is used to classify the target image to be classified.

[0202] In this embodiment of the invention, a target dataset is first obtained, wherein the target dataset is an image classification dataset. A second network model is constructed based on a first network model and a dynamic routing structure. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select and skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The second network model is trained using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters. The dynamic routing structure is further binarized based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model. The fourth network model is used to classify the target image to be classified. This achieves the purpose of constructing a dynamic prediction network model based on the image classification network model and the dynamic routing structure to classify the target image, thereby improving the learning efficiency and flexibility of the network model and reducing the computational cost of image classification. This solves the technical problems of low learning efficiency, low flexibility, and high computational cost of network model acquisition methods in related technologies.

[0203] It should be noted that the preferred implementation of this embodiment can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0204] Example 4

[0205] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0206] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0207] In this embodiment, the computer terminal described above can execute the program code for the following steps in the method for obtaining a network model: obtaining a target dataset, wherein the target dataset is an image classification dataset; constructing a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model including: multiple network substructures, the dynamic routing structure being used to select to skip or execute at least one of the multiple network substructures, and the second network model being an initial dynamic inference network model based on path selection; training the second network model using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; and binarizing the dynamic routing structure based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model, and the fourth network model is used to classify the target image to be classified.

[0208] Optionally, Figure 12 This is a structural block diagram of another computer terminal according to an embodiment of the present invention, such as... Figure 12 As shown, the computer terminal may include one or more (only one is shown in the figure) processors 122, memory 124, and peripheral interfaces 126.

[0209] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and apparatus for obtaining a network model in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned method for obtaining a network model. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to a computer terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0210] The processor can invoke information and applications stored in the memory via a transmission device to perform the following steps: acquiring a target dataset, wherein the target dataset is an image classification dataset; constructing a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model including: multiple network substructures, the dynamic routing structure being used to select to skip or execute at least one of the multiple network substructures, and the second network model being an initial dynamic inference network model based on path selection; training the second network model using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; and binarizing the dynamic routing structure based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model, and the fourth network model is used to classify the target image to be classified.

[0211] Optionally, the processor may also execute program code that performs the following steps: dividing the first network model into multiple network substructures in blocks; adding a dynamic routing structure to at least one of the multiple network substructures to construct a second network model, wherein the output of the dynamic routing structure is determined by the output of the previous network substructure, and the output of the current network substructure with the dynamic routing structure is jointly determined by the output of the dynamic routing structure and the output of the previous network substructure.

[0212] Optionally, the processor may also execute program code that performs the following steps: dividing the target dataset to obtain multiple batches of images; inputting the multiple batches of images into the second network model for training in batches to determine the feature evolution direction; optimizing the feature evolution direction to determine the first loss; and adjusting the network parameters of the second network model based at least on the first loss to obtain the third network model.

[0213] Optionally, the processor may also execute program code that performs the following steps: inputting multiple batches of images into the second network model for training, aligning the features before each downsampling layer in the second network model to obtain multiple aligned features; calculating multiple evolution angles using the multiple aligned features; and determining the feature evolution direction through the multiple evolution angles.

[0214] Optionally, the processor may also execute program code that performs the following steps: obtaining the cosine similarity of every two adjacent evolution angles among multiple evolution angles; optimizing the feature evolution direction to the same direction by constraining the cosine similarity, so as to determine the first loss.

[0215] Optionally, the processor may also execute program code that performs the following steps: constructing a fifth network model based on the second network model, wherein the network structure of the second network model is the same as that of the fifth network model, and the network parameters of the fifth network model are updated synchronously with the network parameters of the second network model.

[0216] Optionally, the processor may also execute program code that performs the following steps: inserting multiple classifiers into each downsampling layer in the second network model and inserting multiple classifiers into each downsampling layer in the fifth network model; classifying the features before each downsampling layer in the second network model using multiple classifiers to obtain a first classification result, and classifying the features before each downsampling layer in the fifth network model using multiple classifiers to obtain a second classification result; and determining a second loss based on the first classification result and the second classification result.

[0217] Optionally, the processor may also execute program code that performs the following steps: selecting sample images from the target dataset; inputting the sample images into a second network model to establish a first selection path, and inputting the sample images into a fifth network model to establish a second selection path; and determining a third loss based on the first and second selection paths.

[0218] Optionally, the processor may also execute program code that performs the following steps: determining a fourth loss based on the first classification result and a preset label.

[0219] Optionally, the processor may also execute program code that performs the following steps: determining the target loss using the first loss, the second loss, the third loss, and the fourth loss; and adjusting the network parameters of the second network model based on the target loss to obtain the third network model.

[0220] Optionally, the processor may also execute program code that performs the following steps: acquiring the target image to be classified; using a fourth network model to perform classification prediction on the target image to be classified, and obtaining the classification result.

[0221] The processor can access information and applications stored in memory via a transmission device to perform the following steps: acquiring a target image to be classified; using a dynamic prediction network model to predict the classification of the target image to be classified, and obtaining a classification result; wherein, the dynamic prediction network model is determined by a target dataset, a first network model, and a dynamic routing structure, the target dataset is an image classification dataset, the first network model and the dynamic routing structure are used to construct a second network model, the first network model is an image classification network model, the first network model includes: multiple network substructures, the dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures, the second network model is an initial dynamic inference network model based on path selection, the target dataset is used to train the second network model to obtain a third network model, the third network model is a target dynamic inference network model with optimized network parameters, and the third network model is used to binarize the dynamic routing structure to obtain a dynamic prediction network model.

[0222] In this embodiment of the invention, a target dataset is first obtained, wherein the target dataset is an image classification dataset. A second network model is constructed based on a first network model and a dynamic routing structure. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select and skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The second network model is trained using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters. The dynamic routing structure is further binarized based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model. The fourth network model is used to classify the target image to be classified. This achieves the purpose of constructing a dynamic prediction network model based on the image classification network model and the dynamic routing structure to classify the target image, thereby improving the learning efficiency and flexibility of the network model and reducing the computational cost of image classification. This solves the technical problems of low learning efficiency, low flexibility, and high computational cost of network model acquisition methods in related technologies.

[0223] Those skilled in the art will understand that Figure 12 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices. Figure 12 This does not limit the structure of the aforementioned electronic devices. For example, a computer terminal may also include components that are more... Figure 12The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 12 The different configurations shown.

[0224] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0225] According to an embodiment of the present invention, an embodiment of a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the method for obtaining a network model provided in Embodiment 1.

[0226] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0227] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining a target dataset, wherein the target dataset is an image classification dataset; constructing a second network model based on a first network model and a dynamic routing structure, wherein the first network model is an image classification network model, the first network model including: multiple network substructures, the dynamic routing structure being used to select to skip or execute at least one of the multiple network substructures, and the second network model being an initial dynamic inference network model based on path selection; training the second network model using the target dataset to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; and binarizing the dynamic routing structure based on the third network model to obtain a fourth network model, wherein the fourth network model is a dynamic prediction network model, and the fourth network model is used to classify the target image to be classified.

[0228] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: dividing the first network model into multiple network substructures in blocks; adding a dynamic routing structure to at least one of the multiple network substructures to construct a second network model, wherein the output of the dynamic routing structure is determined by the output of the previous network substructure, and the output of the current network substructure with the dynamic routing structure is jointly determined by the output of the dynamic routing structure and the output of the previous network substructure.

[0229] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: dividing the target dataset to obtain multiple batches of images; inputting the multiple batches of images into a second network model for training in batches to determine the feature evolution direction; optimizing the feature evolution direction to determine a first loss; and adjusting the network parameters of the second network model based at least on the first loss to obtain a third network model.

[0230] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: inputting multiple batches of images into the second network model for training in batches, aligning the features before each downsampling layer in the second network model to obtain multiple aligned features; calculating multiple evolution angles using the multiple aligned features; and determining the feature evolution direction through the multiple evolution angles.

[0231] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining the cosine similarity of every two adjacent evolution angles among multiple evolution angles; optimizing the feature evolution direction to the same direction by constraining the cosine similarity, so as to determine the first loss.

[0232] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: constructing a fifth network model based on the second network model, wherein the network structure of the second network model is the same as that of the fifth network model, and the network parameters of the fifth network model are updated synchronously with the network parameters of the second network model.

[0233] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: inserting multiple classifiers into each downsampling layer in the second network model and inserting multiple classifiers into each downsampling layer in the fifth network model; classifying the features before each downsampling layer in the second network model using multiple classifiers to obtain a first classification result, and classifying the features before each downsampling layer in the fifth network model using multiple classifiers to obtain a second classification result; and determining a second loss based on the first classification result and the second classification result.

[0234] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: selecting sample images from the target dataset; inputting the sample images into a second network model to establish a first selection path, and inputting the sample images into a fifth network model to establish a second selection path; and determining a third loss based on the first and second selection paths.

[0235] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining a fourth loss based on the first classification result and a preset label.

[0236] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining a target loss using a first loss, a second loss, a third loss, and a fourth loss; and adjusting the network parameters of a second network model based on the target loss to obtain a third network model.

[0237] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring the target image to be classified; using a fourth network model to perform classification prediction on the target image to be classified, and obtaining the classification result.

[0238] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a target image to be classified; using a dynamic prediction network model to perform classification prediction on the target image to be classified, and obtaining a classification result; wherein, the dynamic prediction network model is determined by a target dataset, a first network model, and a dynamic routing structure, the target dataset is an image classification dataset, the first network model and the dynamic routing structure are used to construct a second network model, the first network model is an image classification network model, the first network model includes: multiple network substructures, the dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures, the second network model is an initial dynamic inference network model based on path selection, the target dataset is used to train the second network model to obtain a third network model, the third network model is a target dynamic inference network model with optimized network parameters, and the third network model is used to binarize the dynamic routing structure to obtain a dynamic prediction network model.

[0239] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0240] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0241] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0242] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0243] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0244] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0245] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for obtaining a network model, characterized in that, include: Obtain the target dataset, wherein the target dataset is an image classification dataset; Based on the first network model and the dynamic routing structure, a second network model is constructed. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The target dataset is divided into multiple batches of images; The multiple batches of images are input into the second network model in batches for training, and the features before each downsampling layer in the second network model are aligned to obtain multiple aligned features; Multiple evolution angles are calculated using the aforementioned alignment features; The direction of feature evolution is determined by the multiple evolution angles; The evolution direction of the aforementioned features is optimized to determine the first loss; The network parameters of the second network model are adjusted based on at least the first loss to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; The dynamic routing structure is binarized based on the third network model to obtain a fourth network model, which is a dynamic prediction network model used to classify the target image to be classified.

2. The method according to claim 1, characterized in that, Based on the first network model and the dynamic routing structure, constructing the second network model includes: The first network model is divided into multiple network substructures, with each substructure being a block. The dynamic routing structure is added to at least one of the plurality of network substructures to construct the second network model, wherein the output of the dynamic routing structure is determined by the output of the previous network substructure, and the output of the current network substructure with the dynamic routing structure is jointly determined by the output of the dynamic routing structure and the output of the previous network substructure.

3. The method according to claim 1, characterized in that, The feature evolution direction is optimized to determine the first loss, which includes: Obtain the cosine similarity between any two adjacent evolution angles among the multiple evolution angles; The first loss is determined by constraining the cosine similarity to optimize the feature evolution direction to the same direction.

4. The method according to claim 1, characterized in that, The method further includes: Based on the second network model, a fifth network model is constructed, wherein the network structure of the second network model is the same as that of the fifth network model, and the network parameters of the fifth network model are updated synchronously with the network parameters of the second network model.

5. The method according to claim 4, characterized in that, The method further includes: In the second network model, multiple classifiers are inserted into each downsampling layer, and in the fifth network model, the multiple classifiers are inserted into each downsampling layer; The features before each downsampling layer in the second network model are classified using the multiple classifiers to obtain a first classification result, and the features before each downsampling layer in the fifth network model are classified using the multiple classifiers to obtain a second classification result; The second loss is determined based on the first classification result and the second classification result.

6. The method according to claim 5, characterized in that, The method further includes: Select sample images from the target dataset; The sample image is input into the second network model to establish a first selection path, and the sample image is input into the fifth network model to establish a second selection path; The third loss is determined based on the first selected path and the second selected path.

7. The method according to claim 6, characterized in that, The method further includes: The fourth loss is determined based on the first classification result and the preset label.

8. The method according to claim 7, characterized in that, The third network model is obtained by adjusting the network parameters of the second network model based at least on the first loss, including: The target loss is determined using the first loss, the second loss, the third loss, and the fourth loss. The network parameters of the second network model are adjusted based on the target loss to obtain the third network model.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the target image to be classified; The fourth network model is used to classify and predict the target image to be classified, and the classification result is obtained.

10. An image classification method, characterized in that, include: Obtain the target image to be classified; A dynamic prediction network model is used to classify and predict the target image to be classified, and the classification result is obtained. The dynamic prediction network model is determined by a target dataset, a first network model, and a dynamic routing structure. The target dataset is an image classification dataset. The first network model and the dynamic routing structure are used to construct a second network model. The first network model is an image classification network model and includes multiple network substructures. The dynamic routing structure is used to select and skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The target dataset is used to train the second network model to obtain a third network model. The third network model is a target dynamic inference network model with optimized network parameters. The third network model is used to binarize the dynamic routing structure to obtain the dynamic prediction network model. The process of training the second network model to obtain the third network model includes: dividing the target dataset into multiple batches of images; inputting the multiple batches of images into the second network model for training in batches, aligning the features before each downsampling layer in the second network model to obtain multiple aligned features; calculating multiple evolution angles using the multiple aligned features; determining the feature evolution direction using the multiple evolution angles; optimizing the feature evolution direction to determine a first loss; and adjusting the network parameters of the second network model based at least on the first loss to obtain the third network model.

11. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the method for obtaining a network model as described in any one of claims 1 to 9 or the image classification method as described in claim 10.

12. An electronic device, characterized in that, include: processor; as well as A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: Obtain the target dataset, wherein the target dataset is an image classification dataset; Based on the first network model and the dynamic routing structure, a second network model is constructed. The first network model is an image classification network model, which includes multiple network substructures. The dynamic routing structure is used to select to skip or execute at least one of the multiple network substructures. The second network model is an initial dynamic inference network model based on path selection. The target dataset is divided into multiple batches of images; The multiple batches of images are input into the second network model in batches for training, and the features before each downsampling layer in the second network model are aligned to obtain multiple aligned features; Multiple evolution angles are calculated using the aforementioned alignment features; The direction of feature evolution is determined by the multiple evolution angles; The evolution direction of the aforementioned features is optimized to determine the first loss; The network parameters of the second network model are adjusted based on at least the first loss to obtain a third network model, wherein the third network model is a target dynamic inference network model with optimized network parameters; The dynamic routing structure is binarized based on the third network model to obtain a fourth network model, which is a dynamic prediction network model used to classify the target image to be classified.

Images