Network architecture search method, image classification method, device and electronic equipment

Abstract

The application relates to the field of artificial intelligence, and provides a network architecture search method, an image classification method, a device and electronic equipment, wherein the search method comprises the following steps: acquiring an initial network structure, determining a search space based on each structure parameter of the initial network structure; building a super network based on the search space, and pre-training the super network based on a first sample image to obtain pre-training weights of the pre-training super network; adopting an evolution algorithm to search and optimize the network structure under a current evolution generation number based on an adaptive probability distribution of a candidate mutation operation under the current evolution generation number, obtaining the network structure under a next evolution generation number, and stopping evolution until a target network structure is obtained. The network architecture search method, the image classification method, the device and the electronic equipment provided by the application improve the search efficiency, and can stably and efficiently optimize a high-performance small-scale ViT network.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a network architecture search method, an image classification method, an apparatus, and an electronic device. Background Technology

[0002] Neural Architecture Search (NAS) is a method for constructing neural network models that can automatically tune hyperparameters, greatly reducing the time required for manual adjustment.

[0003] The ViT (Vision Transformer) network structure typically divides an image into multiple fixed-size image patches and then learns the relationships between these patches through a self-attention mechanism to extract feature information relevant to the target task, thus demonstrating excellent image understanding capabilities in various visual tasks. However, the ViT network structure still performs poorly when the model parameters are small.

[0004] Therefore, there is an urgent need for a network architecture search method to optimize high-performance small-scale ViT network structures. Summary of the Invention

[0005] This invention provides a network architecture search method, an image classification method, a device, and an electronic device to address the shortcomings of existing ViT network structures that still perform poorly when model parameters are small.

[0006] This invention provides a network architecture search method, comprising:

[0007] Obtain the initial network structure, and determine the search space based on the structural parameters of the initial network structure;

[0008] A supernetwork is constructed based on the search space, and the supernetwork is pre-trained based on the first sample image to obtain the pre-trained weights of the pre-trained supernetwork.

[0009] Based on the adaptive probability distribution of candidate mutation operations under the current evolutionary generation, an evolutionary algorithm is used to search and optimize the network structure under the current evolutionary generation to obtain the network structure under the next evolutionary generation, until the evolution stops and the target network structure is obtained.

[0010] The adaptive probability distribution of candidate mutation operations in the current evolutionary generation is determined by evaluating the network structure in the previous evolutionary generation based on the pre-trained weights; the network structure in the first evolutionary generation is determined based on the initial population of the evolutionary algorithm and the supernetwork.

[0011] According to the network architecture search method provided by the present invention, the step of determining the adaptive probability distribution of candidate mutation operations under the current evolutionary generation includes:

[0012] Based on the pre-trained weights, the performance of the network structure under the previous evolutionary generation is evaluated, and the performance evaluation results under the previous evolutionary generation are obtained.

[0013] Based on the performance evaluation results of the previous evolutionary generation, the ranking probability distribution of candidate mutation operations in the previous evolutionary generation is updated to obtain the ranking probability distribution of candidate mutation operations in the current evolutionary generation.

[0014] Based on the sorting probability distribution of candidate mutation operations under the current evolutionary generation, an adaptive probability distribution of candidate mutation operations under the current evolutionary generation is determined.

[0015] According to the network architecture search method provided by the present invention, the step of updating the ranking probability distribution of candidate mutation operations in the previous evolutionary generation based on the performance evaluation results in the previous evolutionary generation to obtain the ranking probability distribution of candidate mutation operations in the current evolutionary generation includes:

[0016] The performance evaluation results of the previous evolutionary generation are sorted, and a preset number of candidate network structures are determined based on the sorting results;

[0017] Based on the number of times each candidate mutation operation appears in the preset number of candidate network structures, the ranking probability distribution under the current evolution generation is determined.

[0018] According to the network architecture search method provided by the present invention, determining the adaptive probability distribution of candidate mutation operations based on the ranking probability distribution of candidate mutation operations at the current evolutionary generation includes:

[0019] Based on the sorting probability distribution, uniform probability distribution and current evolution generation of the candidate mutation operations, an adaptive probability distribution of the candidate mutation operations is determined for the current evolution generation, wherein the uniform probability distribution is determined based on the number of candidate mutation operations.

[0020] According to the network architecture search method provided by the present invention, the initial network structure includes a Transformer encoder and a width attention module. The Transformer encoder includes multiple Transformer modules, and the width attention module includes width connectivity and parameterless attention.

[0021] According to the network architecture search method provided by the present invention, the search space includes the number of attention heads, the number of Transformer modules, the feature dimension, the multilayer perceptron ratio, and the width connection parameter.

[0022] The present invention also provides an image classification method, comprising:

[0023] Identify the image to be classified;

[0024] Based on the image classification model, the image to be classified is classified to obtain the classification result;

[0025] The image classification model is trained on the target network structure based on the second sample image, and the target network structure is determined based on the network architecture search method described above.

[0026] The present invention also provides a network architecture search device, comprising:

[0027] The search space determination unit is used to obtain the initial network structure and determine the search space based on the structural parameters of the initial network structure.

[0028] The pre-training unit is used to build a supernetwork based on the search space and pre-train the supernetwork based on the first sample image to obtain the pre-training weights of the pre-trained supernetwork.

[0029] The search optimization unit is used to search and optimize the network structure under the current evolutionary generation based on the adaptive probability distribution of candidate mutation operations under the current evolutionary generation, and to obtain the network structure under the next evolutionary generation, until the evolution stops and the target network structure is obtained.

[0030] The adaptive probability distribution of candidate mutation operations in the current evolutionary generation is determined by evaluating the network structure in the previous evolutionary generation based on the pre-trained weights; the network structure in the first evolutionary generation is determined based on the initial population of the evolutionary algorithm and the supernetwork.

[0031] The present invention also provides an image classification device, comprising:

[0032] Image determination unit, used to determine the image to be classified;

[0033] An image classification unit is used to classify the image to be classified based on an image classification model to obtain a classification result;

[0034] The image classification model is obtained by training the target network structure based on the second sample image, and the target network structure is determined based on the network architecture search method.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a network architecture search method or an image classification method as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the network architecture search method or image classification method as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a network architecture search method or an image classification method as described above.

[0038] The network architecture search method, image classification method, device, and electronic device provided by this invention, based on traditional evolutionary algorithms, incorporate adaptive learning of the probability distribution of candidate mutation operations, which improves search efficiency compared to traditional evolutionary algorithms and can stably and efficiently optimize high-performance small-scale ViT networks. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is one of the flowcharts illustrating the network architecture search method provided by the present invention;

[0041] Figure 2 This is the width connection paradigm in the pyramid-shaped width ViT provided by the present invention;

[0042] Figure 3 This is the second flowchart illustrating the network architecture search method provided by the present invention;

[0043] Figure 4 This is the third schematic diagram of the network architecture search method provided by the present invention;

[0044] Figure 5 This is a flowchart illustrating the image classification method provided by the present invention;

[0045] Figure 6 This is a schematic diagram of the network architecture search device provided by the present invention;

[0046] Figure 7 This is a schematic diagram of the image classification device provided by the present invention;

[0047] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] Current mainstream ViT architectures draw inspiration from novel architectural designs in convolutional neural networks, such as the pyramid structure, gradually reducing feature resolution and changing feature dimensions at different stages of the neural architecture. Existing pyramid-shaped ViTs primarily achieve changes in feature resolution and dimension between different stages through operations such as convolution, pooling, and linear transformations.

[0050] However, the pyramid-shaped ViT still performs poorly with smaller model parameters, especially compared to convolutional neural networks. Specifically, with approximately 5M parameters, the efficient convolutional neural network EfficientNet-B0 achieves a 2.5% higher classification accuracy on ImageNet than the pooling-based ViT network PiT. When the model has around 30M parameters, the efficient convolutional neural network EfficientNet-B5 achieves a 2.3% higher classification accuracy than the sliding window-based ViT network SwinTransformer. This indicates that convolutional neural networks remain a powerful choice when devices cannot deploy excessively large models.

[0051] Based on the above considerations, in order to solve the problem of poor performance of small-scale ViT networks and improve their image classification performance, the inventive concept of this invention is to add adaptive learning of the probability distribution of candidate mutation operations to the traditional evolutionary algorithm. Compared with the traditional evolutionary algorithm, this improves the search efficiency and can stably and efficiently optimize high-performance small-scale ViT networks.

[0052] Based on the above-mentioned inventive concept, the present invention provides a network architecture search method, an image classification method, a device, and an electronic device, which are applied to image recognition and classification scenarios in artificial intelligence technology to improve the image classification accuracy of small-scale ViT networks.

[0053] Embodiments of this invention can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. This invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0054] The network architecture search method, image classification method, apparatus, and electronic device provided in the embodiments of the present invention will now be described with reference to the accompanying drawings.

[0055] Figure 1 This is one of the flowcharts illustrating the network architecture search method provided by the present invention, such as... Figure 1 As shown, the method may include the following steps:

[0056] Step 110: Obtain the initial network structure and determine the search space based on the structural parameters of the initial network structure.

[0057] Specifically, the initial network structure can be a deep neural network structure, a shallow neural network structure, or a lightweight neural network structure, which will not be elaborated here.

[0058] In some embodiments, the initial network structure can be a ViT network structure. Furthermore, considering that wide attention can jointly focus on attention information from different layers using wide connections and parameterless attention, ViT can perform more comprehensive feature learning and has a better local receptive field, resulting in superior performance on finite-scale datasets. Therefore, exploring efficient wide ViT network architectures has the potential to achieve structural optimization of small-scale ViT networks.

[0059] Based on this, a solution space for optimizing the width ViT network structure can be constructed, which is the search space of the search method.

[0060] The constructed width search space includes different structural parameters in the width ViT, which is mainly composed of a Transformer encoder and a width attention module, which includes width connectivity and parameterless attention.

[0061] The Transformer encoder contains multiple Transformer modules, and its main components are Multi-Head Self-Attention (MHSA) and Multi-Layer Perceptron (MLP).

[0062] Given input x N×d The expression for MHSA is as follows:

[0063]

[0064] Where q = [q 1 q 2 , ..., q h ],q∈N×d a

[0065] k = [k 1 k 2 , ..., k h ],k∈N×d a

[0066] v = [v 1 v 2 , ..., v h ],v∈N×d a

[0067] Among them, To_qkv consists of linear mapping, segmentation, and transformation, w o Here, q, k, and v represent the weights of the last linear mapping in MHSA, respectively, and d represents the query vector, key vector, and value vector in the attention process. a denoted as the attention feature dimension, and h represents the number of attention heads.

[0068] MLP consists of two fully connected layers, and the hidden layer dimension of MLP is... Where r is the MLP ratio.

[0069] Width attention jointly focuses on the query vector q, key vector k, and value vector v of different Transformer modules. It integrates the attention information of different Transformer modules using width connection, and then uses parameterless attention to jointly focus on the attention information.

[0070] Figure 2 This invention provides a width connection paradigm in a pyramid-shaped width ViT, such as... Figure 2 As shown, the wide-connection paradigm is achieved through skip connections between different layers, utilizing these skip connections to transfer and fuse attention information. Wide attention can extract richer and more effective features, enhance the local receptive field of ViT, and promote the learning and utilization of features.

[0071] Based on the structural details of the aforementioned width-based ViT, the constructed width search space includes the number of attention heads h, the number of Transformer modules l, the feature dimension d, the MLP ratio r, and the width connection parameter. Here, the width connection parameter indicates whether the Transformer modules perform width-based attention connections. For example, a width connection parameter of 1 indicates that width-based attention connections are performed, and a width connection parameter of 0 indicates that width-based attention connections are not performed.

[0072] Table 1 shows the candidate values ​​for different architecture parameters in the wide search space, including the search spaces for non-pyramid and pyramid ViTs. Due to the inconsistency in resolution and dimension at different stages in the pyramid structure, the candidate values ​​for the structural parameters at each stage also differ. The search results for these two structure types demonstrate the reliability and robustness of the adaptive ViT structure search method.

[0073] Table 1

[0074]

[0075] Step 120: Construct a supernetwork based on the search space, and pre-train the supernetwork based on the first sample image to obtain the pre-trained weights of the pre-trained supernetwork.

[0076] Specifically, after constructing the width search space in step 110, the constructed width search space can be used to build a supernetwork and then trained.

[0077] By combining different structural parameters, different Transformer modules can be obtained. Different Transformer modules are connected in parallel at each layer and combined to output to the next layer. Multiple structural layers containing different Transformer modules can be connected in series to obtain a supernetwork A.

[0078] Based on this, the first sample image can be collected as a dataset, including training data and test data, to train the constructed hypernetwork.

[0079] In each training step, single-path sampling is performed on the supernetwork A, meaning only one Transformer module is sampled from each layer, and training is performed step by step. The performance of the sampled network is then tested on the test data, and the weights w of the supernetwork are updated. A Repeat single-path sampling and training until the supernetwork training converges, obtaining the pre-trained weights of the pre-trained supernetwork.

[0080] It should be noted that the pre-trained supernetwork obtained in this way can be used as a performance evaluation model for the wide ViT network to evaluate the network's performance, which is beneficial for subsequent search optimization.

[0081] Step 130: Based on the adaptive probability distribution of candidate mutation operations under the current evolutionary generation, use an evolutionary algorithm to search and optimize the network structure under the current evolutionary generation to obtain the network structure under the next evolutionary generation, until the evolution stops and the target network structure is obtained.

[0082] Among them, the network structure and adaptive probability distribution of candidate mutation operations under the current evolutionary generation are determined by performance evaluation of the network structure under the previous evolutionary generation based on pre-trained weights; the network structure under the first evolutionary generation is determined based on the initial population and supernetwork of the evolutionary algorithm.

[0083] Specifically, the target network structure here can be understood as the optimal substructure in the supernetwork. In order to obtain the target network structure, an evolutionary algorithm can be used to optimize the width ViT network structure based on the pre-trained supernetwork, and adaptively search for the target network structure with excellent performance.

[0084] In this embodiment, adaptive learning of the probability distribution of candidate mutation operations is added to the traditional evolutionary algorithm, which improves the search efficiency compared to the traditional evolutionary algorithm and can stably and efficiently optimize high-performance small-scale ViT networks.

[0085] Assuming the current evolutionary generation is t, adaptive mutation can be performed on the t-th generation population of the evolutionary algorithm based on the adaptive probability distribution of the candidate mutation operation, crossover can be performed, and selection can be performed based on the network performance evaluation results to obtain the network structure under the next evolutionary generation, until the evolution stops and the target network structure is obtained.

[0086] The stopping condition here can be pre-set, such as stopping evolution when the number of iterations equals the total number of generations. During the search optimization process, a biased selection is made based on the adaptive probability distribution of candidate mutation operations, thereby promoting the convergence of the search method and obtaining a high-performance target network structure.

[0087] Among them, the network structure under the first evolutionary generation is determined based on the initial population and supernetwork of the evolutionary algorithm.

[0088] In one embodiment, Figure 3 This is the second flowchart illustrating the network architecture search method provided by the present invention, as shown below. Figure 3 As shown, adaptively searching for a high-performance target structure can be achieved through the following steps:

[0089] 1) Based on the hyperparameters and search space of the adaptive evolutionary algorithm, initialize the iteration number t = 0 (i.e., the first evolutionary generation) of the evolutionary algorithm, and initialize the population of the t-th generation of the evolutionary algorithm. The population can be initialized to randomly generate P network structure codes, and the network structure codes can be converted into sub-network structures in the supernetwork to obtain the network structure under the first evolutionary generation.

[0090] The hyperparameters of the adaptive evolutionary algorithm can be set as follows: number of generations G = 20, population size P = 50, adaptive mutation performed 25 times with a mutation probability of 0.5, and crossover performed 25 times. In the evolutionary algorithm, the evolutionary objects need to be encoded, and mutation and crossover are performed on the encoded data. Therefore, in this embodiment, the candidate values ​​used for different structural parameters in the sampled ViT network are directly encoded. Meanwhile, to optimize small-scale ViT models, the number of network parameters is limited; network encodings exceeding the parameter limit are discarded.

[0091] 2) Initialize the adaptive probability distribution of candidate mutation operations.

[0092] Considering that an excessively large search space can make evolutionary algorithms difficult to converge and prone to getting trapped in local optima, this embodiment adaptively learns the direction of mutation, causing mutation operations to tend to select candidate values ​​near the optimal solution of the evolutionary algorithm rather than blindly exploring. Specifically, an adaptive learning mechanism for the probability distribution of candidate mutation operations is designed, and the learning of the probability distribution of candidate mutation operations mainly follows the ranking of the ViT neural architecture in the performance evaluation during the search process.

[0093] 3) Based on the pre-trained weights, evaluate the fitness of the t-th generation population of the evolutionary algorithm. That is, evaluate the performance of the network structure under the previous evolutionary generation and obtain the performance evaluation results under the previous evolutionary generation.

[0094] Load the pre-trained weights of the supernetwork into the evolved network structure. The evolved network structure is evaluated on the test dataset sampled from the training data to obtain the performance evaluation result, i.e., fitness. Here, fitness can be understood as the classification accuracy of the image classification task. The fitness f(a) of network structure a is shown in the following formula:

[0095]

[0096] 4) Based on the adaptive probability distribution of the candidate mutation operation, perform adaptive mutation, crossover, and selection based on the network performance evaluation results for the t-th generation population of the evolutionary algorithm.

[0097] Performing mutation operations on network structure encoding requires selecting from candidate mutation operations based on the adaptive probability distribution π of the candidate mutation operations. aBiased selection is employed to promote convergence of the search method. Crossover operations on the population can be achieved by randomly selecting and swapping codes at the same positions in two network structures. The next generation of the population is generated by selecting the network structure with the highest classification accuracy.

[0098] 5) Based on the evaluation results of the network structure, update the adaptive probability distribution of candidate mutation operations in the evolutionary algorithm. Specifically, this may include:

[0099] Based on the performance evaluation results of the previous evolutionary generation, the ranking probability distribution of candidate mutation operations in the previous evolutionary generation is updated to obtain the ranking probability distribution of candidate mutation operations in the current evolutionary generation.

[0100] First, the performance evaluation results from the previous evolutionary generation are ranked, and a predetermined number of candidate network structures are determined based on the ranking results. For example, the top k network structures in terms of performance evaluation results can be selected as candidate network structures.

[0101] Then, based on the number of times each candidate mutation operation appears in a preset number of candidate network structures, the ranking probability distribution under the current evolution generation is determined.

[0102] In some embodiments, the sorting probability distribution π r It can be represented as:

[0103]

[0104] Where, r j This represents the number of times the j-th candidate mutation operation appears in network structures that rank among the top k in structural performance evaluation. For example, for a Transformer module with three candidate mutation operations, if k=5, the first candidate mutation operation appears twice, the second candidate mutation operation appears once, and the third candidate mutation operation appears twice, then its ranking probability distribution is:

[0105]

[0106] Based on this, an adaptive probability distribution of candidate mutation operations can be determined according to the ranking probability distribution of candidate mutation operations in the current evolutionary generation.

[0107] The adaptive probability distribution is determined based on the sorting probability distribution, uniform probability distribution and current evolution generation of candidate mutation operations, while the uniform probability distribution is determined based on the number of candidate mutation operations.

[0108] Using uniform probability distribution π u With the sorting probability distribution π r The adaptive probability distribution π can be obtained. a Uniform probability distribution πu for:

[0109]

[0110] Where n is the number of candidate mutation operations. For example, for the number of Transformer modules with three candidate mutation operations, its uniform probability distribution is:

[0111]

[0112] Considering that the ordering probability distribution becomes more stable and reliable as it evolves, the adaptive probability distribution π a For a uniform probability distribution π u With the sorting probability distribution π r The choice of is related to the current generation g, specifically expressed as:

[0113] π a =f(g)π u +(1-f(g))π r

[0114] f(g) is a function related to the current evolutionary generation g of the evolutionary algorithm:

[0115]

[0116] Where G is the total number of evolutionary generations in the evolutionary algorithm, g w The warm-up algebra for the adaptive evolution algorithm is to obtain a relatively reliable sorting probability distribution π. r The initialization of the candidate mutation operation probability distribution requires calculating the uniform probability distribution π of candidate mutation operations for different structural parameters of the network structure, based on the number of candidate operations in the search space. u .

[0117] In this embodiment, the sorting probability distribution π r The number of high-performance networks k used in the algorithm can be set to 5, and the warm-up algebra g of the adaptive evolution algorithm is... w It can be set to 3.

[0118] Therefore, network structures can be ranked based on the evaluation results of the evolved structures. The ranking probability distribution π is calculated by statistically analyzing the different structural parameters of the top k network structures. r And based on π r With the current evolutionary algebra g and the adaptive probability distribution π a Update accordingly. As the evolutionary algorithm iterates, the sorting probability distribution π is applied. r The probability gradually increases, which can guide the direction of mutation, promote the convergence of evolutionary algorithms, improve search efficiency, and help explore and utilize novel ViT network architectures.

[0119] 6) Determine if the iteration termination condition of the evolutionary algorithm is met. If yes, decode the optimal solution found by the evolutionary algorithm to obtain the optimal width ViT network, i.e., the target network structure. If not, let t = t + 1, return to the adaptive probability distribution based on the candidate mutation operation under the current evolutionary generation, and use the evolutionary algorithm to search and optimize the network structure under the current evolutionary generation. The termination condition can be set to the iteration number t equal to the total evolutionary generation G.

[0120] The adaptive evolution algorithm can be used to search for high-performance small-scale ViT structures. By comparing the traditional evolution algorithm with the adaptive evolution algorithm during the search process, it can be concluded that the adaptive evolution algorithm significantly improves the search efficiency and obtains better convergence results.

[0121] The network architecture search method provided in this embodiment of the invention incorporates adaptive learning of the probability distribution of candidate mutation operations on the basis of traditional evolutionary algorithms, which improves search efficiency compared to traditional evolutionary algorithms and can stably and efficiently optimize high-performance small-scale ViT networks.

[0122] Furthermore, by utilizing wide connectivity and parameterless attention to jointly focus on the attention information of different layers, ViT can perform feature learning more fully and has a better local receptive field, resulting in excellent performance on datasets with limited size.

[0123] In some embodiments, Figure 4 This is the third schematic diagram of the network architecture search method provided by the present invention, as shown below. Figure 4 As shown, a performance evaluation model for the width ViT network is designed, namely a pre-trained supernetwork; then, an adaptive evolution algorithm is used to optimize the width ViT network structure based on the pre-trained supernetwork, and adaptively search for a target network structure with excellent performance.

[0124] Based on the above embodiments, Figure 5 This is a flowchart illustrating the image classification method provided by the present invention, as shown below. Figure 5 As shown, this image classification method includes:

[0125] Step 510: Determine the image to be classified;

[0126] Step 520: Based on the image classification model, perform image classification on the image to be classified to obtain the classification result;

[0127] The image classification model is trained on the target network structure based on the second sample image, and the target network structure is determined based on the network architecture search method described above.

[0128] Specifically, the image to be classified is the image that needs to be classified. It can be acquired by an image acquisition device or obtained by crawling the Internet. This embodiment does not make specific limitations on this.

[0129] The image to be classified can be input into the trained image classification model, which will then classify the image to obtain the classification result.

[0130] The image classification model can be trained on the target network structure based on the second sample image. The second sample image can be the same as or different from the first sample image. The target network structure is obtained through the network architecture search method described in the above embodiments, and will not be repeated here.

[0131] The image classification method provided in this invention improves the accuracy and reliability of image classification by training a target network structure and performing image classification tasks based on the trained image classification model.

[0132] The network architecture search device provided by the present invention is described below. The network architecture search device described below can be referred to in correspondence with the network architecture search method described above.

[0133] Based on the above embodiments, Figure 6 This is a schematic diagram of the network architecture search device provided by the present invention, as shown below. Figure 6 As shown, the network architecture search device includes:

[0134] The search space determination unit 610 is used to obtain the initial network structure and determine the search space based on the structural parameters of the initial network structure.

[0135] The pre-training unit 620 is used to build a super network based on the search space and pre-train the super network based on the first sample image to obtain the pre-training weights of the pre-trained super network.

[0136] The search optimization unit 630 is used to search and optimize the network structure under the current evolutionary generation based on the adaptive probability distribution of the candidate mutation operation under the current evolutionary generation, and obtain the network structure under the next evolutionary generation until the evolution stops and the target network structure is obtained.

[0137] The adaptive probability distribution of candidate mutation operations in the current evolutionary generation is determined by evaluating the network structure in the previous evolutionary generation based on the pre-trained weights; the network structure in the first evolutionary generation is determined based on the initial population of the evolutionary algorithm and the supernetwork.

[0138] The network architecture search device provided in this embodiment of the invention incorporates adaptive learning of the probability distribution of candidate mutation operations on the basis of traditional evolutionary algorithms, which improves search efficiency compared to traditional evolutionary algorithms and can stably and efficiently optimize high-performance small-scale ViT networks.

[0139] Based on the above embodiments, the network architecture search device further includes a probability distribution determination unit, used for:

[0140] Based on the pre-trained weights, the performance of the network structure under the previous evolutionary generation is evaluated, and the performance evaluation results under the previous evolutionary generation are obtained.

[0141] Based on the performance evaluation results of the previous evolutionary generation, the ranking probability distribution of candidate mutation operations in the previous evolutionary generation is updated to obtain the ranking probability distribution of candidate mutation operations in the current evolutionary generation.

[0142] Based on the sorting probability distribution of candidate mutation operations under the current evolutionary generation, an adaptive probability distribution of candidate mutation operations under the current evolutionary generation is determined.

[0143] Based on the above embodiments, the probability distribution determination unit is specifically used for:

[0144] The performance evaluation results of the previous evolutionary generation are sorted, and a preset number of candidate network structures are determined based on the sorting results;

[0145] Based on the number of times each candidate mutation operation appears in the preset number of candidate network structures, the ranking probability distribution under the current evolution generation is determined.

[0146] Based on the above embodiments, the probability distribution determination unit is specifically used for:

[0147] Based on the sorting probability distribution, uniform probability distribution and current evolution generation of the candidate mutation operations, an adaptive probability distribution of the candidate mutation operations is determined for the current evolution generation, wherein the uniform probability distribution is determined based on the number of candidate mutation operations.

[0148] Based on the above embodiments, the initial network structure includes a Transformer encoder and a width attention module. The Transformer encoder includes multiple Transformer modules, and the width attention module includes width connections and parameterless attention.

[0149] Based on the above embodiments, the search space includes the number of attention heads, the number of Transformer modules, the feature dimension, the multilayer perceptron ratio, and the width connection parameter.

[0150] Based on the above embodiments, Figure 7This is a schematic diagram of the image classification device provided by the present invention, as shown below. Figure 7 As shown, the image classification device includes:

[0151] Image determination unit 710 is used to determine the image to be classified;

[0152] The image classification unit 720 is used to classify the image to be classified based on the image classification model and obtain the classification result;

[0153] The image classification model is trained on the target network structure based on the second sample image, and the target network structure is determined based on the network architecture search method described above.

[0154] The image classification device provided in this embodiment of the invention can improve the accuracy and reliability of image classification by training a target network structure and performing image classification tasks based on the trained image classification model.

[0155] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a network architecture search method, which includes:

[0156] Obtain the initial network structure, and determine the search space based on the structural parameters of the initial network structure;

[0157] A supernetwork is constructed based on the search space, and the supernetwork is pre-trained based on the first sample image to obtain the pre-trained weights of the pre-trained supernetwork.

[0158] Based on the adaptive probability distribution of candidate mutation operations under the current evolutionary generation, an evolutionary algorithm is used to search and optimize the network structure under the current evolutionary generation to obtain the network structure under the next evolutionary generation, until the evolution stops and the target network structure is obtained.

[0159] The adaptive probability distribution of candidate mutation operations in the current evolutionary generation is determined by evaluating the network structure in the previous evolutionary generation based on the pre-trained weights; the network structure in the first evolutionary generation is determined based on the initial population of the evolutionary algorithm and the supernetwork.

[0160] The processor can invoke logical instructions in memory to execute an image classification method, which includes:

[0161] Identify the image to be classified;

[0162] Based on the image classification model, the image to be classified is classified to obtain the classification result;

[0163] The image classification model is trained on the target network structure based on the second sample image, and the target network structure is determined based on the network architecture search method described above.

[0164] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the network architecture search method provided by the above methods, the method comprising:

[0166] Obtain the initial network structure, and determine the search space based on the structural parameters of the initial network structure;

[0167] A supernetwork is constructed based on the search space, and the supernetwork is pre-trained based on the first sample image to obtain the pre-trained weights of the pre-trained supernetwork.

[0168] Based on the adaptive probability distribution of candidate mutation operations under the current evolutionary generation, an evolutionary algorithm is used to search and optimize the network structure under the current evolutionary generation to obtain the network structure under the next evolutionary generation, until the evolution stops and the target network structure is obtained.

[0169] The adaptive probability distribution of candidate mutation operations in the current evolutionary generation is determined by evaluating the network structure in the previous evolutionary generation based on the pre-trained weights; the network structure in the first evolutionary generation is determined based on the initial population of the evolutionary algorithm and the supernetwork.

[0170] When the computer program is executed by the processor, the computer can perform the image classification methods provided by the above methods, which include:

[0171] Identify the image to be classified;

[0172] Based on the image classification model, the image to be classified is classified to obtain the classification result;

[0173] The image classification model is trained on the target network structure based on the second sample image, and the target network structure is determined based on the network architecture search method described above.

[0174] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the network architecture search methods provided by the methods described above, the method comprising:

[0175] Obtain the initial network structure, and determine the search space based on the structural parameters of the initial network structure;

[0176] A supernetwork is constructed based on the search space, and the supernetwork is pre-trained based on the first sample image to obtain the pre-trained weights of the pre-trained supernetwork.

[0177] Based on the adaptive probability distribution of candidate mutation operations under the current evolutionary generation, an evolutionary algorithm is used to search and optimize the network structure under the current evolutionary generation to obtain the network structure under the next evolutionary generation, until the evolution stops and the target network structure is obtained.

[0178] The adaptive probability distribution of candidate mutation operations in the current evolutionary generation is determined by evaluating the network structure in the previous evolutionary generation based on the pre-trained weights; the network structure in the first evolutionary generation is determined based on the initial population of the evolutionary algorithm and the supernetwork.

[0179] When executed by a processor, this computer program is implemented to perform the image classification methods provided by the methods described above, which include:

[0180] Identify the image to be classified;

[0181] Based on the image classification model, the image to be classified is classified to obtain the classification result;

[0182] The image classification model is trained on the target network structure based on the second sample image, and the target network structure is determined based on the network architecture search method described above.

[0183] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0184] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network architecture search method, characterized by, The method comprises the following steps: obtaining an initial network structure, and determining a search space based on structure parameters of the initial network structure; building a super network based on the search space, and pre-training the super network based on a first sample image to obtain pre-training weights of the pre-trained super network; searching and optimizing the network structure at a current evolution generation by using an evolution algorithm based on an adaptive probability distribution of candidate mutation operations at the current evolution generation, to obtain a network structure at a next evolution generation, until stopping evolution, to obtain a target network structure; training the target network structure based on a second sample image to obtain an image classification model, and performing image classification on a to-be-classified image based on the image classification model to obtain a classification result; wherein the adaptive probability distribution of the candidate mutation operations at the current evolution generation is determined based on the pre-training weights, and the network structure at the first evolution generation is determined based on an initialization population of the evolution algorithm and the super network.

2. The network architecture search method of claim 1, wherein, The determination step of the adaptive probability distribution of the candidate mutation operations at the current evolution generation comprises: performing performance evaluation on the network structure at a previous evolution generation based on the pre-training weights to obtain a performance evaluation result at the previous evolution generation; updating a ranking probability distribution of the candidate mutation operations at the previous evolution generation based on the performance evaluation result at the previous evolution generation to obtain a ranking probability distribution of the candidate mutation operations at the current evolution generation; determining the adaptive probability distribution of the candidate mutation operations at the current evolution generation based on the ranking probability distribution of the candidate mutation operations at the current evolution generation.

3. The network architecture search method of claim 2, wherein, The step of updating the ranking probability distribution of the candidate mutation operations at the previous evolution generation based on the performance evaluation result at the previous evolution generation to obtain the ranking probability distribution of the candidate mutation operations at the current evolution generation comprises: ranking the performance evaluation result at the previous evolution generation, and determining a preset number of candidate network structures based on the ranking result; determining the ranking probability distribution at the current evolution generation based on the number of times each candidate mutation operation appears in the preset number of candidate network structures.

4. The network architecture search method of claim 2, wherein, The step of determining the adaptive probability distribution of the candidate mutation operations at the current evolution generation based on the ranking probability distribution of the candidate mutation operations at the current evolution generation comprises: determining the adaptive probability distribution of the candidate mutation operations at the current evolution generation based on the ranking probability distribution of the candidate mutation operations at the current evolution generation, a uniform probability distribution, and the current evolution generation, wherein the uniform probability distribution is determined based on the number of candidate mutation operations.

5. The network architecture search method of any one of claims 1-4, wherein, The initial network structure comprises a Transformer encoder and a width attention module, the Transformer encoder comprises a plurality of Transformer modules, and the width attention module comprises a width connection and a parameter-free attention.

6. The network architecture search method of any one of claims 1-4, wherein, The search space comprises the number of attention heads, the number of Transformer modules, the feature dimension, the multi-layer perception ratio, and the width connection parameter.

7. A network architecture search apparatus, characterized by comprising: The method comprises the following steps: The search space determination unit is configured to obtain an initial network structure and determine a search space based on structure parameters of the initial network structure. The pre-training unit is configured to build a hypernetwork based on the search space, pre-train the hypernetwork based on a first sample image, and obtain pre-training weights of the pre-trained hypernetwork. The search optimization unit is configured to perform search optimization on the network structure at a current evolution generation based on an adaptive probability distribution of candidate mutation operations at the current evolution generation, using an evolution algorithm, to obtain a network structure at a next evolution generation, until evolution is stopped, and obtain a target network structure. The target network structure is trained based on a second sample image to obtain an image classification model. An image classification is performed on a to-be-classified image based on the image classification model to obtain a classification result. The adaptive probability distribution of the candidate mutation operations at the current evolution generation is determined based on performance evaluation of the pre-training weights on the network structure at a previous evolution generation.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the network architecture search method according to any one of claims 1 to 6 when executing the program.

Images