Neural architecture searching method based on diffusion evolutionary algorithm

Abstract

The invention discloses a neural architecture search method based on a diffusion evolution algorithm, and the method comprises the steps: firstly constructing a supernet which comprises subnets, and enabling each subnet to give a weight to each operation on each connection edge through continuous coding; then obtaining different types of pictures to construct a data set; and finally, performing alternative optimization on the super-network weight and the sub-network codes, stopping optimization until constraint conditions are met, obtaining optimal sub-network codes, and further obtaining the architecture of the neural network. According to the method, an iterative denoising mechanism of a diffusion model and a global search strategy of an evolutionary algorithm are creatively fused, a super network is constructed by combining the denoising mechanism of the diffusion model and the global search capability of the evolutionary algorithm, and network weight and subnetwork coding are alternately optimized; through adaptive noise scheduling and density estimation, population diversity is enhanced, local optimum is avoided, and an optimal neural network architecture suitable for a task can be better found.

Description

Technical Field

[0001] The present invention relates to deep neural network technology, and in particular to a neural architecture search method based on a diffusion evolution algorithm. Background Art

[0002] Deep learning models, especially convolutional neural networks, have achieved remarkable results in computer vision tasks such as image classification and object detection. However, manually designing an effective network architecture requires a lot of expertise and repeated trials. To solve this problem, neural network architecture search came into being, aiming to automatically discover the optimal network structure and reduce dependence on manual design.

[0003] Existing neural network architecture search methods are mainly divided into three categories: reinforcement learning-based, gradient-based, and evolutionary algorithm-based. Among them, although the reinforcement learning-based method can obtain better search results, it requires large computing resources; gradient-based methods such as differentiable architecture search can quickly find potential architectures, but it is easy to obtain suboptimal architectures with redundancy and more jump connections, and it is easy to fall into local optimality; evolutionary algorithm-based methods have advantages in exploring the search space, but consume too much computing resources when evaluating candidate networks.

[0004] In recent years, researchers have proposed a variety of improvement methods, but these methods still have problems in practical applications, such as limited population diversity and low performance of the searched architecture. At the same time, although the differentiable architecture search framework improves the search efficiency, the gradient descent method is prone to fall into the local optimal solution, and the simultaneous optimization of the network architecture weights of all candidate operations will cause interference between operations. Therefore, a new search method is needed to solve the defects in the current neural network architecture search process. Summary of the invention

[0005] Purpose of the invention: In view of the above problems, the purpose of the present invention is to provide a neural architecture search method based on a diffusion evolutionary algorithm. By combining the denoising mechanism of the diffusion model with the global search capability of the evolutionary algorithm, a supernet is constructed and network weights and subnet coding are alternately optimized to find the neural network architecture with the best performance.

[0006] Technical solution: A neural architecture search method based on a diffusion evolution algorithm of the present invention comprises the following steps: Step 1: Build a supernet, which includes subnetworks, each of which assigns a weight to each operation on each connecting edge through continuous encoding. The encoding form is A matrix of , where M represents the number of connecting edges in the supernet, and N represents the N candidate operations set on each connecting edge; Step 2: Obtain pictures of different categories to construct a data set, and divide it into a training data set and a validation data set in proportion; Step 3, alternately optimize the supernet weights and subnet codes until the constraints are met, and stop the optimization to obtain the optimal subnet code, and then obtain the neural network architecture; wherein the supernet weights are trained and optimized using the training data set and its labels, and the subnet code is optimized using the diffusion evolution algorithm.

[0007] Furthermore, the optimization of subnet coding using the diffusion evolution algorithm includes the following steps: Step 301, sample and initialize individuals of the population from a normal distribution, and the population is represented as ,in, express dimensional identity matrix, represents the encoding dimension, T represents the total number of iterations, Represents individuals in the population. Each individual includes a subnet code and a scaling factor. The coding dimension of each individual satisfies: , t represents the current iteration number, represents a set of real numbers, and the encoding of the i-th individual at the t-th time step is expressed as , Q represents transpose, represents the encoding method of the Mth connecting edge of the i-th individual at the t-th time step, Indicates that in the encoded information of the i-th individual at the t-th time step, the N-th position corresponds to the value of the N-th candidate operation, represents the lower limit of the value of the Nth candidate operation in the ith individual code at the tth time step, It represents the upper limit of the value of the Nth candidate operation in the ith individual code at the tth time step; represents the scaling factor of the ith individual at the tth time step, is a random number between 0 and 1; Step 302, using the reverse denoising process to update the population individuals, the update formula is: , In the formula, represents cosine scheduling, represents the noise variance, represents noise injection; Step 303, using the validation data set to calculate the fitness value of each individual, the calculation formula is: , Where B represents the number of batches in the validation dataset, Indicates the true label of the b-th sample corresponding to the c-th category, Representation model based on parameters For the bth sample in the validation dataset The probability of predicting category c, C represents the total number of categories; Step 304, compare the fitness values ​​of the i-th individual at the t-th time step and the updated individual, and save the higher one to the next generation population, expressed as: , In the formula, Represents the individuals saved to the next generation, including subnet codes and scaling factors, represents the i-th individual in the t-th time step, Indicates the individual currently traversed The corresponding individuals processed by the diffusion evolution operator are: Represents an individual Subnet code within The fitness value of Represents an individual Subnet code within Adaptability; Step 305, converting the fitness value of the individual into a selection probability through a probability representation function, and using the selection probability to guide the update of the individuals in the population; Step 306, update the current iteration time step , jump to step 302, and stop iterating when the maximum number of iterations is reached.

[0008] Furthermore, the assignment rule for noise injection is: , In the formula, represents the population mean.

[0009] Furthermore, the setting rule of the noise injection ratio is: In the early stages When the noise injection ratio is set to be greater than or equal to ; In the later stages When the noise injection ratio is set to be less than or equal to .

[0010] Furthermore, the probability representation function types include uniform mapping, kernel density estimation, or Gaussian mixture model; Among them, the probability representation function during uniform mapping is expressed as: , The probability representation function for kernel density estimation is expressed as: , In the formula, h represents the bandwidth, represents the jth individual in the population, represents the square of the Euclidean norm; The probability representation function of the Gaussian mixture model is expressed as: , In the formula, represents the weight of the kth Gaussian component in the mixture model, represents the covariance matrix, and the parameters , represents the set of individual indexes belonging to the Kth group, is the number of elements in the set.

[0011] Furthermore, the constraint condition in step 3 is expressed as: Obey the constraints , In the formula, Indicates the subnet code, Indicates the subnet code corresponding to the individual with the highest fitness, represents the supernet weight, Indicates that at a given weight coefficient In the case of, the optimal weight obtained by the supernet after training, represents the validation dataset, Indicates the value of weight W when the loss function value is the lowest, Represents the minimization of the cross entropy loss function in the classification task, Represents the training dataset.

[0012] Furthermore, the encoding matrix is ​​recorded as ,in Represents the weight coefficient of candidate operation n on the connecting edge m, and the formula is: , In the formula, Indicates the number of the connecting edge, The number of the candidate operation. The basic variable used to calculate the weight coefficient of candidate operation n on the connection edge m. It is part of the individual code, which is used to describe the architecture information of the subnet; After the optimization is completed, , the obtained subnet code is discretized, that is, only the code with the largest value after optimization is retained on each connection edge, and the discretized code is converted to 0-1 code. The code corresponding to the operation retained on each connection edge is 1, and the rest are 0. The formula is expressed as: .

[0013] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: 1. The present invention introduces a diffusion evolution algorithm, which compares the evolution process to a denoising process, continuously optimizes individuals in iterations, effectively enhances population diversity, and enables more potential excellent architectures to be explored when searching for neural network architectures, avoiding falling into local optimal solutions due to insufficient population diversity, thereby improving the comprehensiveness and effectiveness of the search; 2. The present invention utilizes the iterative denoising mechanism of the diffusion model to more fully explore the parameter space; 3. The present invention adjusts the noise variance and injects noise, so that in the process of optimizing subnet coding, the search space can be searched extensively or finely at different stages, thereby increasing the probability of finding the optimal neural network architecture and making the search results more reliable and practical; the reverse denoising process and adaptive noise injection mechanism enable the individual to evolve towards a better solution during the evolution process and to jump out of the local optimal area when necessary. This mechanism effectively reduces the risk of the algorithm falling into the local optimal solution, ensures that the search process can continuously approach the global optimal solution, and improves the search efficiency and the performance of the neural network architecture finally obtained; 4. Based on the characteristics of the diffusion evolution algorithm, the present invention constructs a fitness function and deeply studies the impact of different probability distribution representations on search performance, providing a theoretical basis and practical guidance for optimizing algorithm performance, so that the algorithm can be adjusted according to different task requirements and data characteristics, further improving the search effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of the neural architecture search method based on the diffusion evolution algorithm; Figure 2 Flowchart for optimizing subnet coding using diffusion evolution algorithm; Figure 3 This is a schematic diagram of the structure of the super network in the example; Figure 4 Example diagram of subnet encoding during optimization; Figure 5 Schematic diagram of the supernet accuracy obtained through training; Figure 6 Schematic diagram for comparing the method of the present invention with other methods. DETAILED DESCRIPTION

[0015] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0016] The neural architecture search method based on the diffusion evolution algorithm described in this embodiment is as follows: Figure 1 As shown, the method comprises the following steps: Step 1: Build a supernet, which includes subnetworks, each of which assigns a weight to each operation on each connecting edge through continuous encoding. The encoding form is A matrix of , where M represents the number of connecting edges in the supernet, and N represents the N candidate operations set on each connecting edge; Step 2: Obtain pictures of different categories to construct a data set, and divide it into a training data set and a validation data set in proportion; Step 3, alternately optimize the supernet weights and subnet codes until the constraints are met, and stop the optimization to obtain the optimal subnet code, and then obtain the neural network architecture; wherein the supernet weights are trained and optimized using the training data set and its labels, and the subnet code is optimized using the diffusion evolution algorithm.

[0017] A supernet is a hybrid network that integrates all candidate operations and topological connections in the search space; a subnet is a component of a supernet, and its characteristic is that there is only one candidate operation on all connected edges. In this example, a supernet is first constructed, which contains M connected edges, and each connected edge has N candidate operations, such as Dynamic separable convolution, Atrous convolution, Max pooling, skip connections, and no operations are all candidate operations. In the supernet, input information no longer undergoes a single operation, but multiple operations. For example, input information will simultaneously pass through Dynamic separable convolution, Operations such as dilated convolution are then performed, and the feature maps obtained by each of these operations are added together according to the corresponding element positions to generate new mixed feature map information. When using the gradient descent method to train network weights, supernets are crucial because they allow the parameters of all candidate operations in the search space to be trained, in order to provide trained network weights for subsequent evaluation of subnet performance.

[0018] To represent and optimize the subnet, this example uses a In the form of matrix encoding, a continuous value in the range of 0 to 1 is used to assign weights to each operation on each connecting edge. Through this encoding method, the subnet can be discretized into a regular neural network according to the weight size. The specific method is to remove candidate operations and connecting edges with smaller weights. In the process of evolutionary computation, the subnet is treated as an individual in the population. When the encoding presents different values, a variety of different neural networks can be obtained. There is no theoretical limit on the value of N. Any convolution, pooling and other operations can be included. The value of M is related to the internal nodes set in the supernet. If the number of internal nodes in the supernet is H, then In this example, the diffusion evolution algorithm is used to optimize the subnet encoding, with the goal of accurately finding the subnet architecture with the best performance during the search process.

[0019] Specifically, the supernet uses a cell-based search space, which consists of normal cells and dimension reduction cells. Normal cells keep the feature map size unchanged, and dimension reduction cells can use convolution operations with a step size of 2 to halve the output feature map size. Each cell contains The number of connecting edges , encoding dimension The design of the supernet structure provides a basis for the diversification of neural network architectures. Through the combination of different units and the selection of connecting edges and operations, a rich variety of subnet architectures can be generated. Each subnet represents the architecture weight through continuous encoding, and the encoding form is recorded as The matrix ,in Represents the weight coefficient of candidate operation n on the connecting edge m, and the formula is: , In the formula, Indicates the number of the connecting edge, The number of the candidate operation. The base variable used to calculate the weight coefficient of candidate operation n on edge m.

[0020] The set of candidate operations includes, but is not limited to, convolution operations, such as Separable convolution, Separable convolution, Atrous convolution, Atrous convolution, pooling operation such as Max pooling, Average pooling, skip connections, and null operations are encoding methods that can accurately represent different subnetwork architectures, and in the subsequent optimization process, they can be discretized according to the weight size to determine the final neural network architecture.

[0021] In step 2, images of different categories are obtained to construct a data set, and the data set is divided into a training data set and a validation data set in a ratio of 5:5. Taking medical image classification as an example, the images in the training data are labeled with different disease categories, such as normal, disease A, disease B, etc. The supernet needs to be trained first before the weights of various operations inside it can be used. In this example, the Adam optimizer is used to train the training data set. The network weight W of the supernet is trained, the initial learning rate of the Adam optimizer is set to 0.025, and the weight decay is set to The network weight W is continuously adjusted to gradually reduce the difference between the prediction results of the Adam optimizer on the training data and the true label, thus optimizing the performance of the super network. The cross entropy loss function commonly used in classification tasks is used, and its expression is: , In the formula, represents the predicted probability of the Adam optimizer for category c, C represents the total number of categories, Represents the target output or prediction target in the neural network task, Represents the input sample x and its corresponding true label y.

[0022] Furthermore, if Figure 2 As shown in FIG, the optimization of subnet coding using the diffusion evolution algorithm includes the following steps: Step 301, sample and initialize individuals of the population from a normal distribution, and the population is represented as ,in, for dimensional identity matrix, represents the encoding dimension, T represents the total number of iterations, Represents individuals in the population. Each individual includes a subnet code and a scaling factor. The coding dimension of each individual satisfies: , t represents the current iteration number, represents a set of real numbers, and the encoding of the i-th individual at the t-th time step is expressed as , Q represents transpose, represents the encoding method of the Mth connecting edge of the i-th individual at the t-th time step, In the encoded information of the i-th individual at the t-th time step, the N-th position corresponds to the value of the N-th candidate operation. represents the lower limit of the value of the Nth candidate operation in the ith individual code at the tth time step, represents the upper limit of the value of the Nth candidate operation in the ith individual code at the tth time step, represents the scaling factor of the ith individual at the tth time step, is a random number between 0 and 1; Step 302, using the reverse denoising process to update the population individuals, the update formula is: , In the formula, represents cosine scheduling, represents the noise variance, represents noise injection; Step 303, using the validation data set to calculate the fitness value of each individual, the calculation formula is: , Where B represents the number of batches in the validation dataset, Indicates the true label of the b-th sample corresponding to the c-th category, Indicates that based on parameters For the bth sample in the validation dataset The probability of predicting category c, C represents the total number of categories; Step 304, compare the fitness values ​​of the i-th individual at the t-th time step and the updated individual, and save the higher one to the next generation population, expressed as: , In the formula, Represents the individuals saved to the next generation, including subnet codes and scaling factors, represents the i-th individual in the t-th time step, Indicates the individual currently traversed The corresponding individuals processed by the diffusion evolution operator are: Represents an individual Subnet code within The fitness value of Represents an individual Subnet code within Adaptability; Step 305, converting the fitness value of the individual into a selection probability through a probability representation function, and using the selection probability to guide the update of the individuals in the population; Step 306, update the current iteration time step , jump to step 302, and stop iterating when the maximum number of iterations is reached.

[0023] Furthermore, the assignment rule for noise injection is: , In the formula, represents the population mean.

[0024] In the above assignment rule, the intensity of noise injection is dynamically adjusted according to the degree of dispersion of individuals in the population, where the degree of dispersion can be determined by calculating the distance between the individual and the population mean. is a parameter that controls the overall intensity of the noise, and e is the random noise sampled from the standard normal distribution. When the population individuals are relatively concentrated, that is, In order to increase population diversity and allow the algorithm to escape from the local optimum, the random noise will be adjusted so that it does not exceed ; When the individual distribution is dispersed, save the random noise as , and further dynamically adjust the noise injection according to the distribution of individuals in the population. The noise is regulated from the perspective of macro time steps, and e is combined with the individual states of the population to make more detailed noise adjustments. The two complement each other and serve the optimization process of the algorithm together.

[0025] Furthermore, the setting rule of the noise injection ratio is: In the early stages When the noise injection ratio is set to be greater than or equal to ; In the later stages When the noise injection ratio is set to be less than or equal to .

[0026] In this example, the current time step t is determined if The noise variance can be calculated when , then set the noise injection ratio to be greater than or equal to Specifically, in the early stage, if the noise variance , then through ensure , so the noise injection ratio is greater than or equal to , allowing the algorithm to find potential solutions more efficiently in the large space search phase; if set up , the noise injection ratio is less than or equal to , specifically, through Make , the noise injection ratio is less than or equal to .

[0027] After updating the population individuals, it is necessary to calculate the fitness value of each individual and convert the selection probability, using the validation data set Evaluate the performance of each individual, i.e., the candidate architecture. The model processes the current batch of input data to obtain a prediction result, calculates the difference between the prediction result and the true label through the loss function, and takes the negative value of the difference as the fitness value of the individual in the current batch. After all batch calculations are completed, if a valid fitness value is obtained, the fitness values ​​of multiple batches are averaged to obtain the final fitness value of each individual; if no valid fitness value is obtained, an empty tensor is returned. In practical applications, such as image classification tasks, the model performs classification predictions on images in the validation set, calculates the difference between the prediction result and the true label through the cross entropy loss function, and uses its negative value as the basis for calculating the fitness value. In general, the fitness function is defined as the negative validation loss, and the formula is ,in, represents the fitness value of the i-th individual, B is the number of batches in the validation set, L is the loss function, and are the input data and labels of the bth batch, is the parameter of the model in the batch. Compare the fitness values ​​of the i-th individual and the mutated individual at the t-th time step, and the higher one is saved in the next generation population.

[0028] Individual fitness values ​​can be converted into selection probabilities through different selection probability representation functions according to different application scenarios and needs, so as to comply with the data type of the diffusion evolution algorithm and the principle of individual update by the diffusion evolution algorithm. The direct mapping method is used as the fitness mapping function, and the value calculated by the individual fitness value through different selection probability representation functions is directly used as its selection probability. The unique probabilistic reasoning of the diffusion evolution algorithm is used to further guide the optimization process of population coding, and determine which individuals are more likely to participate in the evolution of the next generation, so as to facilitate individual update through the diffusion evolution algorithm.

[0029] Furthermore, the probability representation function types include uniform mapping, kernel density estimation, or Gaussian mixture model; Among them, the probability representation function during uniform mapping is expressed as: , The probability representation function for kernel density estimation is expressed as: , In the formula, h represents the bandwidth, represents the square of the Euclidean norm; The probability representation function of the Gaussian mixture model is expressed as: , In the formula, represents the weight of the kth Gaussian component in the mixture model, represents the covariance matrix, and the parameters , represents the set of individual indexes belonging to the Kth group, is the number of elements in the set.

[0030] When the probability representation function type is uniform mapping, in the process of updating individuals, it is assumed that the probability of individuals in the population being selected follows a uniform distribution, that is, the probability of each individual being selected to participate in the next generation of evolution is When selecting individuals for updating, individuals are randomly selected from the current population according to this probability, and then participate in the individual update process.

[0031] When the probability representation function type is kernel density estimation, select the probability function , where bandwidth , dimension . In updating individual When the probability representation function is used to measure the individual With other individuals , which in turn affects the probability of an individual being selected to participate in the update. hour, The probability of selection is given by the selection probability function Determine so that Individuals with high similarity have a greater probability of participating in In the calculation of , the individual update formula can be expressed as: , In the formula, is based on The individual noise term selected by probability is related to e sampled from a normal distribution, but the selection process is subject to Regulation, if Calculate an individual With the current individual The higher the similarity, the more likely the noise item corresponding to the individual is to be selected. , It carries noise information related to the selected individuals, affecting the direction and degree of individual updating.

[0032] When the probability representation function type is a Gaussian mixture model, the selected probability function is: , In the formula, the number of components , the covariance matrix is a diagonal matrix. hour To calculate individual The probability of belonging to different Gaussian distributions determines the probability of an individual being selected to participate in the update. When, according to Probability selects different Gaussian components, and the parameters corresponding to different components , Individuals that will affect calculation The choice of and the calculation of the noise term, where , Diagonal elements , represents the set of individual indexes belonging to the Kth group, is the number of elements in the set, and the individual update formula can be expressed as: , in, is based on The probability selection is related to the noise term, which is related to e sampled from a normal distribution, but the selection process is subject to Regulation, if Calculate individual is more likely to belong to a Gaussian component, then the noise term corresponding to that component will be selected , The Gaussian mixture model incorporates considerations of individual states and probability distributions, and plays a regulatory role in individual updates.

[0033] This example adopts the method of alternating optimization of network weights and subnet codes until the error rate of the network architecture in the population converges. For example, the supernet weight update cycle can be set to perform a complete training every 5 generations of evolution. At each update, the supernet weight is trained using the training data set to update the network weight W, providing a more accurate evaluation basis for the evolution of subnet codes; the subnet code evolution cycle is set to perform 100 reverse denoising iterations per generation, and a better neural network architecture is explored by continuously updating the subnet code. The training data set is divided into two parts, respectively recorded as and , represents the validation dataset, represents the training data set, where represents the data used to train the weights of the supernet network, Indicates the data for evaluating the performance of the subnet in the process of optimizing the subnet encoding. At the same time, an early stopping mechanism is set. When the verification error rate on the verification set decreases by less than 10 generations in a row The optimization process is terminated when .

[0034] Furthermore, the constraint condition in step 3 is expressed as: Obey the constraints , In the formula, Indicates the subnet code, Indicates the subnet code corresponding to the individual with the highest fitness, represents the supernet weight, Indicates that at a given weight coefficient In the case of, the optimal weight obtained by the supernet after training, represents the validation dataset, Indicates the value of weight W when the loss function value is the lowest, Represents the minimization of the cross entropy loss function in the classification task, Represents the training dataset.

[0035] After the optimization is completed, , the obtained subnet code is discretized, that is, only the code with the largest value after optimization is retained on each connection edge, and the discretized code is converted to 0-1 code. The code corresponding to the operation retained on each connection edge is 1, and the rest are 0. The formula is expressed as: .

[0036] The superiority of the neural architecture search method based on the diffusion evolution algorithm described in the present invention is further illustrated by the following examples.

[0037] Build as Figure 3 The supernet shown in Figure 1 contains 4 nodes representing feature graphs. There is a sequential relationship between the nodes. Feature graphs can only be passed from the previous order nodes to the next order nodes after processing. X1, X2, X3, and X4 represent different feature graphs. The three connecting edges of the supernet represent different candidate operation processing, such as express convolution, express convolution, represents a skip connection, Figure 4 This is an example diagram of subnet encoding during the optimization process. Figure 4 The upper part of the table is composed of multiple two-dimensional tables, each of which represents the coding status of a subnet individual. The horizontal header of the table is the candidate operation , , , indicating that there are three optional operations on the connected edges of the supernet. Figure 4 The lower middle part is a directed graph corresponding to the table in the upper part. The directed edges between the nodes in the graph represent the connection relationship. Edges of different colors, such as blue, orange, and green, represent different connection paths. These paths are actually determined by the encoding weights in the table above. Each directed graph represents a subnet architecture, showing the direction of data or information flow in the subnet. Taking the processing of input data on node 0 in the supernet as an example, the feature maps obtained by the three candidate operations processing the input information will be weighted and summed according to the encoded weights, and the resulting feature map information is stored on node 1. For subsequent nodes such as node 2 and node 3, they obtain feature map information from multiple preceding nodes. At this time, multiple feature map information needs to be spliced ​​in the channel dimension. In practical applications, the supernet adopts a unit-based search space, which consists of normal units and dimensionality reduction units. Each unit contains Intermediate nodes, connected edges , the number of candidate operations on each edge in the supernet is 8, and the encoding dimension By alternately optimizing the network weights and subnet encodings through the method described in the present invention, the operation with the largest weight on each connection edge is retained, and the neural network architecture suitable for a specific task is finally obtained. Finally, the optimal subnet is obtained. Specifically expressed as: ,in, Indicates that the convolution kernel size is Dynamic separable convolution, Indicates that the convolution kernel size is Dynamic separable convolution, Indicates that the convolution kernel size is The dilated convolution, Indicates that the convolution kernel size is The dilated convolution, Indicates the size The maximum pooling of represents a skip connection, Indicates the size Average pooling.

[0038] The performance of the neural network architecture optimized by the method of the present invention was tested on the CIFAR-100 public dataset, which has 100 categories and 600 images in each category. Using the method of the present invention on GPU 4090, a high accuracy neural network architecture can be found. The neural network architecture of Specifically, we first construct a supernet, then sample an operation from each edge in the supernet to obtain a subnet, and repeat the sampling steps. The initial population is obtained by training. The final super network accuracy is as follows: Figure 5 shown.

[0039] In the process of alternately optimizing the supernet network weights and the population individual codes, the present invention compares the average verification accuracy of all individuals in each generation with other methods, such as Figure 6 As shown, compared with the premature convergence problem caused by the small model trap of other methods, the method of the present invention can converge normally to a lower verification error rate. It further shows that the excellent global search ability of the diffusion evolution operator of the present invention can alleviate the problem of existing methods falling into local optimality during the search process.

Claims

1. A neural architecture search method based on diffusion evolution algorithm, characterized in that: The steps include: Step 1: Build a supernet, which includes subnets, each of which assigns a weight to each operation on each edge through continuous encoding. The encoding form is an M×N matrix, where M represents the number of edges in the supernet and N represents the N candidate operations set on each edge. Step 2: Obtain pictures of different categories to construct a data set, and divide it into a training data set and a validation data set in proportion; Step 3, alternately optimize the supernet weights and subnet codes until the constraints are met, stop the optimization, and obtain the optimal subnet code, and then obtain the neural network architecture; The training data set and its labels are used to train and optimize the supernet weights, and the diffusion evolution algorithm is used to optimize the subnet coding.

2. According to claim 1, a neural architecture search method based on a diffusion evolution algorithm is characterized in that: The optimization of subnet coding using the diffusion evolution algorithm includes the following steps: Step 301, sample and initialize individuals of the population from a normal distribution, and the population is represented as ,in, express dimensional identity matrix, represents the encoding dimension, T represents the total number of iterations, Represents individuals in the population. Each individual includes a subnet code and a scaling factor. The coding dimension of each individual satisfies: , t represents the current iteration number, represents a set of real numbers, and the encoding of the i-th individual at the t-th time step is expressed as , Q represents transpose, represents the encoding method of the Mth connecting edge of the i-th individual at the t-th time step, Indicates that in the encoded information of the i-th individual at the t-th time step, the N-th position corresponds to the value of the N-th candidate operation, represents the lower limit of the value of the Nth candidate operation in the ith individual code at the tth time step, It represents the upper limit of the value of the Nth candidate operation in the ith individual code at the tth time step; represents the scaling factor of the ith individual at the tth time step, is a random number between 0 and 1; Step 302, using the reverse denoising process to update the population individuals, the update formula is: ， In the formula, represents cosine scheduling, represents the noise variance, represents noise injection; Step 303, using the validation data set to calculate the fitness value of each individual, the calculation formula is: ， Where B represents the number of batches in the validation dataset, Indicates the true label of the b-th sample corresponding to the c-th category, Representation model based on parameters For the bth sample in the validation dataset The probability of predicting category c, C represents the total number of categories; Step 304, compare the fitness values ​​of the i-th individual at the t-th time step and the updated individual, and save the higher one to the next generation population, expressed as: ， In the formula, Represents the individuals saved to the next generation, including subnet codes and scaling factors, represents the i-th individual in the t-th time step, Indicates the individual currently traversed The corresponding individuals processed by the diffusion evolution operator are: Represents an individual Subnet code within The fitness value of Represents an individual Subnet code within Adaptability; Step 305, converting the fitness value of the individual into a selection probability through a probability representation function, and using the selection probability to guide the update of the individuals in the population; Step 306, update the current iteration time step , jump to step 302, and stop iterating when the maximum number of iterations is reached.

3. The neural architecture search method based on diffusion evolution algorithm according to claim 2 is characterized in that: The assignment rule for noise injection is: ， In the formula, represents the population mean.

4. The neural architecture search method based on diffusion evolution algorithm according to claim 3 is characterized in that: The setting rule of noise injection ratio is: In the early stages When the noise injection ratio is set to be greater than or equal to ; In the later stages When the noise injection ratio is set to be less than or equal to .

5. The neural architecture search method based on diffusion evolution algorithm according to claim 4 is characterized in that: Probabilistic representation function types include uniform mapping, kernel density estimation, or Gaussian mixture models; Among them, the probability representation function during uniform mapping is expressed as: ， The probability representation function for kernel density estimation is expressed as: ， In the formula, h represents the bandwidth, represents the jth individual in the population, represents the square of the Euclidean norm; The probability representation function of the Gaussian mixture model is expressed as: ， In the formula, represents the weight of the kth Gaussian component in the mixture model, represents the covariance matrix, and the parameters , represents the set of individual indexes belonging to the Kth group, Indicates the number of elements in this collection.

6. The neural architecture search method based on diffusion evolution algorithm according to claim 5, characterized in that: The constraints in step 3 are expressed as: Obey the constraints , In the formula, Indicates the subnet code, Indicates the subnet code corresponding to the individual with the highest fitness, represents the supernet weight, Indicates that at a given weight coefficient In the case of, the optimal weight obtained by the supernet after training, represents the validation dataset, Indicates the value of weight W when the loss function value is the lowest, Represents the minimization of the cross entropy loss function in the classification task, Represents the training dataset.

7. The neural architecture search method based on diffusion evolution algorithm according to claim 6, characterized in that: The encoding matrix is ​​recorded as ,in Represents the weight coefficient of candidate operation n on the connecting edge m, and the formula is: ， In the formula, Indicates the number of the connecting edge, The number of the candidate operation. The basic variable used to calculate the weight coefficient of candidate operation n on the connection edge m; after the optimization is completed, let , the obtained subnet code is discretized, that is, only the code with the largest value after optimization is retained on each connection edge, and the discretized code is converted to 0-1 code. The code corresponding to the operation retained on each connection edge is 1, and the rest are 0. The formula is expressed as: 。

Images