Image classification neural network architecture searching method based on social evolution algorithm

Abstract

An image classification neural network architecture searching method based on a social evolution algorithm. The method comprises: constructing an image classification neural network; executing in the image classification neural network an individual screening operation, a group organization tackling operation, an individual innovation operation, a failure tolerance mechanism and a development resource supply operation; searching for an optimal neural network architecture; and using the calculated image classification neural network to perform image classification.

Description

A neural network architecture search method for image classification based on social evolution algorithm Technical Field

[0001] The present invention relates to the field of image classification neural network architecture search, and in particular to an image classification neural network architecture search method based on a social evolution algorithm. Background Art

[0002] Deep neural networks (DNNs) have achieved tremendous success in many applications. The architecture of a DNN plays a crucial role in its performance and is typically manually designed using extensive expertise. However, designing an excellent network structure requires significant time and resources. Even experienced researchers often need to conduct repeated trials and adjustments, which consumes significant computational resources and time.

[0003] Artificially designed neural network structures may be limited by the designer's ideas and experience, and cannot fully explore the vast space of network structures. With the development of deep learning technology, network structures have become more complex, and even experienced neural network architecture researchers have difficulty coping with complex structures and components.

[0004] Neural network architecture search utilizes algorithms and optimization techniques to automatically search for optimal network structures, reducing the time and cost of manual network design. Automating the process of manually designing, implementing, and tuning neural networks allows for automated exploration of a wide range of network architectures and can find optimal solutions for specific tasks or datasets, alleviating the burden of manual network design. Neural network architecture search can more comprehensively search the space of network structures, identifying structures that are difficult to design manually, thus promoting innovation in auditing network architecture design. Overall, the emergence of neural network architecture search makes network design more intelligent and efficient, providing better solutions for solving practical problems. Technical issues

[0005] The present invention provides a neural network architecture search method based on a social evolution algorithm, which includes individual screening operations, organizational problem-solving operations, individual innovation operations, failure tolerance mechanisms, and resource supply operations. Technical Solutions

[0006] To achieve the above objectives, the present invention provides a method for searching image classification neural network architectures based on a social evolutionary algorithm, the method comprising:

[0007] Step 1: Construct an image classification neural network. The main part of the image classification neural network includes multiple neural network building blocks, each of which is a convolutional neural network.

[0008] Step 2: Use a string encoding to represent the solution to the neural network architecture search problem. The encoding consists of multiple substrings, each of which contains the name of the neural network block and a number enclosed in a pair of brackets. The numbers in the brackets represent the number of input channels, number of output channels, step size, and number of additional sublayers of the neural network block respectively.

[0009] Step 3: Randomly generate a group Π containing |Π| individuals. The code of each individual in the group corresponds to the code of the entire image classification neural network.

[0010] Step 4: Calculate the fitness of each individual;

[0011] Step 5: Set up a new individual set as ;

[0012] Step 6: Set i=1;

[0013] Step 7: Implement organizational research operations:

[0014] Step 7.1: Randomly select α individuals from the population Π to form a problem-solving team;

[0015] Step 7.2: Perform a fusion operation on the α individuals in the research team. Each individual takes a coded substring in order. That is, the first individual takes the first substring, the second individual takes the second substring, and so on. The αth individual takes the αth substring. The α substrings are fused to form a new individual q.

[0016] Step 7.3: Calculate the fitness of the new individual q. If the fitness of q is greater than that of some individuals in the population P, replace the one with the worst fitness among these individuals with q.

[0017] Step 7.4: Repeat steps 7.1 and 7.3 for β times;

[0018] Step 8: Implement individual innovation operations:

[0019] Step 8.1: Randomly select an individual p from Р;

[0020] Step 8.2: Implement innovative exploration operations on p:

[0021] Step 8.2.1: Randomly select a substring position in the code of individual p;

[0022] Step 8.2.2: Randomly extract a neural network block name from the search space to replace the part of the original substring that represents the neural network block name;

[0023] Step 8.2.3: Explore the output channels represented in the atomic string;

[0024] Step 8.2.4: Explore the number of additional sublayers represented in the atom string;

[0025] Step 8.2.5: Create a new individual q and calculate its fitness score. If q's fitness score is greater than the scores of some individuals in the population P, replace the worst of these individuals with q. Otherwise, execute the failure tolerance mechanism and continuously perform γ neural network architecture encoding innovation operations until a new individual q is found that is better than some individuals in the population P, and replace the worst of these individuals with q.

[0026] Step 8.3: Repeat steps 8.1 and 8.2 Second-rate;

[0027] Step 9: Provide new individuals with development resource mechanisms based on the double triangle mechanism;

[0028] Step 9.1: Let r = i / ε, where ε is a constant set according to the situation;

[0029] Step 9.2: If r is an integer and the number of individuals in the population is greater than the threshold η, then execute steps 9.3 and 9.4;

[0030] Step 9.3: Remove the worst θ*r individuals from the existing population Р, where θ is an artificially set benchmark, which is multiplied by r to control the number of individuals to be removed;

[0031] Step 9.4: Randomly generate θ*r new solutions and add them to the existing population Р;

[0032] Step 10: Perform the next iteration, i.e. return to step 7, and increase the number of iterations i by 1;

[0033] Step 11: If the number of iterations i = σ, where σ represents the maximum evolutionary generation or τ consecutive generations do not improve the highest fitness of individuals in population P, then stop the calculation. Otherwise, go to step 6. The value of τ is set artificially according to the complexity of the problem and the experiment.

[0034] Step 12: Use the image classification neural network at the time of calculation stop to perform image classification.

[0035] Furthermore, the fitness is calculated as follows:

[0036] S1: Remove all residual links in the image classification neural network;

[0037] S2: Pass Initialize all neurons in the image classification neural network, represents the standard normal distribution;

[0038] S3: Sampling Respectively represent a tensor filled with random numbers in a standard normal distribution with a mean of 0 and a variance of 1, used as input and perturbation respectively;

[0039] S4: Calculation ; represents the finite differential of the characteristic map, represents the perturbation coefficient, Indicates taking the mean value, Indicates taking the absolute value, represents the main part of F, that is, F without the tail adaptive average pooling layer, flatten layer and fully connected layer. F represents the image classification neural network;

[0040] S5: For the i-th BN layer with m output channels, calculate ,in is the mini-batch standard deviation statistic of the j-th channel in the BN layer; represents the standard deviation of the BN layer, and m represents the number of output channels of the BN layer;

[0041] S6: ;in, Represents the fitness score of the final classification neural network. Beneficial effects

[0042] Starting from step 9, the evolutionary population reduces its size every generation, retaining the most outstanding individuals. The reduction in size becomes larger and larger as the evolutionary process progresses. At the same time, each time the population size is reduced, new individuals of the same size are added. These new individuals provide new genetic resources, which mix with the existing excellent genes to provide a hybrid force for social development.

[0043] Compared to traditional neural network architecture search methods, this invention primarily improves the speed and quality of image neural network architecture search. Regarding speed: Traditional image classification methods require training the searched neural network architecture to obtain fitness, and neural network training generally takes a long time to obtain a clear evaluation metric. This invention uses a new fitness calculation method that does not require image neural network training, significantly reducing the time required for neural network architecture search.

[0044] Regarding quality, this invention utilizes a social evolutionary algorithm and incorporates individual screening, group-organized problem-solving, individual innovation, a failure tolerance mechanism, and development resource provision. These processes retain high-performing individuals and eliminate underperforming ones, thereby improving overall population quality. The failure tolerance mechanism and development resource provision expand the search space, preventing the algorithm's final results from falling into local optima, which can compromise solution quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] FIG1 is a flow chart of the social evolution algorithm process of the method of the present invention.

[0046] FIG2 is a schematic diagram of the encoding method of the present invention.

[0047] FIG3 is a schematic diagram of the random initialization part in the social evolution algorithm of the method of the present invention.

[0048] FIG4 is a schematic diagram of the fusion operation in the social evolution algorithm process of the method of the present invention.

[0049] FIG5 is a schematic diagram of the innovative exploration operation during the social evolution algorithm of the method of the present invention.

[0050] FIG6 is a flowchart of the individual removal operation in step 9.3 of the social evolution algorithm process of the method of the present invention. Best Mode for Carrying Out the Invention

[0051] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings. The following embodiments or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0052] Build an image classification neural network. This neural network consists of multiple neural network building blocks, each of which is a convolutional neural network. For example, a SuperConvK3BNRELU neural network block contains a two-dimensional convolutional layer, a batch normalization layer, and a Reluctance Unified Unit (RELU) nonlinear activation layer. These three layers are connected in sequence to form a SuperConvK3BNRELU neural network block. Following this main structure, an adaptive average pooling layer, a flattening layer, and a fully connected layer are added to convert the two-dimensional feature map output by the network into a one-dimensional vector for image classification.

[0053] The seed strings SuperConvK3BNRELU(3,8,1,1)SuperResK3K3(8,16,1,8,1)SuperConvK1BNRELU(16,128,1,1) were used to randomly generate the initial population, and the social evolution algorithm of the present invention was used for iteration. After 22 minutes of iteration, the population P was obtained. The highest individual fitness in the population P was 135.127, and the lowest fitness was 116.039. For the individual with a fitness score of 135.127, the resulting classification neural network architecture is SuperConvK3BNRELU(3,24,1,1), SuperResK1K7K1(24,320,1,16,5), SuperResK1K5K1(320,160,1,32,3), SuperResK1K5K1(160,160,1,32,4), SuperResK1K7K1(160,48,1,16,4), and SuperConvK1BNRELU(48,320,1,1). This architecture is trained on the CIFAR 100 dataset. After 1440 epochs, the model with the best validation result is selected and saved as the model file.

[0054] After saving, read the model data and perform the image classification task on the cifar 100 dataset. The results are as follows:

[0055] ACC1 scoreACC5 scoreNumber of recognized images (pieces)Total recognition time (s)Average time spent on each image (ms)81.27999%96.06999%100005.821490.582149

[0056] Figure 2 is a schematic diagram of the encoding method of the present invention. The encoding SuperConvK3BNRELU(3,24,1,1) represents a substring of the neural network structure, where the name SuperConvK3BNRELU represents a neural network block in the search space. As shown in Figure 2, this block has three sub-components: the Conv2d layer, the Batch Normalization layer, and the RELU layer. These three layers are interconnected to form the neural network block. (3,24,1,1) represent the relevant parameters of the neural network block, namely the number of input channels, number of output channels, stride length, and number of additional sub-layers. This encoding method allows a neural network block to be represented by a single string.

[0057] Figure 3 is a schematic diagram of the random initialization part of the social evolution algorithm of the method of the present invention. When the number of individuals is less than the specified number of the population, the present invention randomly mutates the seed string and adds a new substring to the original string to generate a new neural network structure, thereby randomly initializing the population.

[0058] Figure 5 illustrates the innovative exploration operation during the social evolution algorithm of the present invention. A substring position in the code is randomly selected, and the innovative exploration operation primarily explores the neural network block name, neural network block output channels, and number of additional sublayers within that substring. A neural network block name is randomly selected from the search space to replace the portion of the original substring representing the neural network block name. The numbers representing the neural network block output channels in the substring are multiplied by 1, 1.25, 1.5, 2, and 2.5, and then divided by 1.25, 1.5, 2, and 2.5, respectively. These resulting numbers are added to a set, and a number is randomly selected from this set to replace the number representing the output channels in the atomic string. The number representing the number of additional sublayers in the substring is multiplied by +1, +2, -1, -2, and +0, respectively. These resulting numbers are added to a set, and a number is randomly selected from this set to replace the number representing the number of additional sublayers in the atomic string.

[0059] Figure 6 is a flowchart of the individual removal operation in step 9.3 of the social evolution algorithm process of the method of the present invention. In order to maintain the stability of the population size and gradually eliminate the neural network structure with low fitness, the present invention selects a part of the individuals in the population for elimination. First, the difference between the number of individuals in the current population and the specified population size is calculated and the absolute value is taken to obtain the number m. Then, the 2*m structures with the lowest fitness in the population are found, and then the m structures with the longest survival time are found in these 2*m structures for elimination. This approach is to eliminate individuals with low fitness while retaining some relatively new structures, thereby avoiding the algorithm from falling into local optimality and affecting the final search results.

[0060] Table 1 is a comparison of the effects of the method of the present invention and the reference paper

[0061] Paper 1: Lin, Ming, Pichao Wang, Zhenhong Sun, Hesen Chen, Xiuyu Sun, Qi Qian, Hao Li, and Rong Jin. "Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition." arXiv, August 22, 2021.

[0062] Paper 2: Abdelfattah, Mohamed S., Abhinav Mehrotra, Łukasz Dudziak, and Nicholas D. Lane. “Zero-Cost Proxies for Lightweight NAS.” arXiv, March 19, 2021.

[0063] Paper 3: Chen, Wuyang, Xinyu Gong, and Zhangyang Wang. “Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective.” arXiv, March 15, 2021.

[0064] Paper 4: Mellor, Joseph, Jack Turner, Amos Storkey, and Elliot J. Crowley. “Neural Architecture Search without Training”. arXiv, June 11, 2021.

Claims

1. A method for searching neural network architectures for image classification based on social evolution algorithm. include: Step 1: construct an image classification neural network, the main part of which includes multiple neural network building blocks, each of which is a convolutional neural network; Step 2: Use a string code to represent the solution to the neural network architecture search problem. The code consists of multiple substrings. Each substring contains the name of the neural network block and a number enclosed in a pair of brackets. The numbers in the brackets represent the number of input channels, the number of output channels, the step size, and the number of additional sublayers of the neural network block. Step 3: Randomly generate a group Р containing |Р| individuals, and the code of each individual in the group corresponds to the code of the entire image classification neural network; Step 4: Calculate the fitness of each individual; Step 5: Set a new individual set as ； Step 6: Set i=1; Step 7: Implement organizational research operations: Step 7.1: Randomly select α individuals from the population Р to form a problem-solving team; Step 7.2: Perform a fusion operation on the α individuals in the research team. Each individual takes a coded substring in order. That is, the first individual takes the first substring, the second individual takes the second substring, and so on. The αth individual takes the αth substring, and the α substrings are fused to form a new individual q. Step 7.3: Calculate the fitness of the new individual q. If the fitness of q is greater than that of some individuals in the population P, then use q to replace the one with the worst fitness among these individuals. Step 7.4: Repeat steps 7.1 and 7.3 for β times; Step 8: Implement individual innovation operations: Step 8.1: Randomly select an individual p from Р; Step 8.2: Implement innovative exploration operations on p: Step 8.2.1: Randomly select a substring position in the code of individual p; Step 8.2.2: Randomly extract a neural network block name in the search space to replace the part of the original substring representing the name of the neural network block; Step 8.2.3: Explore the output channels represented in the atomic string; Step 8.2.4: Explore the number of additional sublayers represented in the atom string; Step 8.2.5: Form a new individual q, calculate the fitness score of the new individual q, if the fitness score of q is greater than the scores of some individuals in the population P, then use q to replace the worst of these individuals, otherwise execute the failure tolerance mechanism, and continuously perform γ neural network architecture encoding change innovation operations until a new individual q that is better than some individuals in the population P is found, and use q to replace the worst of these individuals; Step 8.3: Repeat Steps 8.1 and 8.2 Second-rate; Step 9: Provide new individuals with development resource mechanisms based on the double triangle mechanism; Step 9.1: Let r = i / ε, where ε is a constant set according to the situation; Step 9.2: If r is an integer and the number of individuals in the population is greater than the threshold η, then execute steps 9.3 and 9.4; Step 9.3: Remove the worst θ*r individuals from the existing population Р, where θ is an artificially set benchmark, and the number of individuals to be removed is controlled by multiplying it with r; Step 9.4: Randomly generate θ*r new solutions and add them to the existing population Р; Step 10: Perform the next iteration, that is, return to Step 7, and increment the iteration count i by 1; Step 11: If the iteration count i = σ, where σ represents the maximum number of evolutionary generations or there has been no improvement in the highest fitness of individuals in population P for τ consecutive generations, then stop the calculation; otherwise, go to Step 6. The value of τ is set artificially according to the problem complexity and experiments; Step 12: Use the image classification neural network at the end of the calculation to perform image classification.

2. A method for searching an image classification neural network architecture based on a social evolution algorithm as claimed in claim 1, characterized in that the calculation method of the fitness is as follows: S1: Remove all residual connections in the image classification neural network; S2: Through Initialize all neurons in the image classification neural network, represents the standard normal distribution; S3: Sampling respectively represent tensors filled with random numbers from a standard normal distribution with a mean of 0 and a variance of 1, and are used as input and perturbation respectively; S4: Calculate ； Denote the finite difference of the feature map, Indicates the perturbation coefficient, Indicates taking the mean value, Indicates taking the absolute value, represents the main body of F, that is, the part of F after removing the adaptive average pooling layer, flatten layer and fully connected layer at the end. F represents the image classification neural network; S5: For the i-th BN layer with m output channels, calculate , among which is the mini-batch standard deviation statistic of the j-th channel in the BN layer; represents the standard deviation of this BN layer, and m represents the number of output channels of this BN layer; S6: ; among them, represents the fitness score of the finally obtained classification neural network.

Images