An image classification neural network architecture search method based on a social evolution algorithm

Abstract

The application discloses a neural network architecture search method based on a social evolution algorithm, relates to the field of image classification neural network architecture search, and improves the image classification network architecture design efficiency and facilitates the discovery of more complex and efficient neural network architectures. Through analysis of the experience of human social development, the individual quality of a continuously developing and optimizing society should be better and better, and a social evolution algorithm is proposed by referring to social development to solve image classification neural network architecture design. Individual screening operation, group organization attack operation, individual innovation operation, failure tolerance mechanism and development resource supply operation in the image classification neural network are designed. The method does not need to train the image neural network, thereby greatly saving the neural network architecture search time.

Description

Technical Field

[0001] This invention relates to the field of image classification neural network architecture search, and specifically to an image classification neural network architecture search method based on a social evolution algorithm. Background Technology

[0002] Deep neural networks (DNNs) have achieved tremendous success in many applications. The architecture of a deep neural network plays a crucial role in its performance and is typically designed manually using extensive expertise. However, designing excellent network architectures requires significant time and resources; even experienced researchers need to experiment and fine-tune them, consuming substantial computational resources and time.

[0003] Manually designed neural network architectures may be limited by the designer's ideas and experience, failing to fully explore the vast network structure space. With the development of deep learning technology, network structures have become increasingly complex, making it difficult even for experienced neural network architecture researchers to handle intricate structures and components.

[0004] Neural network architecture search (NNAS) can automatically search for optimal network structures using algorithms and optimization techniques, reducing the time and cost of manual network design. It automates the process of manually designing, implementing, and tuning neural networks, allowing for the automated exploration of numerous network architectures and finding optimal solutions for specific tasks or datasets, thus alleviating the burden of manual network design. NNAS can more comprehensively search the network structure space, identifying structures that are difficult to design manually, which is beneficial for promoting innovation in network architecture design. Overall, the emergence of NNAS makes network design more intelligent and efficient, providing better solutions for solving practical problems. Summary of the Invention

[0005] This invention provides a neural network architecture search method based on social evolution algorithms, which includes individual selection operation, organizational problem-solving operation, individual innovation operation, failure tolerance mechanism, and resource supply operation.

[0006] To achieve the above objectives, this invention provides a method for searching image classification neural network architectures based on social evolution algorithms, the method comprising:

[0007] Step 1: Construct an image classification neural network. The main body of this image classification neural network includes multiple neural network structure blocks, each of which is a convolutional neural network.

[0008] Step 2: Represent the solution to the neural network architecture search problem using string encoding. The encoding consists of multiple substrings, each containing the name of the neural network block and a number enclosed in parentheses. The numbers in the parentheses represent the number of input channels, the number of output channels, the stride, and the number of additional sublayers for that neural network block, respectively.

[0009] Step 3: Randomly generate a population Р containing |Р| individuals, where the code of each individual in the population corresponds to the code of an entire image classification neural network;

[0010] Step 4: Calculate the fitness of each individual;

[0011] Step 5: Set a new set of individuals as Ω = φ;

[0012] Step 6: Let i = 1;

[0013] Step 7: Implement organizational efforts to address the problem.

[0014] Step 7.1: Randomly select α individuals from the population Р to form a task force;

[0015] Step 7.2: Perform a fusion operation on the α individuals in the research team. Each individual takes one encoded substring in sequence, that is, the first individual takes the first substring, the second individual takes the second substring, and so on, until the αth individual takes the αth substring. The α substrings are merged to form a new individual q.

[0016] Step 7.3: Calculate the fitness of the new individual q. If the fitness of q is greater than that of some individuals in the population P, then replace the individual with the worst fitness among these individuals with q.

[0017] Step 7.4: Repeat steps 7.1 and 7.3 β times;

[0018] Step 8: Implement individual innovation operations:

[0019] Step 8.1: Randomly select an individual p from Р;

[0020] Step 8.2: Perform innovative exploration operations on p:

[0021] Step 8.2.1: Randomly select a substring position in the individual p encoding;

[0022] Step 8.2.2: Randomly extract a neural network block name from the search space to replace the part of the original substring that represents the name of the neural network block;

[0023] Step 8.2.3: Explore the output channels represented in the atomic strings;

[0024] Step 8.2.4: Explore the atomic strings representing the number of additional sub-layers;

[0025] Step 8.2.5: Form a new individual q, calculate the fitness score of the new individual q. If the fitness score of q is greater than the scores of some individuals in the population P, then replace the worst individual among these individuals with q. Otherwise, execute the failure tolerance mechanism and continuously execute the neural network architecture encoding change innovation operation γ times until a new individual q that is better than some individuals in the population P is found, and replace the worst individual among these individuals with q.

[0026] Step 8.3: Repeat steps 8.1 and 8.2 φ times;

[0027] Step 9: Develop resource mechanisms based on the dual-triangle mechanism to provide new individuals;

[0028] Step 9.1: Let r = i / ε, where ε is a constant set according to the situation;

[0029] Step 9.2: If r is an integer and the number of individuals in the population is greater than the threshold η, then execute steps 9.3 and 9.4;

[0030] Step 9.3: Remove the worst θ*r individuals from the existing population Р, where θ is a manually set baseline value, and the number of individuals removed is controlled by multiplying it by r;

[0031] Step 9.4: Randomly generate θ*r new solutions and add them to the existing population Р;

[0032] Step 10: Proceed to the next iteration, i.e., return to step 7 and increment the iteration count i by 1;

[0033] Step 11: If the number of iterations i = σ, where σ represents the maximum number of generations or if the highest fitness of an individual in population P has not been improved in consecutive τ generations, then stop the calculation; otherwise, go to step 6. The value of τ is set manually based on the complexity of the problem and the experiment.

[0034] Step 12: Perform image classification using an image classification neural network that calculates when the computation stops.

[0035] Furthermore, the fitness is calculated as follows:

[0036] S1: Remove all residual connections from the image classification neural network;

[0037] S2: Through Initialize all neurons in the image classification neural network. Represents a standard normal distribution;

[0038] S3: Sampling x, ∈ represent tensors filled with random numbers that conform to a standard normal distribution with a mean of 0 and a variance of 1, respectively, and are used as input and perturbation;

[0039] S4: Calculation Δ represents the finite derivative of the characteristic graph of f(x), and α represents the perturbation coefficient. This indicates taking the average, ||.|| F Let f(x) represent the absolute value, which is the main part of F after removing the adaptive average pooling layer, flattening layer and fully connected layer at the tail. F represents an image classification neural network.

[0040] S5: For the i-th BN layer with m output channels, calculate... Where σ i,j It is the mini-batch standard deviation statistic of the j-th channel in the BN layer; represents the standard deviation of the BN layer, and m represents the number of output channels of the BN layer;

[0041] S6: Here, Zen(F) represents the fitness score of the final classification neural network F. This fitness score reflects the expressive power of the image classification neural network; the higher the score, the stronger the expressive power of the neural network, and consequently, the better the classification ability. Therefore, in the calculation process, we continuously iterate to find high-fit individuals, thereby finding a better auditing network for image classification.

[0042] The beneficial effects of this invention are as follows:

[0043] In step 9, the population size is reduced every ε generations, retaining the most superior individuals. The reduction in size increases with the evolutionary process. At the same time, new individuals of the same size are added to the population each time it is reduced. These new individuals provide new genetic resources, which mix with the existing superior genes to provide a hybrid power for social development.

[0044] Compared to traditional neural network architecture search methods, this invention primarily improves the speed and quality of image neural network architecture search. Regarding speed: traditional image classification methods require training the searched neural network architecture to obtain fitness, and neural network training generally takes a considerable amount of time to obtain a clear evaluation metric. This invention uses a novel fitness calculation method that does not require training the image neural network, thus significantly saving the time spent searching for neural network architectures.

[0045] Regarding quality: This invention employs a social evolutionary algorithm, designing individual selection operations, group organization and problem-solving operations, individual innovation operations, a failure tolerance mechanism, and a development resource supply operation. Individual selection, group organization and problem-solving, and individual innovation operations retain excellent individuals while eliminating poorly performing ones, thereby improving the overall quality of the population. Furthermore, the failure tolerance mechanism and development resource supply operation expand the search space, preventing the algorithm from getting stuck in local optima and thus affecting the quality of the solution. Attached Figure Description

[0046] Figure 1 This is a flowchart of the social evolution algorithm process of the method of the present invention.

[0047] Figure 2 This is a schematic diagram of the coding method of the present invention.

[0048] Figure 3 This is a schematic diagram of the random initialization part in the social evolution algorithm of the present invention.

[0049] Figure 4 This is a schematic diagram of the fusion operation in the social evolution algorithm of the present invention.

[0050] Figure 5 This is a schematic diagram illustrating the innovative exploration operations in the social evolution algorithm of the present invention.

[0051] Figure 6 This is a flowchart of step 9.3, the removal of an individual operation, in the social evolution algorithm of the present invention. Detailed Implementation

[0052] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The following embodiments or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0053] An image classification neural network is constructed, comprising various neural network blocks. Each block is a convolutional neural network (CNN). Taking the SuperConvK3BNRELU block as an example, it contains a 2D convolutional layer, a batch normalization layer, and a ReLU non-linear activation layer. These three layers are concatenated to form the SuperConvK3BNRELU block. Following the main body, an adaptive average pooling layer, a flattening layer, and a fully connected layer are added to transform the network's output 2D feature map into a 1D vector, thereby classifying the image.

[0054] Use seed string

[0055] The initial population is randomly generated using the formulas SuperConvK3BNRELU(3,8,1,1), SuperResK3K3(8,16,1,8,1), and SuperConvK1BNRELU(16,128,1,1). The social evolution algorithm of this invention is used for iteration. After 22 minutes of iteration, population P is obtained. The highest fitness of an individual in population P is 135.127, and the lowest fitness is 116.039. The individual with a fitness score of 135.127 is selected, resulting in the following classification neural network structure:

[0056] The model structure is defined as follows: SuperConvK3BNRELU(3,24,1,1), SuperResK1K7K1(24,320,1,16,5), SuperResK1K5K1(320,160,1,32,3), SuperResK1K5K1(160,160,1,32,4), SuperResK1K7K1(160,48,1,16,4), and SuperConvK1BNRELU(48,320,1,1). This structure is trained on the CIFAR-100 dataset for 1440 epochs. After training, the model with the best validation result is saved as its model file.

[0057] After saving, the model data was read and an image classification task was performed on the CIFAR-100 dataset. The results are as follows:

[0058]

[0059] Figure 2 This is a schematic diagram of the encoding method of the present invention. The encoding SuperConvK3BNRELU(3,24,1,1) represents a substring of the neural network structure, where SuperConvK3BNRELU, as the name, represents a neural network block in the search space. Figure 2 As can be seen, this block consists of three smaller parts: a Conv2d layer, a Batch Normalization layer, and a ReLU layer. These three layers are interconnected to form the neural network block. (3,24,1,1) represents the relevant parameters of this neural network block, namely the number of input channels, the number of output channels, the stride, and the number of additional sub-layers, respectively. This encoding method allows a neural network block to be represented by a single string.

[0060] Figure 3 This is a schematic diagram of the random initialization part in the social evolution algorithm of the present invention. When the number of individuals is less than the specified population size, the present invention randomly mutates the seed string and adds a new substring on the basis of the original string to generate a new neural network structure, thereby randomly initializing the population.

[0061] Figure 5 This diagram illustrates the innovative exploration operations during the social evolution algorithm of the present invention. A substring position is randomly selected in the encoding. The innovative exploration operation mainly focuses on exploring the neural network block name, output channel, and additional sub-layers of the neural network block within that substring. A neural network block name is randomly selected from the search space to replace the part representing the neural network block name in the original substring. The numbers representing the output channels of the neural network block in this substring are multiplied by 1, 1.25, 1.5, 2, and 2.5 respectively, and then divided by 1.25, 1.5, 2, and 2.5. These resulting numbers are added to a set, and then a number is randomly selected from this set to replace the number representing the output channel in the atomic string. Similarly, the numbers representing the number of additional sub-layers of the neural network block in this substring are multiplied by +1, +2, -1, -2, and +0 respectively, and then added to a set. Finally, a number is randomly selected from this set to replace the number representing the number of additional sub-layers in the atomic string.

[0062] Figure 6 This is a flowchart of step 9.3, the removal of individuals, in the social evolution algorithm of the present invention. In order to maintain the stability of the population and gradually eliminate neural network structures with low fitness, the present invention selects a portion of individuals in the population for elimination. First, the difference between the current number of individuals in the population and the specified population size is calculated and the absolute value is obtained as m. Then, the 2*m structures with the lowest fitness in the population are found, and then the m structures with the longest survival time are found among these 2*m structures and eliminated. This approach is to eliminate individuals with low fitness while retaining some relatively new structures, thereby avoiding the algorithm from getting stuck in local optima and affecting the final search result.

[0063] Table 1 compares the effects of the method of this invention with those of the reference paper.

[0064]

[0065] Paper 1 is: Lin, Ming, Pichao Wang, Zhenhong Sun, Hesen Chen, Xiuyu Sun, Qi Qian, Hao Li and Rong Jin. "Zen-NAS: A Zero-Shot NAS for High-Performance DeepImage Recognition". arXiv, August 22, 2021;

[0066] Paper 2 is: Abdelfattah, Mohamed S., Abhinav Mehrotra, Dudziak and Nicholas D. Lane. Zero-Cost Proxies for Lightweight NAS. arXiv, March 19, 2021;

[0067] Paper 3 is: Chen, Wuyang, Xinyu Gong, and Zhangyang Wang. "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective". arXiv, March 15, 2021;

[0068] Paper 4 is: Mellor, Joseph, Jack Turner, Amos Storkey, and Elliot J. Crowley. "Neural Architecture Search without Training". arXiv, June 11, 2021.

Claims

1. A method for searching image classification neural network architectures based on social evolution algorithms, the method comprising: Step 1: Construct an image classification neural network. The main body of this image classification neural network includes multiple neural network structure blocks, each of which is a convolutional neural network. Step 2: Represent the solution to the neural network architecture search problem using string encoding. The encoding consists of multiple substrings, each containing the name of the neural network block and a number enclosed in parentheses. The numbers in the parentheses represent the number of input channels, the number of output channels, the stride, and the number of additional sublayers for that neural network block, respectively. Step 3: Randomly generate a population Р containing |Р| individuals, where the code of each individual in the population corresponds to the code of an entire image classification neural network; Step 4: Calculate the fitness of each individual; Step 5: Set a new set of individuals as Ω = φ; Step 6: Let i = 1; Step 7: Implement organizational efforts to tackle the problem. Step 7.1: Randomly select α individuals from the population Р to form a task force; Step 7.2: Perform a fusion operation on the α individuals in the research team. Each individual takes one encoded substring in sequence, that is, the first individual takes the first substring, the second individual takes the second substring, and so on, until the αth individual takes the αth substring. The α substrings are merged to form a new individual q. Step 7.3: Calculate the fitness of the new individual q. If the fitness of q is greater than that of some individuals in the population P, then replace the individual with the worst fitness among these individuals with q. Step 7.4: Repeat steps 7.1 and 7.3 β times; Step 8: Implement individual innovation operations: Step 8.1: Randomly select an individual p from Р; Step 8.2: Perform innovative exploration operations on p: Step 8.2.1: Randomly select a substring position in the individual p encoding; Step 8.2.2: Randomly extract a neural network block name from the search space to replace the part of the original substring that represents the name of the neural network block; Step 8.2.3: Explore the output channels represented in the atomic strings; Step 8.2.4: Explore the atomic strings representing the number of additional sub-layers; Step 8.2.5: Form a new individual q, calculate the fitness score of the new individual q. If the fitness score of q is greater than the scores of some individuals in the population P, then replace the worst individual among these individuals with q. Otherwise, execute the failure tolerance mechanism and continuously execute the neural network architecture encoding change innovation operation γ times until a new individual q that is better than some individuals in the population P is found, and replace the worst individual among these individuals with q. Step 8.3: Repeat steps 8.1 and 8.2 φ times; Step 9: Develop resource mechanisms based on the dual-triangle mechanism to provide new individuals; Step 9.1: Let r = i / ε, where ε is a constant set according to the situation; Step 9.2: If r is an integer and the number of individuals in the population is greater than the threshold η, then execute steps 9.3 and 9.4; Step 9.3: Remove the worst θ*r individuals from the existing population Р, where θ is a manually set baseline value, and the number of individuals removed is controlled by multiplying it by r; Step 9.4: Randomly generate θ*r new solutions and add them to the existing population Р; Step 10: Proceed to the next iteration, i.e., return to step 7 and increment the iteration count i by 1; Step 11: If the number of iterations i = σ, where σ represents the maximum number of generations or if the highest fitness of an individual in population P has not been improved in consecutive τ generations, then stop the calculation; otherwise, go to step 6. The value of τ is set manually based on the complexity of the problem and the experiment. Step 12: Perform image classification using an image classification neural network that calculates when the computation stops.

2. The image classification neural network architecture search method based on social evolution algorithm as described in claim 1, characterized in that, The fitness calculation method in step 4 is as follows: S1: Remove all residual connections from the image classification neural network; S2: Through Initialize all neurons in the image classification neural network. Represents a standard normal distribution; S3: Sampling x, ∈ represent tensors filled with random numbers that conform to a standard normal distribution with a mean of 0 and a variance of 1, respectively, and are used as input and perturbation; S4: Calculation Δ represents the finite derivative of the characteristic graph of f(x), and α represents the perturbation coefficient. This indicates taking the average, ||.|| F Let f(x) represent the absolute value, which is the main part of F after removing the adaptive average pooling layer, flattening layer and fully connected layer at the tail. F represents an image classification neural network. S5: For the i-th BN layer with m output channels, calculate... Where σ i,j It is the mini-batch standard deviation statistic of the j-th channel in the BN layer; represents the standard deviation of the BN layer, and m represents the number of output channels of the BN layer; S6: Here, Zen(F) represents the fitness score of the final classification neural network F.

Images