An improved super-network-based evolutionary neural network architecture search method

Abstract

This invention relates to an improved method for searching evolutionary neural network architectures based on hypernetworks. It includes the following steps: Step S1, encapsulating five computational modules with the input layer as the first layer; Step S2, binarizing the connections of the computational nodes within the neural network; Step S3, learning structural weights for each computational node; Step S4, constructing a parent population P using a binary tournament selection method; Step S5, forming a offspring population Q; Step S6, performing mutation operations on individuals in the offspring population Q; Step S7, decoding each individual in the offspring population Q into its corresponding neural network and obtaining structural weights; Step S8, merging the parent population P and the offspring population Q into a population R, selecting several individuals as the original population for the next generation using an environmental selection method, and feeding this information back to Step S4, until a predetermined maximum number of generations is reached. After evolution, the individual with the highest fitness value is output as the optimal neural network architecture.

Description

Technical Field

[0001] This invention relates to the field of image classification model construction technology, and in particular to an improved method for searching evolutionary neural network architectures based on hypernetworks. Background Technology

[0002] Image classification is an image processing technique that distinguishes different categories of objects based on the different features reflected in an image. Because many models applied to image classification can be transferred to other computer vision fields as feature extraction networks, image classification is a fundamental task in computer vision, and the design of image classification models is a hot topic for researchers. However, manually designing neural network models requires experienced experts who carefully study the distribution and characteristics of the dataset and conduct repeated experiments to design high-performance neural network models. Therefore, it requires a huge investment of time and manpower.

[0003] Currently, Neural Architecture Search (NAS) algorithms have attracted widespread attention from researchers. These algorithms enable the automatic design of effective neural network architectures based on a given dataset, even without extensive technical expertise. However, NAS algorithms typically require continuous evaluation of neural network models within the search space, necessitating significant computing power. To improve the search efficiency of NAS algorithms, two main methods are currently employed:

[0004] The first approach is to construct an end-to-end performance predictor. This method requires an encoding method that uniquely maps the neural network architecture to a set of numerical decision variables. The encoding of the neural network architecture and its performance (e.g., classification accuracy) are then paired and used as input to train the performance predictor. Once trained, the performance predictor can directly predict the performance of the neural network model within the search space without requiring further training of the model, thus improving search efficiency. However, this method follows a train-then-predict approach, requiring the performance predictor to be trained using a set of training samples. Generally, the more training samples, the better the predictor's performance. However, collecting more training samples means consuming more computational resources, thus impacting search efficiency. Therefore, in practical applications, an incremental strategy is needed to sample more efficient neural network architectures, which incurs a certain computational cost.

[0005] The second approach is the One-Shot Neural Architecture Search (NAS). This method first trains a one-shot model as the search space; then, a certain number of sub-networks are randomly sampled from this one-shot model for performance evaluation, and the sub-networks are ranked according to their performance. Finally, the sub-network with the best performance is output. Since sub-networks can inherit weights from the one-shot model and can be evaluated without training, this effectively improves the search efficiency of the NAS algorithm. However, existing one-shot neural architecture search algorithms have certain drawbacks. First, uneven training of nodes within the one-shot model leads to inaccurate performance ranking during the sub-network evaluation phase, resulting in the algorithm failing to find the optimal network architecture. Second, interference between different sub-networks during one-shot model training can cause instability in the one-shot neural architecture search algorithm, resulting in slow or even non-convergent convergence of the one-shot model, leading to poor performance predictions for the sub-models. Summary of the Invention

[0006] To address the shortcomings of existing hypernetwork-based neural network architecture search methods, such as unstable performance, slow convergence speed, or even failure to converge, the present invention aims to provide an evolutionary neural network architecture search method based on hypernetworks. This method uses an evolutionary algorithm as a search strategy to automatically generate neural network architectures based on hypernetworks, thereby improving the classification accuracy of image classification tasks.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] An improved method for searching evolutionary neural network architectures based on hypernetworks, characterized by comprising the following steps:

[0009] Step S1: Using the input layer as the first layer, five computation modules are encapsulated; each module encapsulates M computation nodes, and finally, a fully connected layer is used as the output layer of the neural network; where M is a natural number greater than 1.

[0010] Step S2: Encode the neural network structure using a hybrid encoding method and binarize the connections of the internal computing nodes of the neural network; randomly generate N chromosomes to construct the initial population; where N is a natural number greater than 1.

[0011] Step S3: Uniformly sample individuals in the population, train based on training data, learn structural weights for each computing node, and use the classification accuracy of the validation set as the fitness function to evaluate the fitness of individuals.

[0012] Step S4: Use the binary tournament selection method to construct the parent population P.

[0013] Step S5, based on the given crossover rate p c The chromosomes in the parent population are crossed in pairs using a hybrid crossover method to obtain several new chromosomes, which form the offspring population Q.

[0014] Step S6, based on the given mutation rate p m The mixed mutation method is used to perform mutation operations on individuals in the offspring population Q.

[0015] In step S7, each individual in the offspring population Q is decoded into a corresponding neural network, and structural weights are obtained through inheritance or random initialization. The fitness of the individual is evaluated using the classification accuracy of the validation set as the fitness function.

[0016] In step S8, the parent population P and the offspring population Q are merged into a single population R. Environmental selection is used to select several individuals as the initial population for the next generation, and this selection is fed back to step S4 until the predetermined maximum number of generations is reached. After evolution is complete, the individual with the highest fitness value is output as the optimal neural network architecture.

[0017] Furthermore, in step S1 above, the input layer is sequentially encapsulated by a convolutional layer, a ReLU activation function, and a batch normalization (BN) layer.

[0018] Furthermore, in step S1 above, the computing nodes are computing units in the neural network, which can be randomly selected from the operation search space θ. All computing nodes in the first, third, and fifth computing modules have a step size of 1; all computing nodes in the second and fourth computing modules have a step size of 2.

[0019] Furthermore, in step S2 above, the hybrid encoding method is an encoding method combining integers and binary numbers. Integer encoding is used to describe the types of computational nodes within the neural network architecture and the connections between nodes; binary numbers are used to binarize the connections between the computational nodes within the neural network architecture, describing whether a connection between two computational nodes is activated. Specifically:

[0020] Furthermore, in step S21 above, a computing node is encoded as a quintuple. in, I1 and I2 represent the indices of the computing units a contained in computing node i; i.e., computing node i and computing nodes I1 and I2 are interconnected. I1 and I2 are a set of integers; J1 and J2 are a set of binary numbers representing four states of the connection between computation node i and computation nodes I1 and I2. Specifically: J1 = 0, J2 = 0, indicating that the connection between computation node i and computation nodes I1 and I2 is active. In this case, the feature maps of the outputs of computation nodes I1 and I2 are fused and used as the input of computation node i. The output δ of computation node i is:

[0021]

[0022] J1 = 0, J2 = 1, indicating that the connection between computing node i and computing node I1 is activated, and the connection between computing node i and computing node I2 is closed; then, the output δ of computing node i is:

[0023]

[0024] J1 = 1, J2 = 0, indicating that the connection between computing node i and computing node I1 is closed, and the connection between computing node i and computing node I2 is activated; then, the output δ of computing node i is:

[0025]

[0026] J1=1, J2=1, indicating that the connections between computation node i and computation nodes I1 and I2 are both closed; that is, computation node i is currently masked. Therefore, the feature maps output by computation nodes I1 and I2, after fusion, do not pass through the computation nodes... Process the output value δ directly as the computation node i:

[0027] δ=I1(x c )+I2(x d )

[0028] Where, x c x d These are the inputs to compute nodes I1 and I2, respectively, where I1(x) is the input to the compute nodes I2. c ), I2(x d The outputs of computing nodes I1 and I2 are respectively. This represents the output feature map I1(x) of computing nodes I1 and I2. c ), I2(x d The fusion, as input to computing node i, is performed by the computing unit. After processing, it becomes the output of computing node i.

[0029] Furthermore, in step S22 above, the computing module contains M computing nodes. Therefore, the encoding structure of a computing module at this time is:

[0030]

[0031] Step S23: The chromosome is a neural network architecture, and each neural network architecture contains five computational modules. Therefore, the encoding structure of a neural network architecture at this point is:

[0032]

[0033] Furthermore, in step S3 above, individuals in the population are uniformly sampled, trained based on training data, structural weights are generated for each computing node, and the classification accuracy of the validation set is used as the fitness function to evaluate the fitness of individuals. Specifically:

[0034] Furthermore, in step S31 above, the predetermined training dataset is divided into B batches according to a given batch size. B is a natural number greater than N. For each batch, an individual is randomly selected from the parent population P, and that individual is decoded into the corresponding neural network for training, until the maximum training batch size B is reached.

[0035] Furthermore, in step S32 above, the fitness value of each individual in the parent population is evaluated. The classification accuracy of the images in the validation dataset is used as the fitness function for fitness evaluation, and its expression is:

[0036]

[0037] Where G is the number of images correctly identified by the model, and H is the total number of images in the validation set.

[0038] Furthermore, in step S4 above, the steps for the binary tournament selection method are as follows:

[0039] Step S41: Randomly select two individuals from the original population. Based on their fitness values, retain the individual with the higher fitness value in the parent population P, and return the individual with the lower fitness value to the original population.

[0040] Step S42: Repeat step S41 until the number of individuals in the parent population P reaches the preset number of individuals K, where K is a natural number greater than 1.

[0041] Furthermore, in step S5 above, based on the given crossover rate p c The mixed crossover method is used to perform pairwise crossovers on individuals with different chromosomes in the parent population P, resulting in several individuals with different chromosomes. The specific steps are as follows:

[0042] Step S51: Split the integer part and binary part of each chromosome into an integer chromosome part and a binary chromosome part.

[0043] Step S52: Generate a random number r in the interval [0, 1], randomly select two individuals p1 and p2 from the parent population P, and use the random number r to determine whether the two individuals p1 and p2 should perform a crossover operation.

[0044] Step S53, if r≤p m Single-point crossover is performed by aligning the left sides of the integer chromosome portions of the two chromosomes. This involves randomly selecting a crossover point on each chromosome and exchanging the genes at that point. The crossover points on both chromosomes should be in the same position. Multi-point crossover is then performed by aligning the left sides of the binary chromosome portions of the two chromosomes. This involves randomly selecting several crossover points on each chromosome and exchanging the genes at those points. The crossover points on both chromosomes should also be in the same position. The two individuals q1 and q2 generated by this hybrid crossover method are then stored in the offspring population Q.

[0045] Step S54, if r ≥ p m Then, store the two individuals p1 and p2 selected in step S52 into the offspring population Q.

[0046] Furthermore, in step S6 above, based on the given mutation rate p m The mutation operation is performed on individuals in the offspring population Q using a mixed mutation method. The specific steps are as follows:

[0047] Step S61: Split the integer part and binary part of each chromosome into an integer chromosome part and a binary chromosome part.

[0048] Step S62: For any gene locus in any chromosome individual, a random number t corresponding to any chromosome individual is randomly generated in the interval [0, 1]. The random number is used to determine whether to perform a mutation operation on the gene locus of the individual.

[0049] Step S63, if t≤p m Then, a polynomial mutation operation is performed on the integer chromosomal portion of the chromosome.

[0050]

[0051] Among them, a i This represents the gene at the i-th gene locus in the chromosome, a′ i Indicates based on the gene a i The newly generated gene; u is a random number generated in the interval [0, 1]; Each represents the gene a i Upper and lower bounds of the variation.

[0052] Step S64, if t≤pm Then, a flip mutation operation is performed on the binary chromosome portion of the chromosome. That is, several mutation points are randomly selected in the chromosome, and the gene position corresponding to each mutation point is mutated. Genes with a gene position of 0 are mutated to 1, and genes with a gene position of 1 are mutated to 0.

[0053] In step S7, each individual in the offspring population obtains structural weights through inheritance or random initialization. Specifically, for any chromosome individual in the offspring population Q, any computational node in that chromosome individual, if obtained by the hybrid crossover method in step S5, inherits the weights from the corresponding computational nodes in its parent chromosome individual; if obtained by the hybrid mutation method in step S6, the weights of that computational node are generated by random initialization.

[0054] In step S8, the parent population P and the offspring population Q are merged into a single population R. Environmental selection is then used to select several individuals as the original population for the next generation. The specific steps are as follows:

[0055] Step S81: Sort the individuals in population R in descending order of fitness value.

[0056] Step S82: According to the preset population size N, select individuals ranked from 1 to N from the population R to be the next generation population.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) This invention utilizes a hybrid encoding method to encode the hypernetwork. Integer encoding is used to describe the types and connections of computational nodes within a neural network architecture, while binary encoding is used to binarize the connections of computational nodes within a neural network architecture. The advantage of this design is that, during population evolution, different parts of chromosomes can be randomly selected for crossover, enabling simultaneous global and local searches of the search space. Specifically, single-point crossover operations generate new neural network architectures by exchanging computational nodes within two individuals, achieving global exploration of the search space. Multi-point crossover operations merely exchange the binarized information of neural network connections, changing the direction of data flow within a single neural network to generate new neural network architectures, achieving local exploration of the search space.

[0059] (2) Based on the aforementioned hybrid encoding method, this invention can randomly select different parts of the chromosome for mutation during the population evolution process, and introduce computational nodes that do not belong to the supernetwork through polynomial mutation. The weights of these computational nodes are randomly initialized. The advantage of this design is that it can solve the problem of deep coupling relationships formed by the weights of computational nodes in the training of supernetworks in current methods, which leads to the difficulty in convergence of supernetwork training in the later stages. The introduced computational nodes that do not belong to the supernetwork will be integrated into the supernetwork as the population evolves. Since the weights of the computational nodes that do not belong to the supernetwork are randomly assigned, the deep coupling relationship in the training of supernetworks can be reduced. This can not only help the algorithm escape local optima, but also avoid the difficulty in convergence of supernetwork training.

[0060] Based on the aforementioned beneficial effects (1) and (2), the method proposed in this invention can solve the problem that supernetworks are difficult to train to convergence. Based on this, compared with existing methods, the method proposed in this invention can realize neural network architecture search based on large-scale datasets (e.g., ImageNet). Attached Figure Description

[0061] Figure 1 This is the overall neural network architecture of the present invention.

[0062] Figure 2 This is a flowchart of the algorithm of the present invention.

[0063] Figure 3 A flowchart for chromosome generation in this invention is provided.

[0064] Figure 4 This is a schematic diagram of the hybrid crossover method and hybrid variation method of the present invention.

[0065] Figure 5 This invention describes the optimization process of the neural network architecture based on the ImageNet classification task.

[0066] Figure 6 This describes the training process of the neural network architecture discovered in this invention, based on the ImageNet classification task. Detailed Implementation

[0067] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0068] like Figures 1 to 3 As shown, this embodiment provides an improved method for searching evolutionary neural network architectures based on hypernetworks, which mainly includes the following:

[0069] The first step involves encapsulating five computation modules within the input layer as the first layer. Each module encapsulates M computation nodes, and a fully connected layer serves as the output layer of the neural network. In this embodiment, each computation module contains 9 computation nodes, i.e., M = 9. The input layer is sequentially encapsulated by a convolutional layer, a ReLU activation function, and a batch normalization (BN) layer. The computation nodes are computational units in the neural network and can be randomly selected from the operation search space θ. All computation nodes in the first, third, and fifth computation modules have a stride of 1; all computation nodes in the second and fourth computation modules have a stride of 2.

[0070] The second step is to encode the neural network structure using a hybrid encoding method and binarize the connections of the internal computing nodes of the neural network; N chromosomes are randomly generated to construct the initial population; in this embodiment, the initial population contains 40 chromosomes, i.e., N=40.

[0071] In this embodiment, a hybrid encoding method is used to randomly generate an initial population for population initialization. Each individual in the initial population represents a corresponding neural network architecture, and the connection method of the internal computing nodes is binarized. Each computing node represents a computing unit of a neural network, and the encoding information of the computing unit is shown in Table 1. During gene encoding, the computing units are randomly encoded into the overall neural network architecture to form a chromosome, which constitutes the final neural network architecture.

[0072] Table 1. Encoding information of neural network computing units

[0073]

[0074]

[0075] The specific encoding method is as follows:

[0076] The hybrid encoding method is a combination of integer and binary encoding. Integer encoding is used to describe the types of computational nodes within the neural network architecture and the connections between nodes; binary numbers are used to binarize the connections between the computational nodes within the neural network architecture, describing whether a connection between two computational nodes is activated. Specifically:

[0077] Step S21, a computing node is encoded as a quintuple. in, I1 and I2 represent the computing units a contained in computing node i; I1 and I2 represent the indices of the computing units connected to computing node i, that is, computing node i and computing nodes I1 and I2 are interconnected. I1 and I2 are a set of integers; J1 and J2 are a set of binary numbers representing four states of how to connect computing node i with computing nodes I1 and I2, specifically:

[0078] J1 = 0, J2 = 0, indicating that the connections between computation node i and computation nodes I1 and I2 are both active; in this case, the feature maps of the outputs of computation nodes I1 and I2 are fused and used as the input to computation node i. The output δ of computation node i is:

[0079]

[0080] J1 = 0, J2 = 1, indicating that the connection between computing node i and computing node I1 is activated, and the connection between computing node i and computing node I2 is closed; then, the output δ of computing node i is:

[0081]

[0082] J1 = 1, J2 = 0, indicating that the connection between computing node i and computing node I1 is closed, and the connection between computing node i and computing node I2 is activated; then, the output δ of computing node i is:

[0083]

[0084] J1=1, J2=1, indicating that the connections between computation node i and computation nodes I1 and I2 are both closed; that is, computation node i is currently masked. Therefore, the feature maps output by computation nodes I1 and I2, after fusion, do not pass through the computation nodes... Process the output value δ directly as the computation node i:

[0085] δ=I1(x c )+I2(x d )

[0086] Where, x c x d These are the inputs to compute nodes I1 and I2, respectively, where I1(x) is the input to the compute nodes I2. c ), I2(x d The outputs of computing nodes I1 and I2 are respectively. This represents the output feature map I1(x) of computing nodes I1 and I2. c ), I2(x d The fusion, as input to computing node i, is performed by the computing unit. After processing, it becomes the output of computing node i.

[0087] Step S22, the computing module contains M computing nodes. Then, the encoding structure of a computing module at this time is:

[0088]

[0089] Step S23: The chromosome is a neural network architecture, and each neural network architecture contains five computational modules. Therefore, the encoding structure of a neural network architecture at this point is:

[0090]

[0091] The third step involves uniformly sampling individuals from the population, training based on the training data, learning structural weights for each computing node, and using the validation set classification accuracy as the fitness function to evaluate the fitness of individuals. Specifically, the steps are as follows:

[0092] Step S31: Divide the predetermined training dataset into B batches according to the given batch size. B is a natural number greater than N. For each batch, randomly select an individual from the parent population P, decode that individual into the corresponding neural network, and train the neural network until the maximum training batch size B is reached. Therefore, in this embodiment, the batch size is set to 256.

[0093] Step S32: Evaluate the fitness value of each individual in the parent population. The classification accuracy of images in the validation dataset is used as the fitness function for evaluation, and its expression is:

[0094]

[0095] Where G is the number of images correctly identified by the model, and H is the total number of images in the validation set.

[0096] The fourth step involves constructing the parent population P using a binary tournament selection method. Specifically:

[0097] Step S41: Randomly select two individuals from the original population. Based on their fitness values, retain the individual with the higher fitness value in the parent population P, and return the individual with the lower fitness value to the original population.

[0098] Step S42: Repeat step S41 until the number of individuals in the parent population P reaches the preset number of individuals K. In this embodiment, K = 40.

[0099] Fifth step, based on the given crossover rate p c The parent population's chromosomes are crossbred pairwise using a hybrid crossover method to obtain several new chromosomes, which form the offspring population Q. In this embodiment, p c =0.95, mixed crossover method such as Figure 4 As shown, the specific steps are as follows:

[0100] Step S51: Split the integer part and binary part of each chromosome into an integer chromosome part and a binary chromosome part.

[0101] Step S52: Generate a random number r in the interval [0, 1], randomly select two individuals p1 and p2 from the parent population P, and use the random number r to determine whether the two individuals p1 and p2 should perform a crossover operation.

[0102] Step S53, if r≤p m Single-point crossover is performed by aligning the left sides of the integer chromosome portions of the two chromosomes. This involves randomly selecting a crossover point on each chromosome and exchanging the genes at that point. The crossover points on both chromosomes should be in the same position. Multi-point crossover is then performed by aligning the left sides of the binary chromosome portions of the two chromosomes. This involves randomly selecting several crossover points on each chromosome and exchanging the genes at those points. The crossover points on both chromosomes should also be in the same position. The two individuals q1 and q2 generated by this hybrid crossover method are then stored in the offspring population Q.

[0103] Step S54, if r ≥ p m Then, store the two individuals p1 and p2 selected in step S52 into the offspring population Q.

[0104] Step 6, based on the given mutation rate p m A hybrid mutation method is used to perform mutation operations on individuals in the offspring population Q. In this embodiment, p m =0.1, mixed mutation method such as Figure 5 As shown, the specific steps are as follows:

[0105] Step S61: Split the integer part and binary part of each chromosome into an integer chromosome part and a binary chromosome part.

[0106] Step S62: For any gene locus in any chromosome individual, a random number t corresponding to any chromosome individual is randomly generated in the interval [0, 1]. The random number is used to determine whether to perform a mutation operation on the gene locus of the individual.

[0107] Step S63, if t≤p m Then, a polynomial mutation operation is performed on the integer chromosomal portion of that chromosome.

[0108]

[0109] Among them, a i This represents the gene at the i-th gene locus in the chromosome, a′ i Indicates based on the gene a iThe newly generated gene; u is a random number generated in the interval [0, 1]; Each represents the gene a i Upper and lower bounds of the variation.

[0110] Step S64, if t≤p m Then, a flip mutation operation is performed on the binary chromosome portion of the chromosome. That is, several mutation points are randomly selected in the chromosome, and the gene position corresponding to each mutation point is mutated. Genes with a gene position of 0 are mutated to 1, and genes with a gene position of 1 are mutated to 0.

[0111] The seventh step involves decoding each individual in the offspring population Q into a corresponding neural network, obtaining structural weights through inheritance or random initialization, and using the classification accuracy of the validation set as the fitness function to evaluate the fitness of the individual.

[0112] Each individual in the offspring population obtains structural weights through inheritance or random initialization. Specifically, for any chromosome individual in the offspring population Q, any computational node in that chromosome individual, if obtained by the hybrid crossover method in step S5, inherits the weights from the corresponding computational nodes in its parent chromosome individual; if obtained by the hybrid mutation method in step S6, the weights of that computational node are generated by random initialization.

[0113] In step eight, the parent population P and the offspring population Q are merged into a single population R. Environmental selection is used to select several individuals as the initial population for the next generation, and this selection is fed back to step S4 until the predetermined maximum number of generations is reached. After evolution is complete, the individual with the highest fitness value is output as the optimal neural network architecture.

[0114] Step S81: Sort the individuals in population R in descending order of fitness value.

[0115] Step S82: According to the preset population size N, select individuals ranked from 1 to N from the population R to be the next generation population.

[0116] To verify the advantages of this invention, the following comparison is made:

[0117] The dataset used in this invention is ImageNet. ImageNet is a large visualization dataset used for research on visual object recognition. It contains over 14 million images, divided into training, validation, and test sets, and includes more than 20,000 categories.

[0118] The hyperparameter design of the algorithm used in this invention is as follows:

[0119] Initial channel count C = 32, maximum generation = 100. Initialize SGD optimizer parameters, including: initial learning rate lr = 0.1, weight decay coefficient w = 0.0003, momentum coefficient m = 0.9.

[0120] After the algorithm completes its iterations, it outputs the individual with the optimal fitness value. This individual is then decoded into the corresponding neural network architecture, EvoNet. The network architecture parameters are reinitialized, and the architecture is trained using the training dataset until it converges. Finally, the performance of the neural network architecture is tested using the test dataset.

[0121] In this invention, the ImageNet-based optimization process and the final individual testing process are respectively handled by... Figure 5 and Figure 6 As shown, it can be seen that the present invention achieves high prediction and classification accuracy during the search, with a top-1 classification accuracy of 77.4%.

[0122] Table 2 shows the performance comparison between the neural network architectures searched by this invention and existing manually designed neural network architectures and neural network architecture search algorithms. Table 2 demonstrates that the neural network architectures searched by this invention outperform existing manually designed neural network architectures and neural network architecture search algorithms.

[0123] Table 2 Comparison of Experimental Results

[0124]

Claims

1. An improved method for searching evolutionary neural network architectures based on hypernetworks, characterized in that, Includes the following steps: Step S1: Using the input layer as the first layer, five computation modules are encapsulated; each module encapsulates M computation nodes, and finally a fully connected layer is used as the output layer of the neural network; where M is a natural number greater than 1. Step S2: Encode the neural network structure using a hybrid encoding method and binarize the connections of the computational nodes within the neural network; randomly generate N chromosomes to construct the original population; the number of computational nodes within any chromosome is less than the total number of computational nodes of the preset chromosome; N is a natural number greater than 1. Step S3: Uniformly sample individuals in the population, train based on training data, generate structural weights for each computing node, and use the classification accuracy of the validation set as the fitness function to evaluate the fitness of individuals. In step S3, uniform sampling is performed on individuals in the population, training is conducted based on the training data, structural weights are generated for each computing node, and the fitness of individuals is evaluated using the classification accuracy of the validation set as the fitness function, including: Step S31: Divide the predetermined training dataset into B batches according to the given batch size; B is a natural number greater than N; for each batch, randomly select an individual from the parent population P, decode the individual into the corresponding neural network for training, until the maximum training batch B is reached. Step S32: Evaluate the fitness value (fitness) of each individual in the parent population; use the classification accuracy of images in the validation set as the fitness function for fitness evaluation, the expression of which is: Where G is the number of images correctly identified by the model, and H is the total number of images in the validation set; Step S4: Use the binary tournament selection method to construct the parent population P; Step S5: Based on the given crossover rate pc, the chromosome individuals in the parent population are crossovered pairwise using the mixed crossover method to obtain several new chromosomes, which form the offspring population Q. Step S6: Based on the given mutation rate pm, perform mutation operations on individuals in the offspring population Q using the mixed mutation method; Step S7: Each individual in the offspring population Q is decoded into a corresponding neural network, and structural weights are obtained through inheritance or random initialization. The fitness of the individual is evaluated using the classification accuracy of the validation set as the fitness function. In step S8, the parent population P and the offspring population Q are merged into a population R. Several individuals are selected as the original population for the next generation using environmental selection, and the results are fed back to step S4 until the predetermined maximum number of generations is reached. After the evolution is complete, the individual with the highest fitness value is output as the optimal neural network architecture.

2. The improved method for searching evolutionary neural network architectures based on hypernetworks according to claim 1, characterized in that, The input layer is composed of a convolutional layer, a ReLU activation function, and a batch normalization layer.

3. The improved method for searching evolutionary neural network architectures based on hypernetworks according to claim 1, characterized in that, In step S1, the computing node is a computing unit in the neural network, randomly selected from the operation search space θ; the step size of all computing nodes in the first computing module, the third computing module, and the fifth computing module is 1. The step size of all computing nodes in the second and fourth computing modules is 2.

4. The improved method for searching evolutionary neural network architectures based on supernetworks according to claim 1, characterized in that, In step S2, the hybrid encoding method is an encoding method that combines integers and binary numbers; integer encoding is used to describe the types of computing nodes within the neural network architecture and the connection relationships between nodes; binary numbers are used to binarize the connection relationships between computing nodes within the neural network architecture, describing whether the connection between two computing nodes is activated; specifically: Step S21, a computing node is encoded as a quintuple. in, I1 and I2 represent the indices of the computing units a contained in computing node i; i.e., computing node i and computing nodes I1 and I2 are interconnected. I1 and I2 are a set of integers; J1 and J2 are a set of binary numbers representing four states of the connection between computing node i and computing nodes I1 and I2. Specifically: J1 = 0, J2 = 0, indicating that the connection between computing node i and computing nodes I1 and I2 is active. In this case, the feature maps of the outputs of computing nodes I1 and I2 are fused and used as the input of computing node i; the output δ of computing node i is: J1 = 0, J2 = 1, indicating that the connection between computing node i and computing node I1 is activated, and the connection between computing node i and computing node I2 is closed; then, the output δ of computing node i is: J1 = 1, J2 = 0, indicating that the connection between computing node i and computing node I1 is closed, and the connection between computing node i and computing node I2 is closed. If node i is activated, then the output δ of node i is: J1=1, J2=1, indicating that the connections between computing node i and computing nodes I1 and I2 are both closed; that is, computing node i is currently masked. Therefore, the feature maps output by computing nodes I1 and I2, after fusion, do not pass through the computing unit. Process the output value δ directly as the computation node i: δ = I1 (xc) + I2 (xd) Where xc and xd are the inputs of computing nodes I1 and I2, respectively, and I1(xc) and I2(xd) are the outputs of computing nodes I1 and I2, respectively. The output feature maps I1(xc) and I2(xd) of computing nodes I1 and I2 are fused and used as the input of computing node i by the computing unit. After processing, it becomes the output of computing node i; Step S22, the computing module contains M computing nodes; then the encoding structure of a computing module at this time is: Step S23: The chromosome is a neural network architecture, and each neural network architecture contains five computing modules; then, the encoding structure of a neural network architecture is as follows:

5. An improved method for searching evolutionary neural network architectures based on supernetworks according to claim 1, characterized in that, In step S4, the steps for the binary tournament selection method are as follows: Step S41: Randomly select two individuals from the original population. Based on their fitness values, retain the individual with the higher fitness value in the parent population P, and return the individual with the lower fitness value to the original population. Step S42: Repeat step S41 until the number of individuals in the parent population P reaches the preset number of individuals K, where K is a natural number greater than 1.

6. The improved method for searching evolutionary neural network architectures based on supernetworks according to claim 1, characterized in that, In step S5, based on the given crossover rate pc, the mixed crossover method is used to perform pairwise crossovers on the chromosome individuals in the parent population P to obtain several chromosome individuals. The specific steps are as follows: Step S51: Split the integer part and binary part of each chromosome into an integer chromosome part and a binary chromosome part; Step S52: Randomly generate a random number r in the interval [0, 1], randomly select two individuals p1 and p2 from the parent population P, and use the random number r to determine whether the two individuals p1 and p2 should perform a crossover operation; Step S53: If r ≤ pm, perform single-point crossover by aligning the left sides of the integer chromosome portions of the two chromosomes. That is, randomly select a crossover point in the two integer chromosomes and exchange the genes at the crossover point. The crossover points of the two integer chromosomes should be in the same position. Perform multi-point crossover by aligning the left sides of the binary chromosome portions of the two chromosomes. That is, randomly select several crossover points in the two binary chromosomes and exchange the genes at the crossover points. The crossover points of the two binary chromosomes should be in the same position. Store the two individuals q1 and q2 generated by the hybrid crossover method into the offspring population Q. Step S54: If r > pm, store the two individuals p1 and p2 selected in step S52 into the offspring population Q.

7. An improved method for searching evolutionary neural network architectures based on hypernetworks according to claim 1, characterized in that, In step S6, based on the given mutation rate pm, the mutation operation is performed on individuals in the offspring population Q using the mixed mutation method; the specific steps are as follows: Step S61: Split the integer part and binary part of each chromosome into an integer chromosome part and a binary chromosome part; Step S62: For any gene locus in any chromosome individual, randomly generate a random number t corresponding to any chromosome individual in the interval [0, 1], and use the random number to determine whether to perform a mutation operation on the gene locus of the individual. Step S63: If t≤pm, then perform a polynomial mutation operation on the integer part of the chromosome. Where ai represents the gene at the i-th gene locus in the chromosome, a′i represents a new gene generated based on the gene ai; u is a random number generated in the interval [0, 1]. , These represent the upper and lower bounds of the ai variation of the gene, respectively; Step S64: If t > pm, then perform a flip mutation operation on the binary chromosome part of the chromosome. That is, randomly select several mutation points in the chromosome, and perform mutation on the gene position corresponding to each mutation point. Genes with a gene position of 0 are mutated to 1, and genes with a gene position of 1 are mutated to 0.

8. An improved method for searching evolutionary neural network architectures based on hypernetworks according to claim 1, characterized in that, In step S7, each individual in the offspring population obtains structural weights through inheritance or random initialization. Specifically, for any chromosome individual in the offspring population Q, any computation node in that chromosome individual, if obtained by the hybrid crossover method in step S5, inherits the weights from the corresponding computation nodes in its parent chromosome individual. If obtained by the hybrid mutation method in step S6, the weight of the computation node is generated by random initialization.

9. An improved method for searching evolutionary neural network architectures based on supernetworks according to claim 1, characterized in that, In step S8, the parent population P and the offspring population Q are merged into a single population R. Environmental selection is then used to select several individuals as the original population for the next generation. The specific steps are as follows: Step S81: Sort the individuals in population R in descending order of fitness value. Step S82: According to the preset population size N, select individuals ranked from 1 to N in population R as the next generation population.

Images