A multi-objective hybrid evolutionary neural architecture search method and system assisted by a dominance classifier

Abstract

The present invention provides a multi-objective hybrid evolutionary neural architecture search method and system assisted by a superiority classifier. The method includes: Step 1, designing and training an autoencoder based on contrastive learning for extracting features of network architectures; Step 2, designing and training an end-to-end superiority classifier based on comparison of architecture performance advantages and disadvantages; Step 3, initializing an architecture population from the search space by random sampling and comprehensively training the individuals in the population to achieve real evaluation; Step 4, taking the current population as the parent generation, generating candidate architectures, and selecting the optimal architecture through an environmental selection strategy to update the population; Step 5, performing diversity evaluation on the new population through a projection and clustering method; Step 6, outputting a set of globally optimal architectures. The present invention greatly reduces the consumption of computing resources and lays a solid foundation for the wide promotion of neural architecture search technology in practical applications.

Description

Technical Field

[0001] The present invention relates to the automated machine learning technology in the field of artificial intelligence, and particularly relates to a multi-objective hybrid evolutionary neural architecture search method and system assisted by a dominance classifier. Background Art

[0002] In recent years, deep learning algorithms have made breakthroughs in engineering fields such as image classification, speech recognition, and natural language processing, significantly promoting technological innovation in various industries. It is worth noting that the rise of generative artificial intelligence technologies such as generative adversarial networks and diffusion models, as well as the rapid development of ultra-large-scale pre-trained language models, further highlights the strategic value of deep learning model architecture innovation. However, traditional deep learning model architectures highly rely on manual design by domain experts and have the following technical bottlenecks: (1) The design process highly depends on expert experience, resulting in a long R & D cycle and high labor costs; (2) The generalization ability of fixed architectures is limited and it is difficult to meet the requirements of complex scenarios such as multi-modal data fusion and cross-domain migration. As an automated architecture design method, neural architecture search provides a more efficient solution, which can effectively mine the dependencies of architecture parameters, reduce the complexity and time cost of manual design, discover more efficient and adaptable network architectures, and improve model performance. Therefore, it is crucial to explore more advanced and efficient neural architecture search technologies.

[0003] The two main optimization strategies relied on by existing neural architecture searches are gradient algorithms and evolutionary algorithms. Among them, in the face of the huge search space of neural architecture search problems, methods based on evolutionary algorithms have excellent global search capabilities and can effectively explore the entire search space. However, such methods may require a large number of algebraic iterations to obtain high-quality solutions, and the convergence speed is slow. Methods based on gradient algorithms use gradient information for optimization and can achieve fast convergence in local areas, accelerating the search process. However, gradient algorithms are limited to the vicinity of the current solution and are prone to falling into local optima. In addition, no matter which optimization algorithm is used, a large number of architectures need to be comprehensively trained during the iterative search process for evaluation, and the time cost of comprehensively training an architecture may be as long as several weeks or even months, which is unbearable for ordinary researchers.

[0004] In order to accelerate the evaluation of architecture performance, existing research work focuses on training a performance predictor in advance using limited data samples and then using this predictor to evaluate other candidate architectures to assist neural architecture search. However, most existing performance predictors choose the regression model framework and focus on minimizing the mean squared error during training to pursue a high degree of closeness between the predicted value and the true value, but often ignore the relative ranking relationship between architecture performances. This approach is prone to the problem of unordered ranking, that is, high-performance architectures are severely underestimated in the predicted ranking. Summary of the Invention

[0005] Objective of the Invention: Aiming at the deficiencies of the prior art, the technical problem to be solved by the present invention is to provide a multi-objective hybrid evolutionary neural architecture search method and system assisted by a superiority classifier.

[0006] The method includes the following steps:

[0007] Step 1: Design and train an autoencoder based on contrastive learning to extract the features of the network architecture, and map the complex and disordered search space to a continuous coding space that is orderly distributed according to performance advantages and disadvantages.

[0008] Step 2: Design and train an end-to-end superiority classifier based on the comparison of architecture performance advantages and disadvantages. The superiority classifier can select neural architectures that perform excellently in more than two evaluation metrics from the candidate architectures.

[0009] Step 3: Initialize an architecture population from the search space by random sampling, and comprehensively train the individuals in the population to achieve real evaluation.

[0010] Step 4: Take the current population as the parent generation, generate candidate architectures, extract the features of the candidate architectures using the autoencoder, and input the extracted latent feature vectors into the superiority classifier to predict the fitness values of the candidate architectures; select potential architectures as the offspring architectures according to the predicted fitness values, and conduct real evaluation on the offspring architectures; subsequently, merge the offspring architectures and the parent architectures, and select the optimal architectures through the environmental selection strategy to update the population.

[0011] Step 5: Evaluate the diversity of the new population through the projection and clustering method. If the population diversity is excellent, perform local search optimization on the individuals in the population; otherwise, activate the diversity enhancement module, and re-select diverse individuals from the merged offspring architectures and parent architectures to update the population.

[0012] Step 6: Repeat Step 4 to Step 5 until the population performance converges, and finally output a set of globally optimal architectures; the globally optimal architectures refer to the set of Pareto front architectures formed in multi-objective optimization; in the large-scale image recognition scenario, the obtained globally optimal architectures are directly deployed in the image classification task of the CIFAR-10 dataset. Input a natural image, and the architecture automatically extracts the image features and outputs the corresponding class labels.

[0013] Step 1 specifically includes the following steps:

[0014] Step 1.1, Convert the architecture into vector encoding: Use the NASNet search space. The NASNet search space only searches for the optimal basic units and constructs the network architecture by stacking the optimal basic units. In the NASNet search space, each unit consists of two input nodes, five intermediate nodes, and one output node.

[0015] Use to represent the s-th operation in the predefined operation space. Nodes 0 and 1 represent the input nodes, and node 7 represents the output node. For the NASNet search space, only the five intermediate nodes need to be encoded. Each intermediate node is represented by a vector of length 4. The first two elements of the vector represent the numbers of the two predecessor nodes of the node, and the last two elements represent the operation types corresponding to the edges connecting the two predecessor nodes to the node. Subsequently, the vectors of the five intermediate nodes are concatenated in sequence to obtain a one-dimensional vector encoding , which is used to initially represent the architecture ;

[0016] Step 1.2, Design an autoencoder: The autoencoder includes an encoder and a decoder. The working process of the autoencoder is as follows: Take the vector encoding X as the input, and after being processed by the encoder, output the latent vector , where represents the output of the encoder when is the input; Subsequently, the new latent vector obtained through local search optimization is re-decoded into a discrete encoding with practical meaning, which is used to represent a real-existing architecture. Use the decoder to decode the latent vector and output the reconstructed vector , where represents the output of the decoder when is used as the input;

[0017] Step 1.3, Construct training samples: Construct a training set for the autoencoder , where represents the i-th triple sample, is the anchor sample in the i-th triple sample, is the positive sample in the i-th triple sample, is the negative sample in the i-th triple sample, is the number of all triple samples; is the number of all triple samples; is the number of all triple samples;

[0018] Step 1.4, Train the autoencoder: Calculate the Euclidean distance between the latent vector obtained from the anchor sample and the latent vector obtained from the positive sample ​ , calculate the latent vector and the latent vector obtained from the negative samples to calculate the Euclidean distance ; the optimization goal is to minimize and maximize . An autoencoder is optimized using the following triplet loss function : where m is a hyperparameter used to control the distance;

[0019] ,

[0020] An autoencoder is trained using the following reconstruction loss function

[0021] : where

[0022] ,

[0023] represents the length of the vector encoding, and and represent the element values at the -th position of the anchor sample and the vector encoding reconstructed by the decoder at the -th position, respectively;

[0024] Finally, by setting the parameter to adjust the contribution balance of the two loss functions and , a combined loss function is used to calculate the loss value during training, and the trainable parameters of the autoencoder are updated through backpropagation. The training is repeated iteratively until the model converges.

[0025] Step 1.3 includes: First, architectures are selected from the search space through a random sampling strategy, and the vector encodings corresponding to the architectures are added to the architecture pool , where represents the -th sample in the architecture pool; Second, a sample is randomly selected from the architecture pool as the anchor sample ; Then, through random sampling again, a sample is drawn from the architecture pool, and the performance similarity between the sample and the architecture corresponding to the anchor sample is calculated. For any two samples and , the performance similarity is calculated using the following formula:

[0026] ,

[0027] Among them, and respectively represent the true performance values of the architectures corresponding to the samples and the true performance values of the architectures corresponding to the samples ; e represents the natural constant; if is greater than the threshold , it means that the performances of the two samples are similar, then the currently extracted sample is used as the positive sample of the anchor sample ; otherwise, it means that the performances of the two samples are not similar, and the currently extracted sample is used as the negative sample of the anchor sample ; subsequently, continue to randomly extract samples from the architecture pool until a positive sample and a negative sample are found for the current anchor sample to form a triple sample ; repeat step 1.3 in a loop for a total of times to obtain triple samples.

[0028] In step 2, first, construct paired training samples for the dominant classifier and obtain labels, ; among them, represents the i-th paired sample, represents the th architecture 's vector encoding the latent feature obtained by passing through the encoder, represents the h-th architecture 's vector encoding the latent feature obtained by passing through the encoder, represents the label of the sample pair, when , ; otherwise, ;

[0029] Use the following binary cross-entropy loss function to train and optimize the model:

[0030] ,

[0031] where, represents being superior to 's probability;

[0032] Propose a paired ranking loss function as a penalty term for training :

[0033] ,

[0034] where exp is the natural exponential function, is a sign function, defined as:

[0035] ,

[0036] Finally, by setting the parameter to control the impact of the penalty term on model training, the combined loss function is adopted during the training process to calculate the loss value.

[0037] In step 3, N neural network architectures are selected from the search space as the initial population through random sampling , where N represents the size of the population; subsequently, the individuals in the population are comprehensively trained to achieve a true evaluation; comprehensive training means that on the target dataset, the network architecture is completely trained and the final performance is evaluated on the validation set.

[0038] Step 4 includes:

[0039] In step 4.1, first, two parent architectures are selected from the current population each time through the tournament selection strategy, and crossover and mutation operations are performed on the two parent architectures to generate offspring, and step 4.1 is repeated until candidate architectures are generated; where represents the population at the -th generation of evolution;

[0040] In step 4.2, the dominance classifier is used to predict the fitness scores of the candidate architectures: first, the score of each candidate architecture is initialized to 0, where represents the predicted score of the -th architecture when the weight of the dominance classifier is ; subsequently, an autoencoder is adopted to obtain the latent vector of each candidate architecture, and the dominance classifier predicts the relative superiority relationship between each architecture and other candidate architectures based on the latent vector; for any two architectures and in the search space, if the probability that the output by the dominance classifier is better than is , then: ,

[0041] ,

[0042] where the parameter is used to control the contribution of the probability to the score; subsequently, the score As the first objective, combined with other objectives, non-dominated sorting and crowding distance calculation are performed, and the top performing architectures are selected as the current offspring architectures , and the offspring architectures are further evaluated in a real environment through comprehensive training;

[0043] Finally, the parent architectures and the offspring architectures are combined, and environmental selection is performed on the combined population using the non-dominated sorting genetic algorithm to select the architectures that perform excellently in more than two evaluation metrics for updating the population .

[0044] Step 5 specifically includes the following steps:

[0045] Step 5.1, population diversity assessment:

[0046] The new population obtained in Step 4 is clustered by the method of hyperplane projection, and the population diversity is evaluated using the clustering quality index. First, the unit vector is defined in the normalized objective scale: where n is the number of optimization objectives,

[0047] ,

[0048] where represents the objective value of an individual in the new population, represents the objective value of the individual in the new population on the th optimization objective, , represents the maximum value of the individual in the new population on the th objective;

[0049] The hyperplane is constructed:

[0050] ,

[0051] where is the constant term; is the dot product of and , representing the projection of in the direction of ;

[0052] The orthogonal projection of the objective value of the individual on the hyperplane is calculated:

[0053] ,

[0054] Among them, represents the squared norm of the vector ;

[0055] Project all individuals in the population onto the hyperplane to obtain the projected population , which consists of the orthogonal projections of all individuals ;

[0056] Cluster the projected population to evaluate the diversity of the population: Using the K-cluster clustering method, divide the projected population into subpopulations , where ; represents the th projected subpopulation obtained after clustering. Each projected subpopulation represents a group of similar individuals in the target space;

[0057] Subsequently, use the clustering quality index Q as an indicator to evaluate the population diversity:

[0058] ,

[0059] Among them, represents the number of individuals in the kth projected subpopulation ; represents the th orthogonal projection in the kth projected subpopulation ; represents the cluster center of the kth projected subpopulation ;

[0060] Finally, set the diversity threshold . When , it indicates that the current population has excellent diversity, and execute Step 5.2; otherwise, execute Step 5.3;

[0061] Step 5.2, local search optimization: Randomly select half of the architectures from the new population for local optimization. For the th architecture to be optimized , set the reference point with the objective vector , and calculate the normalized weight vector ; Secondly, use the encoder to extract the architecture Latent vector ; Subsequently, the achievement scalarizing function is used as the objective function :

[0062] ,

[0063] wherein, represents the objective vector of the corresponding architecture of the latent vector , is a positive number, represents the sum of the values to be optimized for all architectures;

[0064] Next, sequential quadratic programming is used on the latent space to perform gradient optimization on the objective function :

[0065] ,

[0066] wherein, the latent vector will be iteratively updated during the optimization process, and finally the optimized latent vector is obtained;

[0067] For the optimized latent vector , the decoder in step 1 is used for reconstruction to obtain the decoded architecture , and through comprehensive training, a real evaluation is performed on , and the dominance relationship between and is compared. If dominates , then in the current population , is used to replace ; otherwise, the population remains unchanged; finally, the non-dominated sorting and crowding distance are recalculated for the newly optimized population;

[0068] Step 5.3, activate the diversity enhancement module to enhance population diversity: Re-select individuals from the set of the merged parental architectures and the offspring architectures to reconstruct the population. First, project and cluster the merged architectures to obtain sub-populations ; Second, perform non-dominated sorting within each sub-population. The j-th layer non-dominated solution of the sub-population is , and the multi-layer Pareto front of each sub-population is recorded using . Subsequently, construct an empty set Traverse the new population after diversity optimization in the order of the levels of the frontiers and select individuals to fill until the number of individuals in reaches the population size ; finally, replace the current population and recalculate the non-dominated sorting and crowding distance of the new population.

[0069] The present invention also provides a multi-objective hybrid evolutionary neural architecture search system based on a dominant classifier assistant implemented according to the above method, including:

[0070] An autoencoder based on contrastive learning: used to map the unordered search space to a low-dimensional coding space that is orderly distributed according to performance merits and demerits, and is also used to reconstruct the locally optimized coding vector to obtain the decoded architecture;

[0071] A dominant classifier based on the comparison of architecture performance merits and demerits: used to predict the superiority and inferiority relationship between architectures, calculate the fitness score of the architectures, and select excellent neural architectures;

[0072] A hybrid evolutionary optimization module: used to evaluate the diversity of each generation of the population, perform local search optimization on the population with excellent diversity, and enhance the diversity of the population.

[0073] The present invention also provides an electronic device, including a processor and a memory, where the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the above method.

[0074] The present invention also provides a storage medium storing a computer program or instruction, and when the computer program or instruction runs on a computer, it executes the steps of the above method.

[0075] The present invention constructs an efficient multi-objective neural architecture search method assisted by a dominance classifier based on hybrid evolution to effectively solve the problems of slow convergence speed, unordered ranking, and lack of regularity in feature learning existing in the existing proxy-assisted neural architecture search method based on evolutionary algorithms. This method combines the global search ability of evolutionary algorithms and the local optimization ability of gradient algorithms, which can effectively alleviate the problems of slow convergence speed of traditional evolutionary algorithms and the tendency of gradient algorithms to fall into local optima. In addition, aiming at the problem of unordered ranking of existing performance predictors, the present invention constructs a dominance classifier based on the comparison of the advantages and disadvantages between architectures to achieve efficient and reliable auxiliary evaluation of architectures. Moreover, in order to effectively guide the auxiliary evaluation of the dominance classifier, the present invention also proposes an autoencoder based on contrastive learning to map the complex and unordered search space into a low-dimensional latent space that is orderly distributed according to performance advantages and disadvantages. Compared with the existing proxy-assisted multi-objective neural architecture search algorithms, the method proposed by the present invention exhibits higher prediction accuracy and search efficiency, bringing a major breakthrough to the field of neural architecture search.

[0076] The beneficial effects of the present invention are as follows: The present invention proposes a multi-objective hybrid evolution neural architecture search method assisted by a dominance classifier, aiming to optimize the efficiency and performance of neural architecture search. This method proposes a multi-objective neural architecture search framework assisted by a dominance classifier based on hybrid evolution, and introduces a local search and diversity enhancement module on the basis of the standard evolutionary multi-objective optimization algorithm to accelerate the convergence speed and improve the quality of solutions. At the same time, a dominance classifier based on the comparison of the advantages and disadvantages between architectures is proposed, and the performance ranking information is introduced to reliably enhance the dominance classification ability of the model, so as to efficiently select network architectures that perform excellently in multiple evaluation indicators from a large number of candidate architectures, thereby assisting the computationally expensive fitness evaluation link in neural architecture search. In addition, the present invention also proposes an autoencoder based on contrastive learning to map the complex and unordered search space into a low-dimensional coding space that is orderly distributed according to performance advantages and disadvantages, aiming to improve the quality of architecture feature representation, promote the search process, and provide continuous coding for local search optimization. The innovative method of the present invention greatly reduces the consumption of computing resources while ensuring the optimization of multiple conflicting objectives, laying a solid foundation for the wide promotion of neural architecture search technology in practical applications. Compared with the existing methods, the present invention exhibits more significant practical value and broader application prospects, bringing breakthrough progress and innovation to the field of evolutionary neural architecture search. Brief Description of the Drawings

[0077] Figure 1 It is the overall framework diagram of the present invention.

[0078] Figure 2 It is a schematic diagram of the NASNet search space.

[0079] Figure 3 Schematic diagram of vector encoding.

[0080] Figure 4 Schematic diagram of the auto - encoder structure.

[0081] Figure 5 Schematic diagram of the structure of the dominance classifier.

[0082] Figure 6 Effect comparison of the auto - encoder Figure 1 .

[0083] Figure 7 Effect comparison of the auto - encoder Figure 2 .

[0084] Figure 8 Effect diagram of hybrid evolution.

[0085] Figure 9 Overall effect diagram of the method. Specific implementation manner

[0086] The following further specifically describes the present invention in conjunction with the accompanying drawings and specific implementation manners, and the above - mentioned and / or other advantages of the present invention will become clearer.

[0087] As Figure 1 shown, an embodiment of the present invention provides a multi - objective hybrid evolutionary neural architecture search method assisted by a dominance classifier, including the following steps:

[0088] Step 1, construct an auto - encoder based on contrastive learning for feature extraction of network architectures, map the complex and disordered search space to a low - dimensional encoding space that is orderly distributed according to performance advantages and disadvantages, aiming to improve the quality of architecture feature representation and provide continuous encoding for gradient - based local search. Specifically, it includes:

[0089] Step 1.1, taking the cell - based NASNet search space as shown in Figure 2 as an example, it only searches for the optimal basic cells instead of the entire network, and then constructs the network architecture by stacking these cells. In the NASNet space, there are mainly two types of cells to be searched: standard cells and down - sampling cells. Among them, the standard cells are mainly used for feature extraction and are composed of two input nodes, five intermediate nodes, and one output node. Each intermediate node can select two specific operations and connect to the previously existing nodes. The selectable operation space includes: Depthwise separable convolution, Depthwise separable convolution, Max pooling, Average pooling, Atrous convolution, Dilated convolution and skip connections, a total of 7 operation types. The structure of the downsampling unit is similar to the standard unit, but contains more downsampling operations, such as pooling or convolution operations with a stride of 2, mainly used for feature dimensionality reduction. In the present invention, the same encoding method is adopted for the standard unit and the downsampling unit.

[0090] The vector encoding of each unit is jointly defined by the connection relationship and the operation type, such as Figure 3 shown. Use to represent the th operation in the predefined operation space, which are in turn depthwise separable convolution, pointwise separable convolution, dilated convolution, dilated convolution, max pooling, average pooling, and skip connection. Nodes 0 and 1 represent the input nodes, and node 7 represents the output node. For the NASNet search space, only five intermediate nodes (nodes 2 - 6) need to be encoded. Each intermediate node is represented by a vector of length 4. The first two elements of the vector represent the numbers of the two predecessor nodes of the node, and the last two elements represent the operation types corresponding to the edges connecting the two predecessor nodes to this node. For example, Figure 3 the encoding of node 2 in is [0, 1, 6, 4], indicating that the two predecessor nodes of this node are node 0 and node 1 respectively, and the corresponding operation types are . Subsequently, the vectors of the five intermediate nodes are concatenated in sequence to obtain a one-dimensional vector encoding , which is used to initially represent the architecture.

[0091] Step 1.2, construct an autoencoder as shown in Figure 4 , which consists of an encoder and a decoder with a symmetric structure, maps the discrete and disordered space to an ordered latent feature space, and ensures that the feature vector can be decoded and reconstructed. Specifically, take the vector encoding as the input, and after being processed by the encoder, output the latent vector , where represents the output of the encoder with as the input. The encoder realizes the continuity of the discrete vector encoding. In addition, the decoder can decode the latent vector and output the reconstructed vector , where represents the output of the decoder with as the input. The encoder and the decoder need to be trained simultaneously, but can work independently.

[0092] Step 1.3, construct a training set for the autoencoder, where Indicates the triplet samples, For the Anchor samples in triplet samples, For the positive sample, For the negative samples, is the number of all triple samples. Specifically, first, we select architecture, and add its corresponding vector encoding to the architecture pool Among them Indicates the first Second, a sample is randomly selected from the architecture pool as the anchor sample. Then, we randomly select a sample from the architecture pool and calculate the performance similarity between the sample and the architecture corresponding to the anchor sample. and The performance similarity calculation formula is: .in, and Represents samples and Corresponding architecture and The true performance value, e represents a natural constant; if Greater than threshold , indicating that the performance of the two samples is similar, the currently extracted sample is used as the positive sample of the anchor sample Otherwise, it means that the performance of the two samples is not similar, and the currently extracted sample is used as the negative sample of the anchor sample. Then, continue to randomly extract samples from the architecture pool until a positive sample and a negative sample are found for the current anchor sample, forming a triplet sample The process repeats itself times, get triplet samples.

[0093] Step 1.4, calculate the anchor point samples respectively With positive samples The obtained latent vector and The Euclidean distance between and by anchor point samples With negative samples The obtained latent vector and The Euclidean distance between The goal of optimization is to minimize and maximize , adopt the following triplet loss function Optimized autoencoder:

[0094] ,

[0095] where m is a hyperparameter used to control the distance, ensuring that the distance between the anchor sample and the positive sample in the latent feature space is at least smaller than the distance between the anchor sample and the negative sample by m.

[0096] In addition, the original objective of the autoencoder is to minimize the reconstruction loss, and the following reconstruction loss function is adopted to train the autoencoder:

[0097] ,

[0098] where represents the length of the vector encoding , and respectively represent the element value at the th position of the vector encoding of the anchor sample and the vector encoding reconstructed by the decoder. This loss ensures that the decoder can accurately reconstruct the input data.

[0099] Finally, by setting the parameter to adjust the contribution balance of the two loss functions, the combined loss function is used to calculate the loss value during training, and the trainable parameters of the autoencoder are updated through backpropagation. The training is repeated iteratively until the model converges.

[0100] Step 2, construct a dominance classifier as shown in Figure 5 to predict the superiority relationship between candidate architectures. For any two architectures and in the search space, if the relationship between the true performance values and of the two architectures is , then the fitness prediction scores and assigned by the dominance classifier should also satisfy . Among them, represents the trainable weight parameter of the dominance classifier, and when the weight of the dominance classifier is , the predicted score of architecture is . The advantage classifier inputs the potential feature vectors of two neural networks to be compared and simultaneously, and finally outputs the probability and that the performance of architecture is better than that of architecture . .

[0101] First, construct paired training samples for the advantage classifier and obtain labels, denoted as . Among them, represents the i-th paired sample, is the number of all training samples, and respectively represent the vector encodings and of architecture and and the potential features obtained by the encoder, represents the label of this sample pair. When , ; otherwise, .

[0102] Before the search, perform separate supervised training on the advantage classifier. Based on the fact that the goal of the advantage classifier is to learn the relative superiority and inferiority relationship between architecture pairs, use the following binary cross-entropy loss function to train and optimize the model:

[0103] ,

[0104] where is the label of the sample pair, is the output value of the model, representing the probability that architecture is better than architecture .

[0105] In addition, the present invention proposes a pairwise ranking loss function as a penalty term for training by introducing performance ranking information to reliably enhance the advantage classification ability of the model during training :

[0106] ,

[0107] where is a sign function, defined as:

[0108] ,

[0109] This indicates that when the relative ranking relationship predicted by the model is correct, that is, , Output "+1", and at this time the penalty term has a small value, indicating that a small penalty is imposed on the correctly predicted architecture pair; while when the prediction is incorrect, that is , the sign function outputs "-1", and at this time the penalty term has a large value, forcing the model to adjust the parameters during training to reduce the situation of incorrect ranking.

[0110] Finally, by setting the parameter to control the influence of the penalty term on the model training, the combined loss function is adopted during training to calculate the loss value. The penalty term can effectively assign a higher optimization weight to the architecture pairs with incorrect ranking, so that the model can more accurately learn the correct relationship between the advantages and disadvantages of the architectures.

[0111] Step 3, through random sampling, select neural network architectures from the search space as the initial population , where represents the size of the population, and the setting of this value generally needs to balance the size of the search space and the computational cost. For a smaller search space, may also have a smaller value, around dozens; while for a larger search space, may take values in the hundreds or even thousands. Subsequently, the individuals in the population are comprehensively trained to achieve a real evaluation. Comprehensive training means that on the target dataset (such as CIFAR-10, CIFAR-100, etc.), the network architecture is fully trained, and its final performance is evaluated on the validation set. Usually, the validation set accuracy of the classification task or the error metric of the regression task is used as the measurement standard to ensure that the evaluation result can truly reflect the performance of the architecture in the actual task, and the value obtained from the real evaluation is the true performance value of the architecture.

[0112] Step 4, first select two parent architectures from the current population each time through the tournament selection strategy, perform crossover and mutation operations on them to generate offspring, and repeat the above process until M candidate architectures are generated. Among them, represents the population at the th generation of evolution, and the value of M is generally set to about 20 to 50 times the population size N.

[0113] Subsequently, use the dominance classifier to predict the fitness scores of the candidate architectures. Specifically, initially set the score of each candidate architecture to 0, and use an autoencoder to obtain the latent vector of each candidate architecture. Based on these latent vectors, the dominance classifier predicts the relative superiority and inferiority relationship between each architecture and other candidate architectures. For any two architectures in the search space and if the probability that the dominant classifier outputs is better than is , then:

[0114] ,

[0115] wherein, " " indicates that in this comparison, is considered better than another architecture, 's score is incremented by one. The parameter is used to control the contribution of the predicted probability value to the score, is added to alleviate the phenomenon of tied scores. Subsequently, the score , that is, the predicted fitness value, is used as the first objective, and non-dominated sorting and crowding distance calculation are performed in combination with other objectives to select the top N architectures with the best performance as the current offspring architectures , and these offspring architectures are further evaluated in reality through comprehensive training. It should be noted that since other objectives (such as the number of parameters, the number of floating-point operations, etc.) are cheap objectives and can be calculated in a very short time, the dominant classifier only needs to predict the classification accuracy performance of the architecture.

[0116] Finally, the parent architectures and the offspring architectures are merged, and environmental selection is performed on the merged population using the non-dominated sorting genetic algorithm to select N architectures that perform excellently in multiple evaluation metrics for updating the population .

[0117] Step 5, a local search module and a population diversity enhancement module are introduced based on the non-dominated sorting genetic algorithm to accelerate the convergence speed and improve the quality of the solution. Specifically, it includes:

[0118] Step 5.1, cluster the new population obtained in Step 4 by the method of hyperplane projection, and evaluate the population diversity using the clustering quality index. First, in the high-dimensional objective space, since the numerical ranges of different objective functions may vary greatly, directly analyzing using the original objective values may cause the optimization process to be biased towards specific objectives. Therefore, first normalize and project the objective values of the individuals to eliminate the influence of different objective scales. Define the unit vector to normalize the objective scale:

[0119] ,

[0120] where n is the number of optimization objectives, represents the objective value of an individual in the new population, where represents the objective value of an individual in the new population on the th optimization objective, represents the maximum value of an individual in the new population on the th objective. A hyperplane is constructed through this vector , which is defined as:

[0121] ,

[0122] where, is the constant term, used to adjust the position of the hyperplane. Generally speaking is taken as 0, indicating that the hyperplane passes through the origin. is the and dot product, representing 's projection in the direction. Subsequently, the objective value of the individual is calculated on the hyperplane 's orthogonal projection PI, and the projection formula is as follows:

[0123] ,

[0124] where, represents the square of the norm of the vector , that is, the inner product of with itself. After projecting all individuals in the population onto the hyperplane, the projected population is obtained, is composed of all individuals' PIs. The projection operation ensures the scale consistency of the individual distribution, thereby improving the stability of subsequent clustering.

[0125] Secondly, the projected population is clustered to evaluate the diversity of the population. The present invention adopts the K-clustering method to divide the projected population into K sub-populations , represents the kth projected sub-population obtained after clustering, and each represents a group of individuals similar in the objective space. Through this process, it can be ensured that during the optimization process, each region can be fully explored to enhance the diversity of the Pareto front.

[0126] Subsequently, the clustering quality index is used as an index to evaluate the population diversity, and the calculation formula is:

[0127] ,

[0128] where, represents the sub-population The number of individuals in denotes the r-th orthogonal projection in denotes the k-th subpopulation of the clustering center, denotes the orthogonal projection to the clustering center of the Euclidean distance.

[0129] The clustering quality index reflects the degree of dispersion of the population. The larger the value, the more uniform the distribution of the population in the target space and the higher the diversity of the Pareto front. On the contrary, if the value is too small, it means that the individuals in the population are overly concentrated, which may lead to convergence to a local optimal solution. In this case, set the diversity threshold , when it indicates that the current population has excellent diversity, and step 5.2 is executed; otherwise, step 5.3 is executed.

[0130] Step 5.2, if the new population obtained in step 4 has excellent diversity, then adopt a local search strategy to further optimize the individuals in the population to accelerate convergence and improve the quality of the solution; otherwise, directly enter step 5.3. First, randomly select half of the architectures from the new population for local optimization. For each architecture to be optimized , set the reference point with its objective vector , and calculate its normalized weight vector . Secondly, use the encoder to extract the latent vector of the architecture, ensuring that the vector is continuous and can be optimized by the gradient. Subsequently, use the achievement scalarization function as the objective function to enable the optimization process to make dynamic trade-offs among multiple objectives:

[0131] ,

[0132] where denotes the objective vector of the architecture corresponding to the latent vector , is a very small positive number used to enhance the convergence stability, denotes the sum of the values of all architectures to be optimized. Then, perform gradient optimization on the achievement scalarization function using sequential quadratic programming in the latent space:

[0133] ,

[0134] Among them, the latent vector will be iteratively updated during the optimization process, and finally the optimized latent vector is obtained.

[0135] For the optimized latent vector , the decoder in Step 1 is used for reconstruction to obtain the decoded architecture , and through comprehensive training, is truly evaluated, and the dominance relationship between and is compared. If dominates , then in the current population , is used to replace ; otherwise, the population remains unchanged. Finally, the non-dominated sorting and crowding distance of the newly optimized and updated population are recalculated.

[0136] Step 5.3, if the diversity of the new population obtained in Step 4 is poor, then the diversity enhancement module is activated, and individuals are reselected from the set of the merged parent architectures and the offspring architectures to reconstruct the population. First, the merged architectures are projected and clustered to obtain subpopulations . Second, non-dominated sorting is performed within each subpopulation, and is used to record the multi-layer Pareto fronts of each subpopulation, indicating that the -th subpopulation 's -th layer of non-dominated solutions is . Subsequently, an empty set is constructed as the newly optimized population in terms of diversity, and is traversed in the order of the levels of the fronts, and individuals are selected to fill . That is to say, first traverse the first-layer non-dominated solutions of each subpopulation in turn and fill them into , then traverse the second-layer fronts of each subpopulation in turn, and so on, until the number of individuals in reaches the population size N. Finally, is used to replace the current population , and the non-dominated sorting and crowding distance of the new population are recalculated.

[0137] Step 6: Repeat Step 4 and Step 5 until the population performance converges, and finally output a set of globally optimal architectures; the globally optimal architectures refer to the set of Pareto front architectures formed in multi-objective optimization, that is, a set of solutions that cannot improve in one objective without causing losses in another while meeting different performance requirements. These architectures have strong adaptability and can meet different search spaces and task requirements, and can be widely applied to applications such as large-scale image recognition, real-time video processing, and medical image analysis. For example, in the large-scale image recognition scenario, the obtained optimal neural network architecture can be directly deployed in image classification tasks of various datasets such as CIFAR-10. With an input of a natural image (such as a picture containing categories like cats, dogs, vehicles, etc.), the architecture can automatically extract image features and output the corresponding class labels, and its classification accuracy is significantly better than existing manually designed networks. In the real-time video processing task, the obtained optimal architecture can be embedded in edge devices to achieve fast recognition and object detection of video frames, ensuring low latency and high accuracy. In medical image analysis, this architecture can be used for lesion detection in lung CT images. After inputting the CT image, it can accurately locate the lesion area and distinguish between benign and malignant, which helps doctors in auxiliary diagnosis. This method can provide high-performance neural network architectures that perform excellently in multiple evaluation criteria (such as accuracy, model complexity, and inference speed) under limited computing resources and without relying on expert experience, providing strong technical support for the automated design and optimization of network architectures and the wide application of artificial intelligence in practical scenarios.

[0138] NAS-Bench-201 is a popular benchmark dataset for neural network architecture search algorithms. Its search space contains a total of 15,625 different deep neural network architectures, and the performance of these architectures in image classification tasks has been comprehensively evaluated on three datasets: CIFAR-10, CIFAR100, and Image-Net-16-120. In addition to recording the classification accuracy of the architectures, NAS-Bench-201 also provides computational cost metrics such as the number of floating-point operations and the number of parameters. The present invention conducts experiments based on the NAS-Bench-201 dataset to verify the effectiveness of the proposed method.

[0139] In the method of the present invention, an autoencoder based on contrastive learning is proposed to map the complex and disordered search space to a low-dimensional and continuous coding space distributed according to performance advantages and disadvantages, aiming to improve the quality of architecture feature representation and promote the search. To verify the effectiveness of the autoencoder, the t-SNE dimensionality reduction technique is used to visualize the feature space, and the feature space obtained by vector coding (such as Figure 6 shown) and the latent feature space obtained by the autoencoder (such as Figure 7 shown) are compared. Figure 6 andFigure 7 The color bar on the right shows the performance ranking of the architectures in the entire search space. The darker the color, the higher the performance ranking of the architecture; the lighter the color, the lower the performance ranking of the architecture. It is observed that in the feature space obtained by vector encoding without autoencoder processing, the distribution of the architectures is discrete and lacks regularity; while in the latent feature space obtained by the autoencoder, the distribution of the architectures is more compact and shows obvious clustering according to the performance. Architectures with similar rankings are mapped to adjacent positions in the latent space, showing high regularity. This feature learning method is beneficial to accelerating the subsequent evolutionary search and optimizing the local search.

[0140] The method proposed by the present invention adopts a hybrid evolutionary multi-objective algorithm, which introduces local search and diversity enhancement strategies on the basis of the traditional non-dominated sorting genetic algorithm. In the experiment to verify the hybrid evolutionary algorithm, the inverse generational distance is used as the evaluation index to evaluate the performance of each generation of the population in the multi-objective task. The inverse generational distance is an important index to measure the multi-objective optimization algorithm, which represents the degree of closeness between the obtained Pareto front solution set and the global Pareto front. The smaller this value is, the higher the degree of coverage of the generated Pareto front to the global front, and the better the quality of the solution; the larger this value is, the farther the generated front is from the true Pareto front, and the worse the performance of the solution. Under the condition of evolving for 20 generations, the hybrid evolutionary algorithm adopted by the present invention is compared with the traditional non-dominated sorting genetic algorithm, and the inverse generational distance values of each generation of the population in the evolutionary process of the two algorithms are respectively shown, as Figure 8 shown. The results show that the broken line of the hybrid evolutionary algorithm shows a significant downward trend in the first 5 generations and tends to be stable after the 5th generation, showing a fast convergence speed and excellent solution set quality; in contrast, the inverse generational distance value of the non-dominated sorting genetic algorithm decreases slowly and fluctuates greatly, and its final convergence and solution set quality are both inferior to those of the hybrid evolutionary algorithm.

[0141] The present invention compares the Pareto front obtained after evolving for 20 generations by the proposed multi-objective hybrid evolutionary neural architecture search method assisted by a dominance classifier with the global Pareto front of the NAS-Bench-201 search space, as Figure 9 shown. The abscissa represents the classification error rate of the architecture on the CIFAR-10 dataset, and the ordinate represents the number of parameters of the architecture. The results show that the obtained Pareto front and the global Pareto front coincide highly, thus verifying that the method proposed by the present invention can effectively search for high-performance architectures in multi-objective optimization. When applied to the image classification task, it has extremely high benefits in terms of classification accuracy and model scale.

[0142] The proposed multi-objective hybrid evolutionary neural architecture search method assisted by a dominance classifier in the present invention is applied to the image classification task, and the publicly available large-scale image classification dataset CIFAR-10 is used as the verification platform. In the NASNet search space, the recognition accuracy and the number of model parameters are used as the joint optimization objectives. Under the condition that the population size is After evolving for 50 generations, the optimal network architecture is obtained. After the optimal architecture is fully trained on the CIFAR-10 test set, the experimental results show that its classification accuracy is as high as 97.34%, and the number of model parameters is only 3.71M. The entire search process takes only 0.3 GPU days. Compared with the classic manually designed architecture DenseNet, the latter has an accuracy of 96.26% and 27.2M parameters on CIFAR-10. The architecture searched by the present invention not only improves the accuracy but also reduces the number of parameters by about 86%. At the same time, compared with the lightweight representative architecture ResNet-101 (accuracy 93.57%, number of parameters 1.7M), this method achieves a more than 5 percentage point increase in accuracy while only increasing about 2.01M parameters, further demonstrating the comprehensive advantages of the present invention in the trade-off between accuracy and model complexity.

[0143] This method has good adaptability and can flexibly cope with diverse search spaces and task requirements. By dynamically evaluating the architecture advantages, it effectively avoids the problems of inefficient random search and experience dependence existing in traditional manual design, showing broad generality. Especially in scenarios with high requirements for model performance, scale limitation, and iteration speed, such as large-scale image recognition, real-time video processing, and medical image analysis, this method can efficiently generate neural network architectures that meet the requirements, providing strong support for the implementation of artificial intelligence technology in practical applications.

[0144] The multi-objective hybrid evolutionary neural architecture search method assisted by a dominance classifier provided by the present invention can efficiently identify neural network architectures with excellent performance. This method significantly reduces the time cost of manual parameter tuning and structure design, and at the same time has good generalization ability. By introducing a multi-objective optimization strategy, it not only achieves a better trade-off between accuracy and model complexity, but also can flexibly adjust the search direction according to different task requirements, improving the overall practicality of the model. In practical applications, this method is applicable to basic tasks such as image classification, and can also be extended to multiple deep learning fields. For example, in the search for large model architectures, this method can quickly locate the optimal model design, significantly shorten the search cycle, reduce the consumption of computing resources, and provide an efficient parameter tuning direction for building large language models such as Transformer and ChatGPT. In addition, this method is also applicable to complex tasks such as object detection, speech recognition, and natural language processing, and can further enhance the model performance by optimizing the structure parameters. The method proposed by the present invention shows good generality and flexibility in multi-task scenarios, provides strong support for designing efficient and intelligent deep learning models, and has broad application prospects.

[0145] The present invention provides a multi-objective hybrid evolutionary neural architecture search method and system assisted by a dominance classifier. There are many methods and ways to specifically implement this technical solution. The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by existing technologies.

Claims

1. A multi-objective hybrid evolutionary neural architecture search method assisted by a dominance classifier, characterized in that, It includes the following steps: Step 1: Design and train an autoencoder based on contrastive learning to extract the features of the network architecture and map the disordered search space to a continuous coding space with orderly distribution according to performance. This includes: designing an autoencoder: the autoencoder includes an encoder and a decoder. The workflow of the autoencoder includes: taking the vector code X as input, and outputting the potential vector after being processed by the encoder. ,in Represents the output of the encoder when X is input; then, the new potential vector obtained by local search optimization is re-decoded into a discrete code with practical meaning to represent a real architecture, and the decoder is used to decode the potential vector and output the reconstructed vector ,in represents the output of the decoder when L is used as input; Step 2, design and train an end-to-end advantage classifier based on comparing the performance of architectures, where the advantage classifier can select a neural architecture that performs excellently in more than two evaluation metrics from candidate architectures; including: First, construct paired training samples for the advantage classifier and obtain labels, wherein, represents the th paired sample, represents the th architecture 's vector encoding the latent feature obtained by passing through the encoder, represents the th architecture 's vector encoding the latent feature obtained by passing through the encoder, represents the label of the sample pair. When , ; otherwise, ; Step 3: Initialize an architecture population from the search space through random sampling, and comprehensively train the individuals in the population to achieve real evaluation. Step 4: Use the current population as the parent generation to generate candidate architectures. Extract features of the candidate architectures using an autoencoder, and input the extracted latent feature vectors into the dominance classifier to predict the fitness values of the candidate architectures. Select potential architectures as the offspring architectures according to the predicted fitness values, and conduct real evaluation on the offspring architectures. Subsequently, merge the offspring architectures and the parent architectures, and select the optimal architectures through the environmental selection strategy to update the population. Step 5: Evaluate the diversity of the new population through projection and clustering methods. If the population diversity is excellent, perform local search optimization on the individuals in the population. Otherwise, activate the diversity enhancement module to reselect diverse individuals from the merged offspring architectures and parent architectures to update the population. Step 6: Repeat Step 4 to Step 5 until the population performance converges, and finally output a set of globally optimal architectures. The globally optimal architectures refer to the set of Pareto front architectures formed in multi-objective optimization. In the large-scale image recognition scenario, the obtained globally optimal architectures are directly deployed in the image classification task of the CIFAR-10 dataset. Input a natural image, and the architecture automatically extracts the image features and outputs the corresponding class labels.

2. The method according to claim 1, characterized in that Step 1 specifically includes the following steps: Step 1.1: Convert the architecture into vector encoding. Adopt the NASNet search space, which only searches for the optimal basic units, and construct the network architecture by stacking the optimal basic units. In the NASNet search space, each unit consists of two input nodes, five intermediate nodes, and one output node. Use denote the sth operation in the predefined operation space. Nodes 0 and 1 represent input nodes, and node 7 represents the output node. For the NASNet search space, only five intermediate nodes need to be encoded; each intermediate node is represented by a vector of length 4. The first two elements of the vector represent the numbers of the two predecessor nodes of the node, and the last two elements of the vector represent the operation types corresponding to the edges connecting the two predecessor nodes to the node. Subsequently, the vectors of the five intermediate nodes are concatenated in sequence to obtain a one-dimensional vector encoding X, which is used to preliminarily represent the architecture ; Step 1.2: Design an autoencoder. Step 1.3, constructing training samples: constructing a training set for the autoencoder , where represents the i-th triple sample, is the anchor sample in the i-th triple sample, is the positive sample in the i-th triple sample, is the negative sample in the i-th triple sample, is the number of all triple samples; Step 1.4, training the autoencoder: Calculate the latent vector obtained from the anchor samples and the latent vector obtained from the positive samples to calculate the Euclidean distance between them. Calculate the Euclidean distance between the latent vector and the latent vector obtained from the negative samples ; The optimization goal is to minimize and maximize . Adopt the following triplet loss function to optimize the autoencoder: while maximizing ; Optimize the autoencoder: ， Among them, m is a hyperparameter for controlling the distance. Adopt the following reconstruction loss function Train the autoencoder: ， Among them, represents the length of the vector encoding, and respectively represent the element value at the th position of the anchor sample and the element value at the th position of the vector encoding reconstructed by the decoder; Finally, by setting parameters to adjust the contribution balance of the two loss functions and a combined loss function is adopted during the training process to calculate the loss value, and the trainable parameters of the autoencoder are updated through backpropagation. The training is repeated iteratively until the model converges.

3. The method according to claim 2, wherein Step 1.3 includes: First, select architectures from the search space through a random sampling strategy, and add the vector encodings corresponding to the architectures to the architecture pool , where represents the th sample in the architecture pool; Second, randomly select a sample from the architecture pool as the anchor sample ; Then, through random sampling again, draw a sample from the architecture pool and calculate the performance similarity between the sample and the architecture corresponding to the anchor sample. For any two samples and , the performance similarity is calculated as follows: ， Among them, and respectively represent the true performance values of the architectures corresponding to the samples and of the samples corresponding to the architectures ; e represents the natural constant; if is greater than the threshold , it means that the performances of the two samples are similar, and the currently extracted sample is used as the positive sample of the anchor sample ; otherwise, it means that the performances of the two samples are not similar, and the currently extracted sample is used as the negative sample of the anchor sample ; subsequently, continue to randomly extract samples from the architecture pool until a positive sample and a negative sample are found for the current anchor sample to construct a triple sample ; repeat step 1.3 in a loop for a total of times to obtain triple samples.

4. The method according to claim 3, characterized in that, In step 2, the following binary cross-entropy loss function is used to train and optimize the model: ， Among them, indicates superior to probability; A pairwise ranking loss function is proposed as a penalty term for training : ， where exp is the natural exponential function, is a sign function, defined as: ， Finally, by setting parameters to control the penalty term on the influence of model training, a joint loss function is used to calculate the loss value during the training process.

5. The method according to claim 4, wherein In step 3, N neural network architectures are selected from the search space as the initial population through random sampling , where N represents the size of the population; subsequently, the individuals in the population are comprehensively trained to achieve a true evaluation; comprehensive training means that on the target data set, the network architecture is completely trained and the final performance is evaluated on the validation set.

6. The method according to claim 5, wherein Step 4 includes: Step 4.

1. First, each time two parental architectures are selected from the current population through the tournament selection strategy, crossover and mutation operations are performed on the two parental architectures to generate offspring, and Step 4.1 is repeated until candidate architectures are generated; where denotes the population at the -th generation when the evolution proceeds to the -th generation; Step 4.2, use the dominance classifier to predict the fitness score of the candidate architectures: First, initialize the score of each candidate architecture to 0, indicating that when the weight of the dominance classifier is , the predicted score of the th architecture is ; Subsequently, use the autoencoder to obtain the latent vector of each candidate architecture. The dominance classifier predicts the relative superiority relationship between each architecture and other candidate architectures based on the latent vector; for any two architectures and in the search space, if the probability that the output by the dominance classifier is better than is , then: ， Among them, the parameter is used to control the probability of the contribution to the score; subsequently, the score is taken as the first objective, and non-dominated sorting and crowding distance calculation are performed, and the top best-performing architectures are selected as the current offspring architectures , and the offspring architectures are further evaluated in a real sense through comprehensive training; Finally, merge the parent architecture and the child architecture , and use the non-dominated sorting genetic algorithm to perform environmental selection on the merged population, and select architectures that perform excellently in more than two evaluation metrics to update the population .

7. The method according to claim 6, wherein Step 5 specifically includes the following steps: Step 5.1: Population diversity evaluation. The new population obtained in step 4 is clustered by the method of hyperplane projection and the population diversity is evaluated by using the clustering quality index. First, the unit vector is defined in the normalized objective scale: ， Among them, is the number of optimization objectives, represents the objective value of an individual in the new population, represents the objective value of an individual in the new population on the th optimization objective, , represents the maximum value of an individual in the new population on the th objective; Construct a hyperplane : ， Among them, is the constant term; is and the dot product of, indicating the projection of in the direction of; Calculate the target value of an individual On the hyperplane Orthogonal projection : ， Among them, represents the square of the norm of the vector ; Project all individuals of the population onto a hyperplane to obtain a projected population , which consists of the orthogonal projections of all individuals . Cluster the projected population to evaluate the diversity of the population: Use the K-clustering method to cluster the projected population into subpopulations , where ; represents the th projected subpopulation obtained after clustering, and each projected subpopulation represents a group of similar individuals in the target space; Subsequently, use the clustering quality index Q as an indicator to evaluate the population diversity. ， Among them, represents the number of individuals in the k-th projected sub-population , represents the th orthogonal projection in the k-th projected sub-population represents the cluster center of the k-th projected sub-population , represents the Euclidean distance from the orthogonal projection to the cluster center ; Finally, set the diversity threshold , when , it indicates that the current population has excellent diversity, and step 5.2 is executed; otherwise, step 5.3 is executed; Step 5.2, Local Search Optimization: Randomly select half of the architectures from the new population for local optimization. For the th architecture to be optimized , set the reference point with the target vector and calculate the normalized weight vector ; secondly, use the encoder to extract the latent vector of the architecture ; subsequently, use the achievement scalarizing function as the objective function : ​ ， Among them, represents the latent vector of the corresponding architecture's target vector, is a positive number, indicating the summation of the values for all architectures to be optimized; Next, sequential quadratic programming is used on the latent space to perform gradient optimization on the objective function : ， Among them, the latent vector will be iteratively updated during the optimization process, and finally the optimized latent vector ; For the optimized latent vector , the decoder in Step 1 is used for reconstruction to obtain the decoded architecture . Through comprehensive training, a true evaluation is performed on , and the dominance relationship between and is compared. If dominates , then in the current population , is used to replace ; otherwise, the population remains unchanged. Finally, the non-dominated sorting and crowding distance are recalculated for the newly optimized and updated population; Step 5.3, activate the diversity enhancement module to enhance population diversity: Re - select individuals from the set of the merged parental architectures and the offspring architectures to reconstruct the population. First, project and cluster the merged architectures to obtain K sub - populations ; Second, perform non - dominated sorting within each sub - population. The j - th layer of non - dominated solutions in the k - th sub - population is . Use to record the multi - layer Pareto fronts of each sub - population. Subsequently, construct an empty set as the new population after diversity optimization. Traverse in the order of the front levels , select individuals to fill , until the number of individuals in reaches the population size N; Finally, use to replace the current population , and recalculate the non - dominated sorting and crowding distance of the new population.

8. The multi-objective hybrid evolutionary neural architecture search system assisted by a dominance classifier implemented by the method according to any one of claims 1 to 7, characterized in that It includes: Autoencoder based on contrastive learning: used to map the disordered search space to a low-dimensional coding space that is ordered according to performance advantages and disadvantages, and is also used to reconstruct the locally optimized coding vectors to obtain the decoded architectures. Dominance classifier based on comparison of architecture performance advantages and disadvantages: used to predict the superiority and inferiority relationship between architectures, calculate the fitness scores of architectures, and select excellent neural architectures. Hybrid evolutionary optimization module: used to evaluate the diversity of each generation of the population, perform local search optimization on the population with excellent diversity, and enhance the diversity of the population.

9. An electronic device, characterized in that, It includes a processor and a memory. The memory stores program codes. When the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, Store computer programs or instructions. When the computer programs or instructions run on the computer, execute the steps of the method according to any one of claims 1 to 7.

Images