A method for generating an adversarial network architecture based on multi-population co-evolution

Abstract

The application provides a kind of generation of adversarial network architecture search method based on multi-population coevolution, comprising: step 1, define the search space for searching generator and discriminator;Step 2, randomly generate the gene population of generator and discriminator;Step 3, activate all candidate operations, and initialize training;Step 4, search the architecture parameters and network weights of generative adversarial network;Step 5, optimize and update the gene population of generator and discriminator;Step 6, alternately optimize the architecture of generator and discriminator until the network performance can not be improved any more;Step 7, the searched generative adversarial network is used for the generation of unseen samples in zero-shot learning.The joint search of the method can fully utilize the interdependent and dynamic game relationship between generator and discriminator in GAN framework, which is conducive to searching better GAN architecture, and has higher efficiency and better search performance compared with traditional optimization methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep neural networks, and particularly relates to a generative adversarial network architecture search method based on multi-population cooperative evolution. BACKGROUND

[0002] In recent years, generative adversarial networks (GAN) have received extensive attention due to their outstanding performance in image generation, data augmentation, style transfer, and other fields. GAN consists of a generator and a discriminator, which achieve data distribution fitting and approximation through an adversarial training mechanism. The generator aims to generate realistic fake samples to deceive the discriminator, while the discriminator tries to distinguish real samples from fake samples. Both sides continuously optimize in a dynamic game, thereby improving network performance together.

[0003] However, the performance of GAN is highly dependent on the design of its network structure, and different generator and discriminator architectures have a significant impact on training stability and generation quality. Due to the lack of unified design criteria, traditional GAN architectures rely heavily on manual experience design, which is inefficient and difficult to adapt to different task scenarios. In order to solve this problem, in recent years, neural network architecture search (NAS) has been introduced into GAN architecture design, optimizing network structure through automated search strategies to improve network performance and generalization ability.

[0004] Although existing NAS-based GAN architecture search methods have alleviated the burden of manual design to some extent, there are still problems: first, most methods optimize the generator and discriminator architectures separately, failing to fully utilize the adversarial coupling relationship between the two, resulting in the inability of the searched structure to coordinate the overall adversarial performance; second, in the population evolution process, population diversity is difficult to maintain, and it is easy to fall into local optimum, affecting search efficiency and effectiveness; third, current architecture search methods rely on fixed training procedures for network structure evaluation, lacking flexible and dynamic training scheduling strategies, increasing search overhead and reducing search quality. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a generative adversarial network architecture search method based on multi-population cooperative evolution, which overcomes the shortcomings of the prior art.

[0006] Step 1, define the search space for searching the generator and discriminator, and use binary encoding;

[0007] Step 2, randomly generate a gene population for the generator and discriminator;

[0008] Step 3, activate all candidate operations, and perform initialization training;

[0009] Step 4, search for the architecture parameters and network weights of the generative adversarial network using a collaborative architecture search method;

[0010] Step 5, use a multi-population collaborative optimization method to optimize and update the gene populations of the generator and discriminator;

[0011] Step 6, loop steps 3-4, alternately optimize the architecture of the generator and discriminator until the network performance cannot be improved;

[0012] Step 7, use the searched generative adversarial network for the generation of unseen samples in zero-shot learning.

[0013] In step 1, the generator G is represented as a directed acyclic graph , and the discriminator D is represented as a directed acyclic graph , where and represent the node sets of the generator and the discriminator, respectively, and represent the connection operations of the generator and the discriminator, and S represents the channel attention module common to the generator and the discriminator.

[0014] In step 1, the node sets of the generator and the discriminator include input nodes, intermediate nodes, and output nodes.

[0015] In step 1, the connection operations of the generator and the discriminator and represent the operations connecting the nodes in the generator and the discriminator, and the specific operations will be searched from the candidate operation set E through model training, and the edge state is represented using one-hot encoding.

[0016] In step 1, the channel attention module S is designed to be placed before each intermediate node , and is used to process the output of the predecessor node of the i-th intermediate node , the input of the intermediate node , and the formula is:

[0017] ,

[0018] where represents the operation connecting node to node , represents the input of the j-th intermediate node , represents the concatenation operation.

[0019] The channel attention module S constructs an attention matrix by calculating the correlation weights between different node features to measure the importance distribution between channels, and introduces a learnable aggregation parameter to compress the feature dimension, thereby extracting a more discriminative feature representation. Finally, the compressed feature is used as the input of the node. The mathematical expression is:

[0020] ,

[0021] in, represents the input feature matrix, represents the real number space, c is the number of channels, f is the feature dimension, is the channel attention coefficient of the i-th channel, is the learnable aggregation weight of the i-th channel.

[0022] In step 2, randomly generate the gene groups of the generator and discriminator: the genes of the generator and discriminator are composed of a 25×5 matrix with values ​​0 and 1, where the 25 rows represent the 25 connecting edges of the generator or discriminator network, and the 5 columns represent the 5 candidate operations on the connecting edges. 0 indicates inactivation and 1 indicates activation. A random candidate operation is effective on each edge.

[0023] Step 4 includes:

[0024] Step 4.1, evaluate and select the optimal discriminator or generator gene, and fix the optimal discriminator or generator gene as the architecture parameter;

[0025] Step 4.2: Randomly select a gene A1 from the unfixed generator or discriminator gene group. Gene A1 represents the architecture parameters of the generator or discriminator that are not fixed during the current batch training.

[0026] Step 4.3: Train the network weights of the generator and discriminator using an adaptive training method based on Wasserstein distance;

[0027] Step 4.4: Repeat steps 4.2 to 4.3 until one round of data training is completed.

[0028] In step 4.1, the method for evaluating the optimal discriminator and generator is:

[0029] For the generator population , directly use the currently fixed discriminator for evaluation, the formula is:

[0030] ,

[0031] in represents the currently fixed discriminator, fitness of the i-th generator represented by the i-th generator gene, denotes the expected value under the probability distribution of noise; i takes values from 1 to n;

[0032] For the discriminator population, the Wasserstein distance is used as its performance evaluation value:

[0033]

[0034] wherein, fitness of the i-th discriminator represented by the i-th discriminator gene, denotes the expected value under the probability distribution of real data, denotes the current fixed generator.

[0035] In step 4.3, the Wasserstein distance is used as the basis for discriminator training, and the discriminator update is adaptively terminated by monitoring the Wasserstein distance of the discriminator to the real sample and the generated sample. The formula for calculating the Wasserstein distance is:

[0036]

[0037] The optimization objective of the generator architecture search is expressed as:

[0038]

[0039]

[0040]

[0041] wherein, denotes the optimal generator architecture parameter currently evaluated, architecture parameter of the i-th generator represented by the i-th generator gene, denotes the weight of the trained generator, denotes the weight of the trained discriminator, denotes the fixed discriminator architecture parameter, denotes the constraint condition, denotes the generator weight, denotes the discriminator weight; denotes the fitness function of the generator; The optimization objective of the discriminator architecture search is expressed as:

[0042] ​​​​​​​​​​​

[0043] ,

[0044] ,

[0045] ,

[0046] wherein, represents the optimal discriminator architecture parameter currently evaluated, represents the architecture parameter of the generator represented by the ith generator gene, represents the fixed generator architecture parameter, represents the fitness function of the discriminator.

[0047] Step 5 includes:

[0048] Step 5.1, using fitness to evaluate and sort the architecture and filter out low fitness genes;

[0049] Step 5.2, select the gene with the optimal fitness that is not assigned as the leader L of a sub-population;

[0050] Step 5.3, traverse the remaining candidate genes g in the set to consider genes that satisfy the similarity higher than the set value as being in the same sub-population as the leader L;

[0051] Step 5.4, loop steps 5.2-5.3 until all genes are assigned to sub-populations;

[0052] Step 5.5, select the top genes in the top sub-population as the parent;

[0053] Step 5.6, generate child individuals using crossover and mutation and random injection operations;

[0054] In step 5.1, use a dynamic threshold to filter genes, and calculate the threshold by the following formula:

[0055] ,

[0056] wherein, is the fitness function of the ith gene, , is the set fitness threshold parameter;

[0057] In step 5.3, the similarity calculation formula is:

[0058] ​ ,

[0059] in, is the element in row i and column j of the gene matrix of leader L, is the element in row i and column j in the gene matrix of the candidate gene g.

[0060] Beneficial effects: The present invention provides a method for searching the architecture of a generative adversarial network based on multi-population co-evolution, which has many beneficial effects. First, by jointly searching the network structure of the generator and the discriminator, the dynamic dependency between the two in the adversarial training process is fully utilized, which helps to improve the collaborative performance and generation quality of the overall network. Secondly, the introduction of a multi-population co-evolution mechanism based on gene similarity division effectively enhances population diversity, avoids falling into local optimality, and improves search efficiency and convergence stability. At the same time, the channel attention mechanism is integrated into the network structure, so that the model can automatically focus on more discriminative feature channels, thereby enhancing the feature expression ability in the generation and discrimination process. In addition, the present invention adopts an adaptive training strategy based on the Wasserstein distance to dynamically adjust the update rhythm of the generator and the discriminator, effectively alleviating the problems of training instability and mode collapse, and improving the robustness of adversarial training. Overall, the method has a flexible structural design and a wide range of applications. It can significantly improve the architectural optimization quality of the generative adversarial network and has good application prospects and practical value in multiple tasks such as image generation and feature synthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is the overall framework diagram of the present invention.

[0062] Figure 2 It is the search space graph in the present invention.

[0063] Figure 3 This is a flow chart of the multi-population evolution strategy in the present invention.

[0064] Figure 4 This is a schematic diagram of the experimental analysis results when the similarity threshold parameter is set to 0.6.

[0065] Figure 5 This is a schematic diagram of the experimental analysis results when the similarity threshold parameter is set to 0.8.

[0066] Figure 6 This is a schematic diagram of the experimental analysis results when the similarity threshold parameter is set to 0.9.

[0067] Figure 7 This is a comparison chart of the experimental results of this discovery and other methods. DETAILED DESCRIPTION

[0068] The above and / or other aspects of the present application will become more apparent by describing in detail the preferred embodiments thereof with reference to the attached drawings, in which:

[0069] As shown in Figure 1 , the embodiment of the present application provides a method for generating an adversarial network architecture based on multi-population co-evolution, which comprises the following steps:

[0070] Step 1, as shown in Figure 2 , define the search space for searching the generator and discriminator, and use binary encoding. The generator and discriminator are respectively represented as directed acyclic graphs and , and the generator and discriminator in the present application use the same search space, as shown in Figure 1 . Among them, and represent the node set of the generator and the discriminator respectively, and represent the connection operation of the generator and the discriminator respectively, and S represents the channel attention module common to the generator and the discriminator.

[0071] Step 1.1, as shown in Figure 2 , the node set of the generator and the discriminator includes: input nodes, intermediate nodes and output nodes. In this example, the generator network contains three input nodes: noise vector z, semantic vector , and splicing vector ; the discriminator network also contains three input nodes: visual feature vector x, semantic vector , and splicing vector ; the generator and the discriminator each contain 4 intermediate nodes and one output node.

[0072] Step 1.2, the connection operation between the nodes of the generator and the discriminator will be searched from the candidate set, and in this example, the candidate operation set E includes:

[0073] ,

[0074] Among them, FC represents a fully connected layer, ReLU and LeakyReLU are activation functions, Dropout is a regularization technique, None represents that there is no connection between nodes, and one-hot encoding is used to represent the state of the edge.

[0075] Step 1.3, the channel attention module S is designed to be placed in front of each intermediate node, for processing the output of the predecessor node of the intermediate node , the intermediate node The input formula of the generator is:

[0076] ,

[0077] wherein, represents a channel attention module, represents an operation of connecting a node to a node , represents an input of a node , represents a splicing operation; the channel attention module S measures the importance distribution between channels by constructing an attention matrix through calculating the correlation weight between different node features, and introduces a learnable aggregation parameter to compress the feature dimension on this basis, so as to extract a more discriminative feature representation, and finally, the compressed feature is taken as the input of the node, and the mathematical expression is:

[0078] ,

[0079] wherein, represents an input feature matrix, represents a real space, c is the number of channels, and f is the feature dimension, is a channel attention coefficient of the i-th channel, is a learnable aggregation weight of the i-th channel.

[0080] Step 2, randomly generate a gene group of the generator and the discriminator. The gene group of the generator and the discriminator is randomly generated. In this example, the gene of the generator and the discriminator is composed of a 25*5 matrix with numerical values of 0 and 1, wherein 25 rows represent 25 connection edges of the generator or discriminator network, and 5 columns represent 5 candidate operations on the connection edge. 0 represents inactivation, and 1 represents activation. Each edge randomly activates a candidate operation to generate a random gene, and at the same time, the gene matrix is standardized to avoid isolated nodes, dead-end nodes or input loss. Finally, 50 different genes are randomly generated and stored.

[0081] Step 3, activate all candidate operations and perform initialization training. Specifically, the gene matrix with all numerical values of 1 is used to set the architecture of the generator and the discriminator and perform training, so as to ensure the fairness at the beginning of the search. In this example, the initialization training is 5 cycles.

[0082] Step 4, as shown in Figure 1 , the architecture parameters and network weights of the generative adversarial network are searched cooperatively, including:

[0083] Step 4.1, evaluate and select the optimal discriminator or generator, and fix the architecture parameters, including the following steps:

[0084] Step 4.1.1, for the generator population , directly use the currently fixed discriminator To evaluate, the formula is:

[0085] ,

[0086] in, Indicates the generator represented by the i-th generator gene The fitness of Represents the probability distribution of noise Expected value under

[0087] Step 4.1.2, for the discriminator population , using Wasserstein distance as its performance evaluation value:

[0088] ,

[0089] in, Indicates the discriminator represented by the i-th discriminator gene The fitness of Represents the probability distribution of real data The expected value under Represents the currently fixed generator;

[0090] Step 4.2: Randomly select a gene from the group of unfixed generator or discriminator genes. This gene represents the architecture parameters of the generator or discriminator that have never been fixed during the training of this batch.

[0091] Step 4.3: Train the network weights of the generator and discriminator using an adaptive training method based on Wasserstein distance, including:

[0092] Step 4.3.1, optimize the discriminator architecture. The optimization objective of the discriminator architecture search is expressed as:

[0093] ,

[0094] ,

[0095] ,

[0096] in, represents the optimal discriminator architecture parameters currently evaluated, Indicates the generator represented by the i-th generator gene The architectural parameters of represents the discriminator weight after training, represents the generator weight after training, denote the fixed generator architecture parameters, s.t. denotes the constraint condition, denote the generator weights denote the discriminator weights

[0097] Step 4.3.2, using the Wasserstein distance as the basis for discriminator training, by monitoring the Wasserstein distance of the discriminator to the real sample and the generated sample, the discriminator update is adaptively terminated, the formula of the Wasserstein distance is as follows:

[0098] ,

[0099] Step 4.3.3, optimizing the generator architecture, the optimization objective of the generator architecture search is represented as:

[0100] ,

[0101] ,

[0102] ,

[0103] wherein, denote the optimal generator architecture parameters currently evaluated, denote the generator architecture parameters represented by the i-th generator gene, denote the fixed discriminator architecture parameters

[0104] Step 4.4, steps 3.2 and 3.3 are executed in a loop until a round of data is trained;

[0105] Step 5, as shown in Figure 3 , a multi-population collaborative optimization method is used to optimize and update the gene population of the generator and the discriminator, including:

[0106] Step 5.1, using fitness to evaluate and sort the architecture and filtering out low fitness genes, setting a dynamic threshold to filter the genes, the threshold is calculated by the following formula:

[0107] ,

[0108] wherein, is the fitness function of the i-th gene, , is the set threshold, in this example, is taken as 0.5;

[0109] Step 5.2, selecting the optimal individual as the leader of a sub-population ​

[0110] Step 5.3, traverse the remaining genes in the set to meet the similarity Higher than the set value The gene g is considered to be in the same subpopulation as the leader L, and the similarity is calculated as follows:

[0111] ,

[0112] in, is the element in row i and column j of the gene matrix of leader L, is the element in row i and column j of the gene matrix of the candidate gene g. In this example, the similarity threshold parameter The value of is obtained through experimental analysis, such as Figure 4 As shown, The value is taken as 0.8;

[0113] Step 5.4, repeat steps 5.2 and 5.3 until all genes are assigned to subpopulations;

[0114] Step 5.5, select the top ranking Ranked top in the subpopulation The gene of the parent generation, the number of child populations and subpopulation capacity The parameter values ​​are obtained through experimental analysis. In this example, the maximum total number of stored genotypes is K=100, the maximum storage capacity of selected genes is K / 2=50, and the subpopulation capacity is is set to:

[0115] ,

[0116] in, Indicates rounding down, such as Figure 4 、 Figure 5 、 Figure 6 As shown, the horizontal axis in the figure is the number of subpopulations The vertical axis is the accuracy of the searched model verified in the generative zero-shot learning task. This embodiment evaluates different similarity threshold parameters. ( Figure 4 The median value is 0.6, Figure 5 The median value is 0.8, Figure 6 The median value is 0.9) and the number of subpopulations The performance of the searched model is shown to decrease as the similarity threshold increases, which is because a higher threshold leads to an increase in genetic similarity among individuals within a subpopulation, thus enabling a smaller number of genotypes to represent local diversity without reducing performance. When the similarity threshold is set to 90%, the performance decreases significantly, which is likely due to a reduction in diversity among subpopulations, because a higher threshold increases redundancy within a population. This loss of diversity ultimately weakens the effectiveness of the multi-population search and reduces the quality of the generated architecture. Therefore, the similarity threshold is finally set to 80% in the experiments. , ;

[0117] Step 5.6, generate offspring individuals using crossover and mutation and random injection operations;

[0118] Step 6, loop steps 3 and 4 to alternately optimize the architectures of the generator and discriminator until the network performance can no longer be improved;

[0119] Step 7, use the searched generative adversarial network for the generation of unseen samples in zero-shot learning, use the searched optimal generator and discriminator architectures to build a generative adversarial network model, and in the training process, the model generates synthetic features for seen classes using their semantic vectors. In the inference stage, the generator synthesizes features according to the semantic vectors of unseen classes. The synthesized unseen class features are combined with seen class data to train a classifier, and then the classifier is evaluated using a test set that includes both seen and unseen data.

[0120] As shown in Figure 7 To verify the performance, the present application performs experiments on the CUB public dataset, which contains 11,788 images covering 200 bird species, and each image is associated with an attribute vector. The figure shows that the present method performs well in training stability, while other architecture search methods frequently exhibit oscillation phenomena during training. This is due to its adaptive adversarial training component, which enhances the robustness of convergence. Although the present method converges more slowly in early training, it ultimately achieves higher performance, which is 4.95% and 3.59% higher than the baseline model and other architecture search methods, respectively. These results confirm the effectiveness of co-evolution in promoting better synergy between the generator and discriminator.

[0121] The application provides a multi-population co-evolution-based generative adversarial network architecture search method. There are many methods and approaches to realize the technical solution, and the above description is only the preferred embodiment of the application. It should be pointed out that, for ordinary skilled persons in the technical field, some improvements and refinements can be made without departing from the principle of the application, and these improvements and refinements should also be regarded as the protection scope of the application. The components not explicitly described in the embodiment can be realized by using the existing technology.

Claims

1. A generative adversarial network architecture search method based on multi-population co-evolution, characterized by: The following steps are involved: Step 1: Define the search space for the generator and discriminator and use binary encoding; Step 2, randomly generate the gene groups of generator and discriminator; Step 3: activate all candidate operations and perform initial training; Step 4: Use a collaborative architecture search method to search for the architecture parameters and network weights of the generative adversarial network; Step 5, using a multi-population collaborative optimization method to optimize and update the gene groups of the generator and the discriminator; Step 6: Repeat steps 3 and 4, alternately optimizing the architectures of the generator and discriminator until the network performance cannot be improved any further. Step 7: Use the searched generative adversarial network to generate unseen samples in zero-shot learning; In step 1, the generator G is represented as a directed acyclic graph , the discriminator D is represented as a directed acyclic graph ,in, and The sets of nodes representing the generator and discriminator respectively, and They represent the connection operations of the generator and the discriminator respectively, and S represents the channel attention module common to the generator and the discriminator; In step 1, the channel attention module S is designed to be placed before each intermediate node to process the i-th intermediate node Predecessor node Output, intermediate node Input The formula is: ， in, Represents a connection node To Node Operation, Represents the jth intermediate node Input, Represents a splicing operation; The channel attention module S constructs an attention matrix by calculating the correlation weights between different node features to measure the importance distribution between channels, and introduces a learnable aggregation parameter to compress the feature dimension, thereby extracting a more discriminative feature representation. Finally, the compressed feature is used as the input of the node. The mathematical expression is: ， in, represents the input feature matrix, represents the real number space, c is the number of channels, f is the feature dimension, is the channel attention coefficient of the i-th channel, is the aggregate weight of the learnable i-th channel; Step 4 includes: Step 4.1, evaluate and select the optimal discriminator or generator gene, and fix the optimal discriminator or generator gene as the architecture parameter; Step 4.2: Randomly select a gene A1 from the unfixed generator or discriminator gene group. Gene A1 represents the architecture parameters of the generator or discriminator that are not fixed during the current batch training. Step 4.3: Train the network weights of the generator and discriminator using an adaptive training method based on Wasserstein distance; Step 4.4: Repeat steps 4.2 to 4.3 until one round of training is completed. In step 4.1, the method for evaluating the optimal discriminator and generator is: For the generator population , directly use the currently fixed discriminator for evaluation, the formula is: ， in represents the currently fixed discriminator, Indicates the generator represented by the i-th generator gene The fitness of Represents the probability distribution of noise The expected value under the condition of ; i ranges from 1 to n; For the discriminator population , using Wasserstein distance as its performance evaluation value: ， in, Indicates the discriminator represented by the i-th discriminator gene The fitness of Represents the probability distribution of real data The expected value under Represents the currently fixed generator; Step 5 includes: Step 5.1, use fitness to evaluate and rank the architectures and filter out low-fitness genes; Step 5.2, select the unassigned gene with the best fitness as the leader L of a subpopulation; Step 5.3, traverse the remaining candidate genes g in the set to meet the similarity Higher than the set value The genes of are considered to be in the same subpopulation as the leader L; Step 5.4: Repeat steps 5.2 to 5.3 until all genes are assigned to subpopulations; Step 5.5, select the top ranking Ranked top in the subpopulation The gene of is the parent; Step 5.6, use crossover, mutation and random injection operations to generate offspring individuals; In step 5.1, use dynamic threshold Filter the genes and calculate the threshold using the following formula : ， in, is the fitness function of the i-th gene, , is the set fitness threshold parameter; In step 5.3, the similarity calculation formula is: ， in, is the element in row i and column j of the gene matrix of leader L, is the element in row i and column j in the gene matrix of the candidate gene g.

2. The method according to claim 1, characterized in that In step 1, the node sets of the generator and discriminator include: input nodes, intermediate nodes, and output nodes.

3. The method according to claim 2, characterized in that In step 1, the connection operation between the generator and the discriminator and Represents the operation of connecting the nodes in the generator and the discriminator. The specific operation used will be searched from the candidate operation set E through model training, and the state of the edge is represented by one-hot encoding.

4. The method according to claim 3, characterized in that In step 2, randomly generate the gene groups of the generator and discriminator: the genes of the generator and discriminator are composed of a 25×5 matrix with values ​​0 and 1, where the 25 rows represent the 25 connecting edges of the generator or discriminator network, and the 5 columns represent the 5 candidate operations on the connecting edges. 0 indicates inactivation and 1 indicates activation. A random candidate operation is effective on each edge.

5. The method according to claim 4, characterized in that In step 4.3, the Wasserstein distance is used as the basis for discriminator training. By monitoring the Wasserstein distance between the discriminator and the generated samples, the discriminator update is adaptively terminated. The Wasserstein distance calculation formula is: ， The optimization goal of the generator architecture search is expressed as: ， ， ， in, Represents the optimal generator architecture parameters currently evaluated, Indicates the generator represented by the i-th generator gene The architectural parameters of represents the generator weight after training, represents the discriminator weight after training, represents the fixed discriminator architecture parameters, represents the constraints, represents the generator weight, represents the discriminator weight; represents the fitness function of the generator; The optimization objective of the discriminator architecture search is stated as: ， ， ， in, represents the optimal discriminator architecture parameters currently evaluated, Indicates the generator represented by the i-th generator gene The architectural parameters of represents the fixed generator architecture parameters, represents the fitness function of the discriminator.

Images