A method and system for searching a generative adversarial network architecture based on architecture distillation technology

Abstract

The present invention provides a method and system for searching a generative adversarial network architecture based on architecture distillation technology. The method defines a search space, constructs a supernet that integrates all candidate operations, performs preheating training on the supernet, and after the initial preheating training, evaluates the performance of the subnet, and screens out and discards operations that are detrimental to network performance. During the training of the supernet, the network weights are optimized by the architecture knowledge distillation method, and a matching optimization network is designed to effectively select a teacher network so that the teacher network and the student subnet are optimally matched. The knowledge of the teacher network is transferred to other subnets through the architecture distillation technology to guide the training of the supernet. A multi-objective optimization strategy is used to select the best performing network architecture in the supernet. The present invention improves the training effect of the subnet by transferring knowledge between architectures. Compared with traditional search methods, it has higher efficiency and better performance, and can find the optimal neural network architecture suitable for the task more quickly.

Description

Technical Field

[0001] The present invention relates to the technical field of deep neural networks, and particularly to a method and system for searching for a generative adversarial network architecture based on architecture distillation technology. Background Art

[0002] Since the generative adversarial network was proposed, it has occupied an important position in the field of deep learning, especially showing powerful generative capabilities in computer vision tasks. The generative adversarial network consists of a generator and a discriminator. Through the dynamic game between the two, the generator can learn the real data distribution, thereby generating high-quality synthetic data. Due to its powerful data generation ability, the generative adversarial network has been widely applied to multiple fields such as image generation, image restoration, super-resolution reconstruction, image translation, image enhancement, and text-to-image generation, greatly promoting the development of these fields. However, the training process of the generative adversarial network is often accompanied by instability problems such as mode collapse and gradient disappearance, which limit the further improvement of its performance and wide application.

[0003] To overcome the instability of generative adversarial network training, researchers have actively explored various strategies, including designing novel network architectures, optimizing loss functions, and introducing regularization methods, etc. Although these manual intervention measures have alleviated some challenges, designing generative adversarial network architectures manually is still time-consuming, sub-optimal, and difficult to generalize. Among them, neural architecture search has been widely applied to the automated design of deep learning models in recent years and has achieved success in tasks such as classification and detection. Neural architecture search automatically discovers the optimal neural network architecture through search algorithms to improve model performance. However, applying neural architecture search methods in generative adversarial network tasks still faces many challenges. Due to the extensive evaluation of candidate architectures, traditional neural architecture search is computationally expensive, limiting its scalability.

[0004] To solve the computational problems faced by neural architecture search in generative adversarial networks, researchers have proposed a one-shot neural architecture search method, using a weight-sharing supernet strategy, enabling multiple sub-networks to share the same weights, effectively reducing the training cost. However, in generative adversarial network tasks, due to insufficient training of sub-networks, the one-shot weight model leads to inaccurate evaluation results, thereby weakening the effectiveness and reliability of the search process. Summary of the Invention

[0005] Object of the Invention: The technical problem to be solved by the present invention is to provide a method and system for searching for a generative adversarial network architecture based on architecture distillation technology in view of the deficiencies of the prior art.

[0006] The method includes the following steps:

[0007] Step 1: Design the search space for generative adversarial network architecture search, define the candidate operation set, and determine the scope of architecture search. The candidate operations in the candidate operation set include two types: ordinary candidate operations and upsampling candidate operations.

[0008] Step 2: According to the search space designed in Step 1, construct a generative adversarial network supernet that includes all candidate architectures. The supernet covers all possible candidate architectures and supports subsequent architecture optimization.

[0009] Step 3: Conduct N w rounds of warm-up training on the supernet, and optimize the weight parameters of the supernet using the generative adversarial loss.

[0010] Step 4: Evaluate the supernet after warm-up training, eliminate low-performance candidate operations, and optimize the search space based on statistical analysis to improve the search efficiency and the performance of the final architecture.

[0011] Step 5: Initialize a subset of subnets P and a matching optimization network where the subset of subnets P is used to maintain a dynamic list of the historical optimal subnets, and the matching optimization network is used to calculate the matching degree between the optimal subnet and the sampled subnet.

[0012] Step 6: Use the matching optimization network to select the historical architecture with the highest matching degree from the subset of subnets P as the teacher model for the current sampled subnet.

[0013] Step 7: Calculate the architecture distillation loss and the generator loss to update the weights of the sampled subnet, and learn the optimization information from the teacher network through the architecture knowledge distillation technique to accelerate the training process.

[0014] Step 8: Through the backpropagation algorithm, use the updated generator loss as feedback to update the weights of the matching optimization network.

[0015] Step 9: Determine whether to update the subset of subnets according to the gap between the evaluation metrics and the historical optimal values. Use the multi-objective optimization algorithm to perform non-dominated sorting on the architectures in the subset of subnets, retain the Pareto front solutions, and remove the worst architecture when the storage capacity is exceeded.

[0016] Step 10: Alternately execute Steps 6 to 9 until the preset maximum number of loops is reached to obtain the trained supernet.

[0017] Step 11: Select the subnet with the optimal performance in the subset of subnets as the final network architecture to ensure that the overall performance of the generative adversarial network reaches the optimal. The generator network architecture searched from the supernet will be applied to image generation, and the input random noise will be converted into a color picture through the generator.

[0018] In step 1, the search space includes N units connected and skip-connected to each other. Each unit is regarded as a directed acyclic graph composed of Q nodes. The directed acyclic graph contains U upsampling candidate operation nodes and R ordinary candidate operation nodes, satisfying , and N, Q, U, and R are all natural numbers.

[0019] In step 2, the generative adversarial network supernet has edges, among which edges are skip-connection edges, edges are the edges where the upsampling candidate operations are located, edges are the edges where the ordinary candidate operations are located; the upsampling candidate operations have T operation choices, and the skip connections and ordinary candidate operations have Y operation choices;

[0020] The supernet includes S subnets, where S is a natural number, , and a subnet represents a network where there is only one operation on all connected edges.

[0021] Step 3 includes:

[0022] Step 3.1, initialize the training parameters, and the training parameters include the learning rate of the generator, the learning rate of the discriminator, the network parameter weights

[0023] of the generator,

[0024] the network parameter weights

[0025] of the discriminator,

[0026] optimizer parameters, and a fixed noise vector; Step 3.2, perform adversarial training on the generator and the discriminator, and use the discriminator to calculate the loss between the generated image and the real image; ;

[0027] Step 3.2.2, randomly sample the discrimination result of the fake image from the standard normal distribution. The higher the result, the higher the possibility that the discriminator thinks the input content is true. Calculate the hinge loss through the output of the discriminator, and update the hinge loss to the discriminator through backpropagation. The loss function of the discriminator is:

[0028] ,

[0029] Among them, represents the subnet The weight inherited from the supernet, E represents the expectation, indicates that the real image x follows the real data distribution , indicates that the noise vector z follows the noise distribution , represents the activation function, D represents the discriminator;

[0030] Step 3.2.4, calculate the loss function of the generator :

[0031] .

[0032] Step 3.3 includes:

[0033] Step 3.3.1, fix the network parameter weights of the generator , keep the generator unchanged, according to the loss function of the discriminator , use the backpropagation method to calculate the gradient , and through the gradient descent method, according to the discriminator learning rate update the network parameter weights of the discriminator , the calculation formula is:

[0034] ;

[0035] Step 3.3.2, fix the network parameter weights of the discriminator , keep the discriminator unchanged, according to the loss function of the generator use the backpropagation method to calculate the gradient , and use the gradient descent method, according to the learning rate of the generator update the network parameter weights of the generator , the calculation formula is:

[0036] .

[0037] Step 4 includes:

[0038] Step 4.1, randomly sample a specified number (usually 42) of subnets from the supernet, and use the Inception Score (IS) to evaluate the performance of the subnets;

[0039] Step 4.2, sort the subnets according to the evaluation results, and count the operation frequencies of each candidate operation node in the top Y 1 (usually taking a value of 50%) of the subnets;

[0040] Step 4.3, discard the 1 operation with the lowest frequency among the upsampling candidate operations and the 3 operations with the lowest frequency among the normal candidate operations to improve the search efficiency.

[0041] In the updated search space, the upsampling candidate operations have T - 1 operation choices, and the skip connections and normal candidate operations have Y - 3 operation choices; the updated supernet includes S u subnets, where S u is a natural number, .

[0042] Step 5 includes:

[0043] Step 5.1, construct a subnet set , which is used to maintain a dynamic list of the historical optimal subnets, where represents the i-th optimal subnet during the supernet training process, is the weights inherited from the supernet. I is the maximum capacity set for the subnet set during initialization, and the perceptual similarity index IS and the Fréchet Inception Distance FID are set as the metrics for evaluating subnets;

[0044] Step 5.2, construct a matching optimization network , which is used to calculate the matching degree between the optimal subnet and the sampled subnet. The matching optimization network includes a single-layer fully connected layer. The input nodes are 32×32×3 (consistent with the image dimension of the training set CIFAR-10), and the output node is one. The output value represents the matching degree between the optimal subnet and the sampled subnet. The weight parameters of the matching optimization network are .

[0045] Step 6 includes:

[0046] Step 6.1, sample a subnet from the supernet , and calculate the output of the subnet through forward propagation;

[0047] Step 6.2, use the matching optimization network to calculate the gap between the output of the subnet and the output of the optimal subnet in the subnet set P, and generate the matching degree;

[0048] Step 6.3, select the historical optimal subnet with the highest matching degree as the teacher network and return the matching degree . The selection formula for the teacher network is:

[0049] ,

[0050] Among them, is the output of the matching optimization network indicating the optimal subnet and the subnet the matching degree between them, indicating the weights inherited by the optimal subnet from the supernet, indicating the subnet the weights inherited from the supernet, indicating the subnet the output image, indicating the output image of the optimal subnet The input of the matching optimization network is the difference between the optimal subnet and the subnet output. This difference reflects the complementarity of the two subnets;

[0051] Step 7 includes:

[0052] Step 7.1, calculate the architecture distillation loss between the teacher network and the subnet :

[0053] ,

[0054] Among them, indicates the weights inherited by the teacher network from the supernet, indicates the teacher network the output image, and the parameter is used to measure the difference between the outputs of the teacher generator and the sampled subnet generator; Through the architecture knowledge distillation technique, the knowledge of the teacher network is transferred to the subnet to accelerate the training process of the supernet;

[0055] Step 7.2, calculate the generator loss of the sampled subnet :

[0056] ;

[0057] Step 7.3, the weight gradient update of the subnet is jointly supervised by the weighted average of the generator loss and the architecture distillation loss . Among them, the weight of the architecture distillation loss is dynamically adjusted by the matching degree output by the matching optimization network to reflect the contribution degree of different teacher networks to the training of the sampled subnet. The weight update formula of the subnet is:

[0058] ,

[0059] Among them, represents the iteration index, represents the weights of the subnet before weight update ; represents the weights of the subnet after weight update ; represents the weights of the teacher network corresponding to the subnet ; represents the subnet gradient calculated at , which is used to guide the update direction of the subnet weights, represents the learning rate of the optimization objective function, which is used to control the amplitude of each update of the subnet weights;

[0060] In step 8, calculate the generator loss using the sampled subnet weights updated according to the architecture distillation operation , and use the generator loss as feedback to update the matching optimization network through learning by the backpropagation algorithm , and the weight update formula of the matching optimization network is as follows:

[0061] ,,

[0062] Among them, represents the gradient of the generator loss with respect to the weight parameters of the matching optimization network, represents the gradient of calculated at the subnet weights with respect to ;

[0063] In step 9, sample a group of subnets from the supernet , and use the perceptual similarity index IS and the Frechet distance FID as metrics to evaluate the performance of the subnets. If the performance of the sampled subnets is better than the worst network architecture in the subnet set , then update the subnet set, and the comparison and update formula is:

[0064] ,

[0065] and ,

[0066] Among them, represents the function for calculating the perceptual similarity index, represents the function for calculating the Frechet distance, represents the update and selection , that is, select the ones that meet the conditions As new ;

[0067] In step 10, the architecture distillation loss and the generator loss are jointly used to optimize the hypernetwork parameters, making the hypernetwork gradually approach the optimal performance;

[0068] In step 11, the IS and FID metrics are combined to quantify the quality and diversity of the generated images. A multi-objective optimization strategy is adopted to balance the image quality and the similarity of the distribution between the generated images and the real images. The subnet with the optimal performance in the optimal subset of subnets is selected as the final optimal network architecture; the generator network architecture searched from the hypernetwork will be applied to the field of image generation, and the input random noise will be converted into a color picture through the generator.

[0069] The present invention also provides a generative adversarial network architecture search system based on the architecture distillation technology, including:

[0070] A construction module for constructing a generative adversarial network hypernetwork, an optimal subset of subnets, and a matching optimization network;

[0071] An architecture distillation module for transmitting the knowledge of the optimal subnets to the sampled subnets to improve the training efficiency of the hypernetwork;

[0072] A training module for training the network weights of the generator hypernetwork using the architecture distillation loss and the adversarial loss to ensure that the performance of the final network architecture reaches the optimal;

[0073] A search module for searching for the subnet architecture with the optimal performance in the hypernetwork, performing multi-objective search in the optimal subset of subnets, and selecting the subnet with the optimal performance as the final network architecture.

[0074] The present invention innovatively introduces the architecture distillation technology. By identifying and screening high-performance architectures as the teacher network, and using the matching optimization network to match the subnets with the teacher network, it is ensured that the subnets can inherit the essence of the teacher network, so as to efficiently guide the optimization of the hypernetwork to accelerate knowledge transfer, and significantly improve the stability of the generative adversarial network training and the quality of the generated images. In the hypernetwork search stage, the non-dominated sorting genetic algorithm II is adopted in this paper, combined with the Inception Score (IS) and the Fréchet Inception Distance (FID) as the performance evaluation metrics to ensure the efficiency of the searched subnet architecture, and realize the dual optimization of the quality of the generated images and the similarity of the real data distribution.

[0075] The beneficial effects of the present invention include: The method of the present invention proposes a generative adversarial network architecture search algorithm based on architecture distillation technology for the neural network architecture search used in generative adversarial networks. By introducing architecture distillation technology into the training process of the supernet of the generative adversarial network, this algorithm effectively improves the training effect of the subnet by mimicking the behavior of a high-performance teacher network, enables efficient knowledge transfer without pre-training the network, significantly enhances the stability and efficiency of the training process, and provides a new idea for the automated optimization of generative adversarial networks. During the search process, by combining IS and FID and using the NSGA-II algorithm for multi-objective optimization, the balance between image quality and the similarity of the distribution between the generated images and the real images is achieved, providing strong support for the search of efficient GANs architectures. This architecture search algorithm for generative adversarial networks based on architecture distillation brings new ideas and breakthroughs to the field of architecture search and has sufficient practical value and application scenarios. Description of the Drawings

[0076] Figure 1 It is the overall framework diagram of the search space in the present invention.

[0077] Figure 2 It is the schematic diagram of the training of the generator supernet in the present invention.

[0078] Figure 3 It is the trend diagram of the IS score for architecture optimization in the present invention.

[0079] Figure 4 It is the trend diagram of the FID score for architecture optimization in the present invention. Detailed Embodiments

[0080] The following further specific descriptions of the present invention are made in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0081] As Figure 1 shown, this embodiment provides a generative adversarial network architecture search method and system based on architecture distillation technology, including the following steps:

[0082] Step 1, first design the search space for the architecture search of the generative adversarial network generator as shown in Figure 1 shown. The search space consists of 3 units connected to each other and with skip connections. Inside the unit, it can be regarded as a directed acyclic graph with 5 nodes connected to each other. The blue arrow represents the upsampling operation, the purple arrow represents the normal operation. The upsampling candidate operations are set as {transpose convolution upsampling, nearest neighbor interpolation upsampling, bilinear interpolation upsampling}, and the normal candidate operations are set as {1×1 convolution, 3×3 convolution, 5×5 convolution, 3×3 dilated convolution, 5×5 dilated convolution, empty operation, skip connection}.

[0083] Step 2, according toFigure 1 For the search space shown, construct a generative adversarial network supernet that includes all candidate architectures, and randomly initialize the supernet. Randomly generate an input noise with 128 channels to input into the supernet to generate images. The input direction of the supernet feature maps is unidirectional from front to back;

[0084] The supernet is the general term for all candidate architectures in the search space. The subnet represents a specific architecture selected in the given search space. Each subnet is a part of the supernet. Traditionally, input information is usually processed through a single operation (such as convolution) to generate new feature maps. However, in the supernet, the input information is processed through multiple operations, such as dynamic separable convolution, dilated convolution, and skip connections. The feature maps generated by these operations are fused by element-wise addition to finally obtain a new mixed feature map. During the training process, through learning and optimization, the supernet can automatically select and adjust the weights of these candidate operations, thereby finally determining the optimal operation combination and network structure.

[0085] In the present invention, there are T candidate operations on the upsampling side of the supernet, including nearest neighbor interpolation upsampling, bilinear interpolation upsampling, and transposed convolution upsampling. There are Y candidate operations in the ordinary candidate operations and skip connections, including 1×1 convolution, 3×3 convolution, 5×5 convolution, 3×3 dilated convolution, 5×5 dilated convolution, skip connection, and no operation.

[0086] Step 3, perform N w rounds of warm-up training on the supernet, and optimize the weight parameters of the supernet using the generative adversarial loss, specifically including the following steps:

[0087] Step 3.1, initialize the training parameters, and the training parameters include the learning rate of the generator , the learning rate of the discriminator , the network parameter weights of the generator , the network parameter weights of the discriminator , optimizer parameters, and a fixed noise vector;

[0088] Step 3.2, perform adversarial training on the generator and the discriminator. During the training process, the generator and the discriminator are alternately optimized, and the discriminator is used to calculate the loss between the generated image and the real image, specifically including the following steps:

[0089] Step 3.2.1, fairly sample a subnet from the supernet ;

[0090] Step 3.2.2: Randomly sample the discrimination results of the fake images from the standard normal distribution. The higher the result, the higher the probability that the discriminator believes the input content is real. Calculate the hinge loss based on the output of the discriminator, and update the discriminator through backpropagation using the hinge loss. The loss function of the discriminator is:

[0091] ,

[0092] where represents the weights inherited by the subnet from the supernet, E represents the expectation, represents that the real image x follows the real data distribution , represents that the noise vector z follows the noise distribution , represents the activation function, and D represents the discriminator;

[0093] Step 3.2.4: Calculate the loss function of the generator :

[0094] .

[0095] Step 3.3: Through the backpropagation algorithm, update the weights of the discriminator and the generator based on the losses obtained in Step 3.2 to ensure the effective convergence of the model. Specifically, it includes the following steps:

[0096] Step 3.3.1: Fix the network parameter weights of the generator , keep the generator unchanged, and according to the loss function of the discriminator , use the backpropagation method to calculate the gradient , and through the gradient descent method, update the network parameter weights of the discriminator according to the discriminator learning rate , and the calculation formula is:

[0097] ;

[0098] Step 3.3.2: Fix the network parameter weights of the discriminator , keep the discriminator unchanged, and according to the loss function of the generator use the backpropagation method to calculate the gradient , and use the gradient descent method to update the network parameter weights of the generator according to the generator learning rate , and the calculation formula is:

[0099] .

[0100] Step 4: Evaluate the pre-trained supernet, eliminate low-performance candidate operations, and optimize the search space based on statistical analysis to improve search efficiency and the performance of the final architecture. The specific steps are as follows:

[0101] Step 4.1: Randomly sample 42 subnets from the supernet to ensure that they cover different structural combinations to provide sufficient architecture search.

[0102] Step 4.2: Sort the subnets according to the evaluation results, and count the operation frequencies of each candidate operation node in the top 50% of the ranked subnets.

[0103] Step 4.3: Discard the 1 operation with the lowest frequency among the upsampling candidate operations and the 3 operations with the lowest frequency among the ordinary candidate operations to improve search efficiency.

[0104] In the updated search space, the upsampling candidate operations have T - 1 operation choices, and the skip connections and ordinary candidate operations have Y - 3 operation choices. The updated supernet includes S u subnets, where S u is a natural number, .

[0105] Step 5: Initialize a subnet set P and a matching optimization network , where the subnet set P is used to maintain a dynamic list of the historical optimal subnets, and the matching optimization network is used to calculate the matching degree between the optimal subnet and the sampled subnets. The specific steps are as follows:

[0106] Step 5.1: Construct the subnet set to maintain a dynamic list of the historical optimal subnets, where represents the i-th optimal subnet during the supernet training process, is the weight inherited from the supernet. I is the maximum capacity set for the subnet set during initialization, and the perceptual similarity index IS and the Fréchet Inception Distance FID are set as the metrics for evaluating the subnets.

[0107] Step 5.2: Construct the matching optimization network to calculate the matching degree between the optimal subnet and the sampled subnets. The matching optimization network includes a single-layer fully connected layer. The input nodes are 32×32×3 (consistent with the image dimension of the training set CIFAR-10), and the output node is one. The output value represents the matching degree between the optimal subnet and the sampled subnet. The weight parameters of the matching optimization network are .

[0108] Step 6: Use the matching optimization network Select the historical architecture with the highest matching degree from the subnet set P as the teacher model of the currently sampled subnet; as Figure 2 shown, it specifically includes the following steps:

[0109] Step 6.1, sample a subnet from the supernet , and calculate the output of the subnet through forward propagation;

[0110] Step 6.2, use the matching optimization network to calculate the gap between the output of the subnet and the output of the optimal subnet in the subnet set P, and generate the matching degree;

[0111] Step 6.3, select the historical optimal subnet with the highest matching degree as the teacher network, and return the matching degree . The selection formula of the teacher network is:

[0112] ,

[0113] where is the output of the matching optimization network , representing the matching degree between the optimal subnet and the subnet , represents the weight inherited by the optimal subnet from the supernet, represents the weight inherited by the subnet from the supernet, represents the output image of the subnet , represents the output image of the optimal subnet . The input of the matching optimization network is the difference between the outputs of the optimal subnet and the subnet . This difference reflects the complementarity of the two subnets;

[0114] Step 7, calculate the architecture distillation loss and the generator loss, which are used to update the weights of the sampled subnet, and learn the optimization information from the teacher network through the architecture knowledge distillation technology to accelerate the training process; it specifically includes the following steps:

[0115] Step 7.1, calculate the architecture distillation loss between the teacher network and the subnet :

[0116] ,

[0117] in, Represents the teacher network weights inherited from the supernet, Represents the teacher network Output image, parameters Used to measure the difference between the output of the teacher generator and the sampled subnet generator; transfer the knowledge of the teacher network to the subnet through the architecture knowledge distillation technique In, accelerate the training process of the supernet;

[0118] Step 7.2, calculate the sampling subnet The generator loss :

[0119] ;

[0120] Step 7.3, Subnet The weight gradient update is determined by the generator loss and architecture distillation loss The weighted average of the two is jointly supervised, where the weight of the architecture distillation loss is determined by the matching degree of the output of the matching optimization network. Dynamically adjust to reflect the contribution of different teacher networks to the training of the sampled subnets. The weight update formula is:

[0121] ,

[0122] in, represents the iteration index, Represents the subnet before weight update The weight of Represents the subnet after weight update The weight of Indicates subnet The corresponding weight of the teacher network, Indicated in The subnetwork gradient calculated at is used to guide the update direction of the subnetwork weight. Represents the learning rate of the optimization objective function, which is used to control the magnitude of each update of the subnetwork weight;

[0123] Step 8: Use the back-propagation algorithm to update the weights of the matching optimization network by taking the generator loss after updating the weights as feedback.

[0124] In step 8, the weights of the sampled subnetworks are updated according to the architecture distillation operation Calculating the generator loss , using the generator loss as feedback, and updating the matching optimization network through back-propagation learning , the weight update formula of the matching optimization network is as follows:

[0125] ,

[0126] Among them, represents the generator loss for the gradient of the matching optimization network weight parameters ; represents the calculated at the subnet weight for gradient;

[0127] Step 9: Determine whether to update the subnet set according to the gap between the evaluation index and the historical optimal value, perform non-dominated sorting on the architectures in the subnet set using a multi-objective optimization algorithm, retain the Pareto front solutions, and remove the worst architecture when the storage capacity is exceeded;

[0128] In Step 9, sample a group of subnets from the supernet , and use the perceptual similarity index IS and the Frechet distance FID as indicators to evaluate the performance of the subnets. If the performance of the sampled subnets is better than the worst network architecture in the subnet set , then update the subnet set. The comparison and update formula is:

[0129] ,

[0130] and ,

[0131] Among them, represents the function for calculating the perceptual similarity index, represents the function for calculating the Frechet distance, represents update and selection , that is, select the eligible as the new ;

[0132] Step 10: Alternately execute Steps 6 to 9 until the preset maximum number of loops is reached, and obtain the trained supernet;

[0133] In Step 10, jointly optimize the supernet parameters using the architecture distillation loss and the generator loss, so that the supernet gradually approaches the optimal performance;

[0134] Step 11: Select the subnet with the optimal performance in the subnet set as the final network architecture to ensure that the overall performance of the generative adversarial network reaches the optimal; the generator network architecture searched from the supernet will be applied to image generation, and the input random noise will be converted into a color picture through the generator.

[0135] In step 11, the IS and FID metrics are combined to quantify the quality and diversity of the generated images. A multi-objective optimization strategy is adopted to balance the image quality and the similarity of the distributions between the generated images and the real images. The subnet with the best performance in the optimal subset of subnets is selected as the final optimal network architecture, which is specifically expressed as:

[0136] {Normal operations: [[skip connection, 5×5 convolution, 1×1 convolution, 3×3 convolution, no operation], [no operation, 3×3 convolution, 1×1 convolution, no operation, 1×1 convolution], [5×5 convolution, 1×1 convolution, skip connection, no operation, 5×5 convolution]], Upsampling operations: [[bilinear interpolation upsampling, bilinear interpolation upsampling], [nearest neighbor interpolation upsampling, nearest neighbor interpolation upsampling], [nearest neighbor interpolation upsampling, bilinear interpolation upsampling]], Skip connections of length 1: [bilinear interpolation upsampling, nearest neighbor interpolation upsampling], Skip connections of length 2: [nearest neighbor interpolation upsampling]}. Here, 1×1, 3×3, etc. represent the convolution kernel sizes. The content within each square bracket represents the operation combination of a unit. The units are arranged in the order of numbering, and the operations within the unit are arranged in the order of node numbering. The length of the skip connection refers to the spanning range of the connection: The skip connection of length 1 includes the connections from unit 0 to unit 1 and from unit 1 to unit 2, and the skip connection of length 2 is the connection from unit 0 to unit 2.

[0137] The generator network architecture searched from the supernet will be applied to the field of image generation, and the input random noise will be converted into a color picture through the generator.

[0138] This embodiment incorporates an architecture distillation technique on the basis of traditional generative adversarial network architecture search methods, achieving efficient and precise design of the generator architecture. It not only overcomes the dependence on external teacher models but also promotes the effective transfer of knowledge from high-performance teacher networks to generator student networks, significantly enhancing the stability and efficiency of the training process and laying a solid foundation for subsequent architecture search. In addition, this embodiment uses a multi-objective optimization algorithm in the search stage, skillfully balancing the delicate relationship between the quality of generated images and the distribution distance between generated images and real images, achieving the improvement of the efficiency and practicality of the architecture while maintaining the ability to generate high-quality images. This embodiment conducts experiments on two public datasets, CIFAR-10 and STL-10, and verifies its performance. CIFAR-10 contains 60,000 images of 10 different categories, of which 50,000 are used for network training and 10,000 are used for network testing, and the size of each image is 32×32. STL-10 contains 13,000 images of 10 different categories, with an original size of 96×96, which is larger than CIFAR-10 and provides a higher resolution. In this embodiment, to balance performance, its size is set to 48×48. The excellent performance of this embodiment on the benchmark dataset not only surpasses current mainstream methods but also demonstrates its strong generalization ability and scalability, opening up new avenues for the application of generative adversarial networks in more extensive image generation tasks. The performance of the network architecture obtained by the search of the present invention during optimization in the standard dataset CIFAR-10 is as Figure 3 and Figure 4 shown, Figure 3 and Figure 4 The Inception Score (IS) and Frechet Inception Distance (FID) in Figure 3 are commonly used performance metrics in generative adversarial networks. Among them, Figure 4 the IS score in is used to evaluate the diversity and inter-class separation of generated images. The magnitude of the score reflects the quality and diversity of the generated images. Therefore, the higher the score, the better. A higher IS score indicates that the generated images are not only visually diverse but also more attractive. At the same time, Figure 4 the FID score in measures the similarity between generated images and real images in the feature space. Therefore, the lower the score, the better. A lower FID score indicates that the feature distribution of the generated images is closer to the feature distribution of the real images, indicating that the generated samples are more realistic at the feature level. The method of this embodiment can discover architectures with excellent performance in an extremely short time, and these architectures can quickly converge to the best performance when retrained.

[0139] The present invention provides a method and system for searching a generative adversarial network architecture based on architecture distillation technology. There are many methods and ways to specifically implement this technical solution. The above description is only a preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by using the prior art.

Claims

1. A generative adversarial network architecture search method based on architecture distillation technology, characterized in that: The following steps are involved: Step 1, designing a search space for a generative adversarial network architecture search, defining a candidate operation set, and determining a scope of the architecture search, wherein the candidate operations in the candidate operation set include two types: ordinary candidate operations and upsampling candidate operations; Step 2: Based on the search space designed in step 1, a generative adversarial network supernet containing all candidate architectures is constructed, where the supernet covers all possible candidate architectures and supports subsequent architecture optimization. Step 3: Perform N w Round of warm-up training, using generative adversarial loss to optimize the weight parameters of the supernet; Step 4: Evaluate the pre-trained supernet, remove low-performance candidate operations, and optimize the search space based on statistical analysis to improve search efficiency and final architecture performance; Step 5: Initialize a subnet set P and a matching optimization network , where the subnet set P is used to maintain a dynamic list of historical optimal subnets, matching the optimized network Used to calculate the matching degree between the optimal subnet and the sampling subnet; Step 6: Use matching to optimize the network Select the historical architecture with the highest matching degree from the subnet set P as the teacher model of the current sampled subnet; Step 7: Calculate the architecture distillation loss and generator loss to update the weights of the sampling subnetwork. Use the architecture knowledge distillation technique to learn optimization information from the teacher network and accelerate the training process. Step 8: Use the back-propagation algorithm to update the weights of the matching optimization network by taking the generator loss after updating the weights as feedback. Step 9: Determine whether to update the subnetwork set based on the gap between the evaluation index and the historical optimal value, use a multi-objective optimization algorithm to perform non-dominated sorting on the architectures in the subnetwork set, retain the Pareto frontier solution, and remove the worst architecture when it exceeds the storage capacity; Step 10, alternately executing steps 6 to 9 until the preset maximum number of cycles is reached to obtain a trained supernet; Step 11: Select the subnet with the best performance from the subnet set as the final network architecture; the generator network architecture searched from the supernet will be applied to image generation, and the input random noise will be converted into a color picture through the generator.

2. The method according to claim 1, characterized in that In step 1, the search space includes N interconnected and skip-connected units, each unit is regarded as a directed acyclic graph consisting of Q nodes, and the directed acyclic graph contains U upsampling candidate operation nodes and R ordinary candidate operation nodes, satisfying , and N, Q, U, and R are all natural numbers.

3. The method according to claim 2, characterized in that In step 2, the generative adversarial network supernet has edge, where The edges are skip connections, The edge is where the upsampling candidate operation is located. The strips are edges where common candidate operations are located; the upsampling candidate operation has T operation options, and the skip connection and the common candidate operation have Y operation options; The supernet includes S subnets, where S is a natural number. , a subnet represents a network with only one operation on all connected edges.

4. The method according to claim 3, characterized in that Step 3 includes: Step 3.1, initialize the training parameters, which include the learning rate of the generator , the learning rate of the discriminator , the network parameter weights of the generator , the network parameter weights of the discriminator , optimizer parameters and fixed noise vector; Step 3.2, conduct adversarial training on the generator and discriminator, and use the discriminator to calculate the loss between the generated image and the real image; Step 3.3, through the back-propagation algorithm, update the weights of the discriminator and generator based on the loss obtained in step 3.

2.

5. The method according to claim 4, characterized in that Step 3.2 includes: Step 3.2.1, from the super network Fair sampling of a subnet ; Step 3.2.2, randomly sample the discrimination results of fake images from the standard normal distribution, calculate the hinge loss through the output of the discriminator, and update the hinge loss to the discriminator through back propagation. The loss function of the discriminator is for: , in, Indicates subnet The weights inherited from the supernet, E represents the expectation, Indicates that the real image x follows the real data distribution , Indicates that the noise vector z follows the noise distribution , represents the activation function, D represents the discriminator; Step 3.2.4, calculate the loss function of the generator : 。 6. The method according to claim 5, characterized in that Step 3.3 includes: Step 3.3.1, fix the network parameter weights of the generator , keeping the generator unchanged, according to the loss function of the discriminator , using the back-propagation method to calculate the gradient , and through the gradient descent method, according to the discriminator learning rate Update the network parameter weights of the discriminator , the calculation formula is: ； Step 3.3.2, fix the network parameter weights of the discriminator , keeping the discriminator unchanged, according to the loss function of the generator Calculate gradients using backpropagation , and use the gradient descent method, according to the learning rate of the generator Update the network parameter weights of the generator , the calculation formula is: 。 7. The method according to claim 6, characterized in that Step 4 includes: Step 4.1, randomly sample a specified number of subnets from the supernet and use the perceptual similarity index IS to evaluate the performance of the subnets; Step 4.2, sort the subnets according to the evaluation results, and count the operation frequency of each candidate operation node in the top Y1 subnets; Step 4.3, discard the one operation with the lowest frequency among the upsampling candidate operations and the three operations with the lowest frequency among the common candidate operations; In the updated search space, the upsampling candidate operation has T-1 operation options, and the skip connection and the normal candidate operation have Y-3 operation options; the updated supernet includes S u subnets, where S u is a natural number, .

8. The method according to claim 7, characterized in that Step 5 includes: Step 5.1: Build a subnet set , used to maintain a dynamic list of historical optimal subnets, where represents the optimal i-th subnet in the supernet training process, for The weights inherited from the supernet, I, set the maximum capacity of the subnet set at initialization, and set the perceptual similarity index IS and Fréchet distance FID as indicators for evaluating the subnet; Step 5.2: Build a matching optimization network , which is used to calculate the matching degree between the optimal subnet and the sampling subnet. The matching optimization network includes a single fully connected layer. The input node is consistent with the image dimension of the training set, and the output node is one. The output value is expressed as the matching degree between the optimal subnet and the sampling subnet. The matching optimization network The weight parameter is .

9. The method according to claim 8, characterized in that Step 6 includes: Step 6.1, from the super network Sample a subnet , calculate the subnetwork through forward propagation Output: Step 6.2, use matching to optimize the network Compute Subnet The output of The gap between the outputs of , generates the matching degree; Step 6.3: Select the historical optimal subnet with the highest matching degree As a teacher network, and will match Return, the selection formula of the teacher network is: ， in, It is a matching optimization network The output of represents the optimal subnet With subnet The matching degree between Represents the optimal subnet weights inherited from the supernet, Indicates subnet weights inherited from the supernet, Indicates subnet The output image, Represents the optimal subnet The output image of the matching optimization network The input is the optimal subnet With subnet The difference between the outputs; Step 7 includes: Step 7.1, calculate the teacher network With subnet The architecture distillation loss between : ， in, Represents the teacher network weights inherited from the supernet, Represents the teacher network Output image, parameters Used to measure the difference between the output of the teacher generator and the sampled subnetwork generator; Step 7.2, calculate the sampling subnet The generator loss : ； Step 7.3, Subnet The weight gradient update is determined by the generator loss and architecture distillation loss The weighted average of the two is jointly supervised, where the weight of the architecture distillation loss is determined by the matching degree of the output of the matching optimization network. Dynamic adjustment, subnet The weight update formula is: ， in, represents the iteration index, Represents the subnet before weight update The weight of Represents the subnet after weight update The weight of Indicates subnet The corresponding weight of the teacher network, Indicated in The subnetwork gradient calculated at is used to guide the update direction of the subnetwork weight. Represents the learning rate of the optimization objective function, which is used to control the magnitude of each update of the subnetwork weight; In step 8, the weights of the sampled subnetworks are updated according to the architecture distillation operation Calculating the generator loss , using the generator loss as feedback, and updating the matching optimization network through back-propagation learning , the weight update formula of the matching optimization network is as follows: , in, represents the generator loss Optimize network weight parameters for matching The gradient of Indicates the weight of the subnet Calculated right The gradient of In step 9, from the super network A set of subnets are sampled from the set, and the perceptual similarity index IS and Fréchet distance FID are used as indicators to evaluate the subnet performance. If the sampled subnet performs better than the subnet set The worst performing network architecture in the network is updated, and the update formula is: ， and , in, represents the function for finding the perceptual similarity index, represents the function of finding the Fréchet distance, Indicates update and selection , that is, select the eligible As new ; In step 10, the architecture distillation loss and the generator loss are used to jointly optimize the supernet parameters so that the supernet gradually approaches the optimal performance; In step 11, the IS and FID indicators are combined to quantify the quality and diversity of the generated images, and a multi-objective optimization strategy is used to balance the image quality and the similarity of the distribution between the generated images and the real images. The subnet with the best performance in the optimal subnet set is selected as the final optimal network architecture; the generator network architecture searched from the supernet will be applied to the field of image generation, and the input random noise will be converted into a color picture through the generator.

10. A generative adversarial network architecture search system based on architecture distillation technology implemented by the method according to any one of claims 1 to 9, characterized in that: include: Construction module, used to build the generative adversarial network supernet, optimal subnet set and matching optimization network; The architecture distillation module is used to transfer the knowledge of the optimal subnet to the sampling subnet, improving the training efficiency of the supernet; The training module is used to train the network weights of the generator supernet using architecture distillation loss and adversarial loss to ensure that the final network architecture performance is optimal; The search module is used to search for the subnet architecture with the best performance in the supernet, perform multi-objective search in the optimal subnet set, and select the subnet with the best performance as the final network architecture.

Images