Generative adversarial network architecture searching method and system based on architecture distillation technology

Abstract

The invention provides a generative adversarial network architecture search method and system based on an architecture distillation technology, and the method comprises the steps: defining a search space, constructing a hypernet integrating all candidate operations, carrying out the preheating training of the hypernet, and evaluating the performance of a subnet after the initial preheating training. And screening out and discarding the operation unfavorable for the network performance. In the training process of the super network, network weight optimization is performed through an architecture knowledge distillation method, a matching optimization network is designed to effectively select a teacher network, so that the teacher network and a student sub-network are optimally matched, and knowledge of the teacher network is transmitted to other sub-networks through an architecture distillation technology to guide super network training. And a multi-objective optimization strategy is used to select the network architecture with the best performance in the super network. According to the method, the training effect of the subnet is improved through knowledge transmission between the architectures, compared with a traditional search method, the efficiency is higher, the performance is better, and the optimal neural network architecture suitable for tasks can be found more quickly.

Description

Technical Field

[0001] The present invention relates to the field of deep neural network technology, and in particular to a generative adversarial network architecture search method and system based on architecture distillation technology. Background Art

[0002] Since its introduction, generative adversarial networks have occupied an important position in the field of deep learning, especially showing strong generative capabilities in computer vision tasks. Generative adversarial networks consist of a generator and a discriminator. Through the dynamic game between the two, the generator can learn the distribution of real data and generate high-quality synthetic data. Due to its strong data generation ability, generative adversarial networks have been widely used in many fields such as image generation, image restoration, super-resolution reconstruction, image translation, image enhancement, and text-to-image generation, greatly promoting the development of these fields. However, the training process of generative adversarial networks is often accompanied by instability problems such as mode collapse and gradient disappearance. These challenges limit the further improvement of its performance and its wide application.

[0003] To overcome the instability of generative adversarial network training, researchers have actively explored a variety of strategies, including designing novel network architectures, optimizing loss functions, and introducing regularization methods. Although these manual interventions have alleviated some challenges, manually designing generative adversarial network architectures is still time-consuming, suboptimal, and difficult to generalize. Among them, neural architecture search has been widely used in the automated design of deep learning models in recent years, and has achieved success in tasks such as classification and detection. Neural architecture search automatically discovers the optimal neural network architecture through a search algorithm to improve model performance. However, the application of neural architecture search methods in generative adversarial network tasks still faces many challenges. Due to the extensive evaluation of candidate architectures, traditional neural architecture search is computationally expensive, which limits its scalability.

[0004] In order to solve the computational problems faced by neural architecture search in generative adversarial networks, researchers proposed a one-time neural architecture search method, which uses a weight-sharing supernet strategy to enable multiple sub-networks to share the same weights, effectively reducing the training cost. However, in the generative adversarial network task, the one-time weight model has inaccurate evaluation results due to insufficient sub-network training, which in turn weakens the effectiveness and reliability of the search process. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a generative adversarial network architecture search method and system based on architecture distillation technology in response to the shortcomings of the prior art.

[0006] The method comprises the following steps: Step 1, designing a search space for a generative adversarial network architecture search, defining a candidate operation set, and determining a scope of the architecture search, wherein the candidate operations in the candidate operation set include two types: ordinary candidate operations and upsampling candidate operations; Step 2: Based on the search space designed in step 1, a generative adversarial network supernet containing all candidate architectures is constructed, where the supernet covers all possible candidate architectures and supports subsequent architecture optimization. Step 3: Perform N w Round of warm-up training, using generative adversarial loss to optimize the weight parameters of the supernet; Step 4: Evaluate the pre-trained supernet, remove low-performance candidate operations, and optimize the search space based on statistical analysis to improve search efficiency and final architecture performance; Step 5: Initialize a subnet set P and a matching optimization network , where the subnet set P is used to maintain a dynamic list of historical optimal subnets, matching the optimized network Used to calculate the matching degree between the optimal subnet and the sampling subnet; Step 6: Use matching to optimize the network Select the historical architecture with the highest matching degree from the subnet set P as the teacher model of the current sampled subnet; Step 7: Calculate the architecture distillation loss and generator loss to update the weights of the sampling subnetwork. Use the architecture knowledge distillation technique to learn optimization information from the teacher network and accelerate the training process. Step 8: Use the back-propagation algorithm to update the weights of the matching optimization network by taking the generator loss after updating the weights as feedback. Step 9: Determine whether to update the subnetwork set based on the gap between the evaluation index and the historical optimal value, use a multi-objective optimization algorithm to perform non-dominated sorting on the architectures in the subnetwork set, retain the Pareto frontier solution, and remove the worst architecture when it exceeds the storage capacity; Step 10, alternately executing steps 6 to 9 until the preset maximum number of cycles is reached to obtain a trained supernet; Step 11: Select the subnet with the best performance from the subnet set as the final network architecture to ensure that the overall performance of the generative adversarial network is optimal; the generator network architecture searched from the supernet will be applied to image generation, and the input random noise will be converted into a color picture through the generator.

[0007] In step 1, the search space includes N interconnected and skip-connected units, each unit is regarded as a directed acyclic graph consisting of Q nodes, and the directed acyclic graph contains U upsampling candidate operation nodes and R ordinary candidate operation nodes, satisfying , and N, Q, U, and R are all natural numbers.

[0008] In step 2, the generative adversarial network supernet has edge, where The edges are skip connections, The edge is where the upsampling candidate operation is located. The strips are edges where common candidate operations are located; the upsampling candidate operation has T operation options, and the skip connection and the common candidate operation have Y operation options; The supernet includes S subnets, where S is a natural number. , a subnet represents a network with only one operation on all connected edges.

[0009] Step 3 includes: Step 3.1, initialize the training parameters, which include the learning rate of the generator , the learning rate of the discriminator , the network parameter weights of the generator , the network parameter weights of the discriminator , optimizer parameters and fixed noise vector; Step 3.2, conduct adversarial training on the generator and discriminator, and use the discriminator to calculate the loss between the generated image and the real image; In step 3.3, the weights of the discriminator and generator are updated based on the loss obtained in step 3.2 through the back-propagation algorithm to ensure that the model converges effectively.

[0010] Step 3.2 includes: Step 3.2.1, from the super network Fair sampling of a subnet ; Step 3.2.2, randomly sample the discrimination results of fake images from the standard normal distribution. The higher the result, the higher the probability that the discriminator thinks the input content is true. The hinge loss is calculated through the output of the discriminator, and the hinge loss is updated to the discriminator through back propagation. The loss function of the discriminator is for: , in, Indicates subnet The weights inherited from the supernet, E represents the expectation, Indicates that the real image x follows the real data distribution , Indicates that the noise vector z follows the noise distribution , represents the activation function, D represents the discriminator; Step 3.2.4, calculate the loss function of the generator : .

[0011] Step 3.3 includes: Step 3.3.1, fix the network parameter weights of the generator , keeping the generator unchanged, according to the loss function of the discriminator , using the back-propagation method to calculate the gradient , and through the gradient descent method, according to the discriminator learning rate Update the network parameter weights of the discriminator , the calculation formula is: ; Step 3.3.2, fix the network parameter weights of the discriminator , keeping the discriminator unchanged, according to the loss function of the generator Calculate gradients using backpropagation , and use the gradient descent method, according to the learning rate of the generator Update the network parameter weights of the generator , the calculation formula is: .

[0012] Step 4 includes: Step 4.1, randomly sample a specified number (usually 42) of subnets from the supernet, and use the perceptual similarity index IS to evaluate the performance of the subnets; Step 4.2, sort the subnets according to the evaluation results, and count the operation frequency of each candidate operation node in the top Y1 (generally 50%) subnets; In step 4.3, the operation with the lowest frequency among the upsampling candidate operations and the three operations with the lowest frequency among the common candidate operations are discarded to improve the search efficiency.

[0013] In the updated search space, the upsampling candidate operation has T-1 operation options, and the skip connection and the normal candidate operation have Y-3 operation options; the updated supernet includes S u subnets, where S u is a natural number, .

[0014] Step 5 includes: Step 5.1: Build a subnet set , used to maintain a dynamic list of historical optimal subnets, where represents the optimal i-th subnet in the supernet training process, for The weights inherited from the supernet, I, set the maximum capacity of the subnet set at initialization, and set the perceptual similarity index IS and Fréchet distance FID as indicators for evaluating the subnet; Step 5.2: Build a matching optimization network , which is used to calculate the matching degree between the optimal subnet and the sampled subnet. The matching optimization network includes a single fully connected layer, with 32×32×3 input nodes (consistent with the image dimensions of the training set CIFAR-10), one output node, and the output value represents the matching degree between the optimal subnet and the sampled subnet. The matching optimization network The weight parameter is .

[0015] Step 6 includes: Step 6.1, from the super network Sample a subnet , calculate the subnetwork through forward propagation Output: Step 6.2, use matching to optimize the network Compute Subnet The output of The gap between the outputs of , generates the matching degree; Step 6.3: Select the historical optimal subnet with the highest matching degree As a teacher network, and will match Return, the selection formula of the teacher network is: , in, It is a matching optimization network The output of represents the optimal subnet With subnet The matching degree between Represents the optimal subnet weights inherited from the supernet, Indicates subnet weights inherited from the supernet, Indicates subnet The output image, Represents the optimal subnet The output image of the matching optimization network The input is the optimal subnet With subnet The difference between the outputs reflects the complementarity of the two subnetworks; Step 7 includes: Step 7.1, calculate the teacher network With subnet The architecture distillation loss between : , in, Represents the teacher network weights inherited from the supernet, Represents the teacher network Output image, parameters Used to measure the difference between the output of the teacher generator and the sampled subnet generator; transfer the knowledge of the teacher network to the subnet through the architecture knowledge distillation technique In, accelerate the training process of the supernet; Step 7.2, calculate the sampling subnet The generator loss : ; Step 7.3, Subnet The weight gradient update is determined by the generator loss and architecture distillation loss The weighted average of the two is jointly supervised, where the weight of the architecture distillation loss is determined by the matching degree of the output of the matching optimization network. Dynamically adjust to reflect the contribution of different teacher networks to the training of the sampled subnets. The weight update formula is: , in, represents the iteration index, Represents the subnet before weight update The weight of Represents the subnet after weight update The weight of Indicates subnet The corresponding weight of the teacher network, Indicated in The subnetwork gradient calculated at is used to guide the update direction of the subnetwork weight. Represents the learning rate of the optimization objective function, which is used to control the magnitude of each update of the subnetwork weight; In step 8, the weights of the sampled subnetworks are updated according to the architecture distillation operation Calculating the generator loss , using the generator loss as feedback, and updating the matching optimization network through back-propagation learning , the weight update formula of the matching optimization network is as follows: , in, represents the generator loss Optimize network weight parameters for matching The gradient of Indicates the weight of the subnet Calculated right The gradient of In step 9, from the super network A set of subnets are sampled from the set, and the perceptual similarity index IS and Fréchet distance FID are used as indicators to evaluate the subnet performance. If the sampled subnet performs better than the subnet set The worst performing network architecture in the network is updated, and the update formula is: , and , in, represents the function for finding the perceptual similarity index, represents the function of finding the Fréchet distance, Indicates update and selection , that is, select the eligible As new ; In step 10, the architecture distillation loss and the generator loss are used to jointly optimize the supernet parameters so that the supernet gradually approaches the optimal performance; In step 11, the IS and FID indicators are combined to quantify the quality and diversity of the generated images, and a multi-objective optimization strategy is used to balance the image quality and the similarity of the distribution between the generated images and the real images. The subnet with the best performance in the optimal subnet set is selected as the final optimal network architecture; the generator network architecture searched from the supernet will be applied to the field of image generation, and the input random noise will be converted into a color picture through the generator.

[0016] The present invention also provides a generative adversarial network architecture search system based on architecture distillation technology, comprising: Construction module, used to build the generative adversarial network supernet, optimal subnet set and matching optimization network; The architecture distillation module is used to transfer the knowledge of the optimal subnet to the sampling subnet, improving the training efficiency of the supernet; The training module is used to train the network weights of the generator supernet using architecture distillation loss and adversarial loss to ensure that the final network architecture performance is optimal; The search module is used to search for the subnet architecture with the best performance in the supernet, perform multi-objective search in the optimal subnet set, and select the subnet with the best performance as the final network architecture.

[0017] This invention innovatively introduces architecture distillation technology, by identifying and selecting high-performance architectures as teacher networks, and matching the subnet with the teacher network using matching optimization network matching, ensuring that the subnet can inherit the essence of the teacher network, and efficiently guiding the optimization of the supernet to accelerate knowledge transfer, significantly improving the stability of generative adversarial network training and the quality of generated images. In the supernet search stage, this paper uses the non-dominated sorting genetic algorithm II, combined with Inception Score (IS) and Fréchet Inception Distance (FID) as performance evaluation indicators to ensure the efficiency of the searched subnet architecture and achieve dual optimization of the generated image quality and the similarity of the real data distribution.

[0018] The beneficial effects of the present invention include: the method of the present invention proposes a generative adversarial network architecture search algorithm based on architecture distillation technology for neural network architecture search for generative adversarial networks. The algorithm effectively improves the training effect of the subnet by introducing architecture distillation technology into the supernet training process of the generative adversarial network by imitating the behavior of the high-performance teacher network, and can achieve efficient knowledge transfer without pre-training the network, significantly enhancing the stability and efficiency of the training process, and providing new ideas for the automated optimization of generative adversarial networks. In the search process, the NSGA-II algorithm is used for multi-objective optimization in combination with IS and FID, which balances the image quality and the similarity of the distribution between the generated image and the real image, providing strong support for the search of efficient GANs architectures. This generative adversarial network architecture search algorithm based on architecture distillation has brought new ideas and breakthroughs to the field of architecture search, and has sufficient practical value and application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is the overall framework diagram of the search space in the present invention.

[0020] Figure 2 Schematic diagram of generator supernet training in the present invention.

[0021] Figure 3 IS score trend chart for architecture optimization in the present invention.

[0022] Figure 4 The FID score trend chart is optimized for the architecture in the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0024] like Figure 1As shown, this example provides a method and system for searching a generative adversarial network architecture based on architecture distillation technology, including the following steps: Step 1, first design Figure 1 The search space for the architecture search of the generative adversarial network generator shown in the figure consists of 3 interconnected and skip-connected units, and the unit can be viewed as a directed acyclic graph with 5 interconnected nodes. The blue arrows represent upsampling operations, and the purple arrows represent common operations. The upsampling candidate operations are set to {transposed convolution upsampling, nearest neighbor interpolation upsampling, bilinear interpolation upsampling}, and the common candidate operations are set to {1×1 convolution, 3×3 convolution, 5×5 convolution, 3×3 dilated convolution, 5×5 dilated convolution, no operation, skip connection}.

[0025] Step 2, according to Figure 1 The search space shown in the figure builds a generative adversarial network supernet containing all candidate architectures, and randomly initializes the supernet. Randomly generate input noise with 128 channels to input the supernet to generate images. The input direction of the supernet feature map is unidirectional from front to back. A supernet is a general term for all candidate architectures in a search space, and a subnet represents a specific architecture selected in a given search space, with each subnet being part of a supernet. Traditionally, input information is usually processed by a single operation (such as convolution) to generate a new feature map. However, in a supernet, the input information is processed by multiple operations, such as dynamic separable convolution, dilated convolution, and skip connections. The feature maps generated by each of these operations are fused by adding the elements bit by bit, and finally a new hybrid feature map is obtained. During the training process, through learning and optimization, the supernet can automatically select and adjust the weights of these candidate operations, thereby ultimately determining the optimal combination of operations and network structure.

[0026] In the present invention, there are T candidate operations on the upsampling edge of the supernet, including nearest neighbor interpolation upsampling, bilinear interpolation upsampling and transposed convolution upsampling, and there are Y candidate operations in ordinary candidate operations and skip connections, including 1×1 convolution, 3×3 convolution, 5×5 convolution, 3×3 dilated convolution, 5×5 dilated convolution, skip connection and empty operation.

[0027] Step 3: Perform N w The warm-up training of the first round uses the generative adversarial loss to optimize the weight parameters of the supernet, which includes the following steps: Step 3.1, initialize the training parameters, which include the learning rate of the generator , the learning rate of the discriminator , the network parameter weights of the generator , the network parameter weights of the discriminator , optimizer parameters and fixed noise vector; Step 3.2, conduct adversarial training on the generator and the discriminator. During the training process, the generator and the discriminator are optimized alternately, and the discriminator is used to calculate the loss between the generated image and the real image. Specifically, the following steps are included: Step 3.2.1, from the super network Fair sampling of a subnet ; Step 3.2.2, randomly sample the discrimination results of fake images from the standard normal distribution. The higher the result, the higher the probability that the discriminator thinks the input content is true. The hinge loss is calculated through the output of the discriminator, and the hinge loss is updated to the discriminator through back propagation. The loss function of the discriminator is for: , in, Indicates subnet The weights inherited from the supernet, E represents the expectation, Indicates that the real image x follows the real data distribution , Indicates that the noise vector z follows the noise distribution , represents the activation function, D represents the discriminator; Step 3.2.4, calculate the loss function of the generator : .

[0028] Step 3.3, through the back propagation algorithm, update the weights of the discriminator and generator based on the loss obtained in step 3.2 to ensure that the model converges effectively. Specifically, the steps are as follows: Step 3.3.1, fix the network parameter weights of the generator , keeping the generator unchanged, according to the loss function of the discriminator , using the back-propagation method to calculate the gradient , and through the gradient descent method, according to the discriminator learning rate Update the network parameter weights of the discriminator , the calculation formula is: ; Step 3.3.2, fix the network parameter weights of the discriminator , keeping the discriminator unchanged, according to the loss function of the generator Calculate gradients using backpropagation , and use the gradient descent method, according to the learning rate of the generator Update the network parameter weights of the generator , the calculation formula is: .

[0029] Step 4: Evaluate the pre-trained supernet, remove low-performance candidate operations, and optimize the search space based on statistical analysis to improve search efficiency and final architecture performance; specifically, the following steps are included: Step 4.1, randomly sample 42 subnets from the supernet to ensure that they cover different structural combinations to provide sufficient architecture search; Step 4.2, sort the subnets according to the evaluation results, and count the operation frequency of each candidate operation node in the top 50% subnets; In step 4.3, the operation with the lowest frequency among the upsampling candidate operations and the three operations with the lowest frequency among the common candidate operations are discarded to improve the search efficiency.

[0030] In the updated search space, the upsampling candidate operation has T-1 operation options, and the skip connection and the normal candidate operation have Y-3 operation options; the updated supernet includes S u subnets, where S u is a natural number, .

[0031] Step 5: Initialize a subnet set P and a matching optimization network , where the subnet set P is used to maintain a dynamic list of historical optimal subnets, matching the optimized network It is used to calculate the matching degree between the optimal subnet and the sampling subnet. It specifically includes the following steps: Step 5.1: Build a subnet set , used to maintain a dynamic list of historical optimal subnets, where represents the optimal i-th subnet in the supernet training process, for The weights inherited from the supernet, I, set the maximum capacity of the subnet set at initialization, and set the perceptual similarity index IS and Fréchet distance FID as indicators for evaluating the subnet; Step 5.2: Build a matching optimization network , which is used to calculate the matching degree between the optimal subnet and the sampled subnet. The matching optimization network includes a single fully connected layer, with 32×32×3 input nodes (consistent with the image dimensions of the training set CIFAR-10), one output node, and the output value represents the matching degree between the optimal subnet and the sampled subnet. The matching optimization network The weight parameter is .

[0032] Step 6: Use matching to optimize the network Select the historical architecture with the highest matching degree from the subnet set P as the teacher model of the current sampling subnet; Figure 2 As shown, the specific steps include: Step 6.1, from the super network Sample a subnet , calculate the subnetwork through forward propagation Output: Step 6.2, use matching to optimize the network Compute Subnet The output of The gap between the outputs of , generates the matching degree; Step 6.3: Select the historical optimal subnet with the highest matching degree As a teacher network, and will match Return, the selection formula of the teacher network is: , in, It is a matching optimization network The output of represents the optimal subnet With subnet The matching degree between Represents the optimal subnet weights inherited from the supernet, Indicates subnet weights inherited from the supernet, Indicates subnet The output image, Represents the optimal subnet The output image of the matching optimization network The input is the optimal subnet With subnet The difference between the outputs reflects the complementarity of the two subnetworks; Step 7: Calculate the architecture distillation loss and generator loss to update the weights of the sampling subnet, learn optimization information from the teacher network through architecture knowledge distillation technology, and accelerate the training process; specifically, the following steps are included: Step 7.1, calculate the teacher network With subnet The architecture distillation loss between : , in, Represents the teacher network weights inherited from the supernet, Represents the teacher network Output image, parameters Used to measure the difference between the output of the teacher generator and the sampled subnet generator; transfer the knowledge of the teacher network to the subnet through the architecture knowledge distillation technique In, accelerate the training process of the supernet; Step 7.2, calculate the sampling subnet The generator loss : ; Step 7.3, Subnet The weight gradient update is determined by the generator loss and architecture distillation loss The weighted average of the two is jointly supervised, where the weight of the architecture distillation loss is determined by the matching degree of the output of the matching optimization network. Dynamically adjust to reflect the contribution of different teacher networks to the training of the sampled subnets. The weight update formula is: , in, represents the iteration index, Represents the subnet before weight update The weight of Represents the subnet after weight update The weight of Indicates subnet The corresponding weight of the teacher network, Indicated in The subnetwork gradient calculated at is used to guide the update direction of the subnetwork weight. Represents the learning rate of the optimization objective function, which is used to control the magnitude of each update of the subnetwork weight; Step 8: Use the back-propagation algorithm to update the weights of the matching optimization network by taking the generator loss after updating the weights as feedback. In step 8, the weights of the sampled subnetworks are updated according to the architecture distillation operation Calculating the generator loss , using the generator loss as feedback, and updating the matching optimization network through back-propagation learning , the weight update formula of the matching optimization network is as follows: , in, represents the generator loss Optimize network weight parameters for matching The gradient of Indicates the weight of the subnet Calculated right The gradient of Step 9: Determine whether to update the subnetwork set based on the gap between the evaluation index and the historical optimal value, use a multi-objective optimization algorithm to perform non-dominated sorting on the architectures in the subnetwork set, retain the Pareto frontier solution, and remove the worst architecture when it exceeds the storage capacity; In step 9, from the super network A set of subnets are sampled from the set, and the perceptual similarity index IS and Fréchet distance FID are used as indicators to evaluate the subnet performance. If the sampled subnet performs better than the subnet set The worst performing network architecture in the network is updated, and the update formula is: , and , in, represents the function for finding the perceptual similarity index, represents the function of finding the Fréchet distance, Indicates update and selection , that is, select the eligible As new ; Step 10, alternately executing steps 6 to 9 until the preset maximum number of cycles is reached to obtain a trained supernet; In step 10, the architecture distillation loss and the generator loss are used to jointly optimize the supernet parameters so that the supernet gradually approaches the optimal performance; Step 11: Select the subnet with the best performance from the subnet set as the final network architecture to ensure that the overall performance of the generative adversarial network is optimal; the generator network architecture searched from the supernet will be applied to image generation, and the input random noise will be converted into a color picture through the generator.

[0033] In step 11, the IS and FID indicators are combined to quantify the quality and diversity of the generated images, and a multi-objective optimization strategy is used to balance the image quality and the similarity of the distribution between the generated images and the real images. The subnet with the best performance in the optimal subnet set is selected as the final optimal network architecture; specifically, it is expressed as: {Normal operation: [[skip connection, 5×5 convolution, 1×1 convolution, 3×3 convolution, no operation], [no operation, 3×3 convolution, 1×1 convolution, no operation, 1×1 convolution], [5×5 convolution, 1×1 convolution, skip connection, no operation, 5×5 convolution]], upsampling operation: [[bilinear interpolation upsampling, bilinear interpolation upsampling], [nearest neighbor interpolation upsampling, nearest neighbor interpolation upsampling], [nearest neighbor interpolation upsampling, bilinear interpolation upsampling]], skip connection of length 1: [bilinear interpolation upsampling, nearest neighbor interpolation upsampling], skip connection of length 2: [nearest neighbor interpolation upsampling]}. Among them, 1×1, 3×3, etc. represent the size of the convolution kernel. The content in each square bracket represents the operation combination of a unit. The units are arranged in numerical order, and the operations within the unit are arranged in order of node numbers. The length of a skip connection refers to the span of the connection: a skip connection of length 1 includes the connection from unit 0 to unit 1 and from unit 1 to unit 2, and a skip connection of length 2 includes the connection from unit 0 to unit 2.

[0034] The generator network architecture searched from the supernet will be applied to the field of image generation, converting the input random noise into a color picture through the generator.

[0035] This embodiment adds architecture distillation technology to the traditional generative adversarial network architecture search method to achieve efficient and accurate design of the generator architecture, which not only overcomes the dependence on the external teacher model, but also promotes the effective transfer of knowledge from the high-performance teacher network to the generator student network, significantly enhancing the stability and efficiency of the training process, and laying a solid foundation for subsequent architecture search. In addition, this embodiment uses a multi-objective optimization algorithm in the search phase, cleverly balancing the subtle relationship between the quality of the generated image and the distribution distance between the generated image and the real image, and achieving while maintaining the ability to generate high-quality images, while improving the efficiency and practicality of the architecture. This embodiment conducted experiments on two public data, CIFAR-10 and STL-10, and verified the performance. CIFAR-10 contains 60,000 pictures of 10 different categories, of which 50,000 are used for network training and 10,000 are used for network testing, and the size of the pictures is 32×32. STL-10 contains 13,000 images of 10 different categories. The image size is 96×96, which is larger than STL-10 and provides a higher resolution. In this embodiment, in order to balance the performance, its size is set to 48×48. The excellent performance of this embodiment on the benchmark dataset not only surpasses the current mainstream methods, but also demonstrates its strong generalization ability and scalability, opening up new ways for the application of generative adversarial networks in a wider range of image generation tasks. The performance of the network architecture searched by the present invention when optimized in the standard dataset CIFAR-10 is shown in the figure below. Figure 3 and Figure 4 As shown, Figure 3 and Figure 4 The Inception Score (IS) and Frechet Inception Distance (FID) in the Generative Adversarial Network are commonly used performance indicators. Figure 3 The IS score in is used to evaluate the diversity and inter-class separation of generated images. The size of the score reflects the quality and diversity of the generated images. Therefore, the higher the score, the better. A higher IS score indicates that the generated images are not only visually diverse, but also more attractive. At the same time, Figure 4 The FID score in measures the similarity between the generated image and the real image in the feature space, so the lower the score, the better. A lower FID score indicates that the feature distribution of the generated image is closer to that of the real image, indicating that the generated sample is more realistic at the feature level. The method of this embodiment can discover architectures with excellent performance in a very short time, and these architectures can quickly converge to the best performance when retraining.

[0036] The present invention provides a method and system for searching a generative adversarial network architecture based on architecture distillation technology. There are many methods and ways to implement the technical solution. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A generative adversarial network architecture search method based on architecture distillation technology, characterized in that: The following steps are involved: Step 1, designing a search space for a generative adversarial network architecture search, defining a candidate operation set, and determining a scope of the architecture search, wherein the candidate operations in the candidate operation set include two types: ordinary candidate operations and upsampling candidate operations; Step 2: Based on the search space designed in step 1, a generative adversarial network supernet containing all candidate architectures is constructed, where the supernet covers all possible candidate architectures and supports subsequent architecture optimization. Step 3: Perform N w Round of warm-up training, using generative adversarial loss to optimize the weight parameters of the supernet; Step 4: Evaluate the pre-trained supernet, remove low-performance candidate operations, and optimize the search space based on statistical analysis to improve search efficiency and final architecture performance; Step 5: Initialize a subnet set P and a matching optimization network , where the subnet set P is used to maintain a dynamic list of historical optimal subnets, matching the optimized network Used to calculate the matching degree between the optimal subnet and the sampling subnet; Step 6: Use matching to optimize the network Select the historical architecture with the highest matching degree from the subnet set P as the teacher model of the current sampled subnet; Step 7: Calculate the architecture distillation loss and generator loss to update the weights of the sampling subnetwork. Use the architecture knowledge distillation technique to learn optimization information from the teacher network and accelerate the training process. Step 8: Use the back-propagation algorithm to update the weights of the matching optimization network by taking the generator loss after updating the weights as feedback. Step 9: Determine whether to update the subnetwork set based on the gap between the evaluation index and the historical optimal value, use a multi-objective optimization algorithm to perform non-dominated sorting on the architectures in the subnetwork set, retain the Pareto frontier solution, and remove the worst architecture when it exceeds the storage capacity; Step 10, alternately executing steps 6 to 9 until the preset maximum number of cycles is reached to obtain a trained supernet; Step 11: Select the subnet with the best performance from the subnet set as the final network architecture; the generator network architecture searched from the supernet will be applied to image generation, and the input random noise will be converted into a color picture through the generator.

2. The method according to claim 1, characterized in that In step 1, the search space includes N interconnected and skip-connected units, each unit is regarded as a directed acyclic graph consisting of Q nodes, and the directed acyclic graph contains U upsampling candidate operation nodes and R ordinary candidate operation nodes, satisfying , and N, Q, U, and R are all natural numbers.

3. The method according to claim 2, characterized in that In step 2, the generative adversarial network supernet has edge, where The edges are skip connections, The edge is where the upsampling candidate operation is located. The strips are edges where common candidate operations are located; the upsampling candidate operation has T operation options, and the skip connection and the common candidate operation have Y operation options; The supernet includes S subnets, where S is a natural number. , a subnet represents a network with only one operation on all connected edges.

4. The method according to claim 3, characterized in that Step 3 includes: Step 3.1, initialize the training parameters, which include the learning rate of the generator , the learning rate of the discriminator , the network parameter weights of the generator , the network parameter weights of the discriminator , optimizer parameters and fixed noise vector; Step 3.2, conduct adversarial training on the generator and discriminator, and use the discriminator to calculate the loss between the generated image and the real image; Step 3.3, through the back-propagation algorithm, update the weights of the discriminator and generator based on the loss obtained in step 3.

2.

5. The method according to claim 4, characterized in that Step 3.2 includes: Step 3.2.1, from the super network Fair sampling of a subnet ; Step 3.2.2, randomly sample the discrimination results of fake images from the standard normal distribution, calculate the hinge loss through the output of the discriminator, and update the hinge loss to the discriminator through back propagation. The loss function of the discriminator is for: , in, Indicates subnet The weights inherited from the supernet, E represents the expectation, Indicates that the real image x follows the real data distribution , Indicates that the noise vector z follows the noise distribution , represents the activation function, D represents the discriminator; Step 3.2.4, calculate the loss function of the generator : 。 6. The method according to claim 5, characterized in that Step 3.3 includes: Step 3.3.1, fix the network parameter weights of the generator , keeping the generator unchanged, according to the loss function of the discriminator , using the back-propagation method to calculate the gradient , and through the gradient descent method, according to the discriminator learning rate Update the network parameter weights of the discriminator , the calculation formula is: ； Step 3.3.2, fix the network parameter weights of the discriminator , keeping the discriminator unchanged, according to the loss function of the generator Calculate gradients using backpropagation , and use the gradient descent method, according to the learning rate of the generator Update the network parameter weights of the generator , the calculation formula is: 。 7. The method according to claim 6, characterized in that Step 4 includes: Step 4.1, randomly sample a specified number of subnets from the supernet and use the perceptual similarity index IS to evaluate the performance of the subnets; Step 4.2, sort the subnets according to the evaluation results, and count the operation frequency of each candidate operation node in the top Y1 subnets; Step 4.3, discard the one operation with the lowest frequency among the upsampling candidate operations and the three operations with the lowest frequency among the common candidate operations; In the updated search space, the upsampling candidate operation has T-1 operation options, and the skip connection and the normal candidate operation have Y-3 operation options; the updated supernet includes S u subnets, where S u is a natural number, .

8. The method according to claim 7, characterized in that Step 5 includes: Step 5.1: Build a subnet set , used to maintain a dynamic list of historical optimal subnets, where represents the optimal i-th subnet in the supernet training process, for The weights inherited from the supernet, I, set the maximum capacity of the subnet set at initialization, and set the perceptual similarity index IS and Fréchet distance FID as indicators for evaluating the subnet; Step 5.2: Build a matching optimization network , which is used to calculate the matching degree between the optimal subnet and the sampling subnet. The matching optimization network includes a single fully connected layer. The input node is consistent with the image dimension of the training set, and the output node is one. The output value is expressed as the matching degree between the optimal subnet and the sampling subnet. The matching optimization network The weight parameter is .

9. The method according to claim 8, characterized in that Step 6 includes: Step 6.1, from the super network Sample a subnet , calculate the subnetwork through forward propagation Output: Step 6.2, use matching to optimize the network Compute Subnet The output of The gap between the outputs of , generates the matching degree; Step 6.3: Select the historical optimal subnet with the highest matching degree As a teacher network, and will match Return, the selection formula of the teacher network is: ， in, It is a matching optimization network The output of represents the optimal subnet With subnet The matching degree between Represents the optimal subnet weights inherited from the supernet, Indicates subnet weights inherited from the supernet, Indicates subnet The output image, Represents the optimal subnet The output image of the matching optimization network The input is the optimal subnet With subnet The difference between the outputs; Step 7 includes: Step 7.1, calculate the teacher network With subnet The architecture distillation loss between : ， in, Represents the teacher network weights inherited from the supernet, Represents the teacher network Output image, parameters Used to measure the difference between the output of the teacher generator and the sampled subnetwork generator; Step 7.2, calculate the sampling subnet The generator loss : ； Step 7.3, Subnet The weight gradient update is determined by the generator loss and architecture distillation loss The weighted average of the two is jointly supervised, where the weight of the architecture distillation loss is determined by the matching degree of the output of the matching optimization network. Dynamic adjustment, subnet The weight update formula is: ， in, represents the iteration index, Represents the subnet before weight update The weight of Represents the subnet after weight update The weight of Indicates subnet The corresponding weight of the teacher network, Indicated in The subnetwork gradient calculated at is used to guide the update direction of the subnetwork weight. Represents the learning rate of the optimization objective function, which is used to control the magnitude of each update of the subnetwork weight; In step 8, the weights of the sampled subnetworks are updated according to the architecture distillation operation Calculating the generator loss , using the generator loss as feedback, and updating the matching optimization network through back-propagation learning , the weight update formula of the matching optimization network is as follows: , in, represents the generator loss Optimize network weight parameters for matching The gradient of Indicates the weight of the subnet Calculated right The gradient of In step 9, from the super network A set of subnets are sampled from the set, and the perceptual similarity index IS and Fréchet distance FID are used as indicators to evaluate the subnet performance. If the sampled subnet performs better than the subnet set The worst performing network architecture in the network is updated, and the update formula is: ， and , in, represents the function for finding the perceptual similarity index, represents the function of finding the Fréchet distance, Indicates update and selection , that is, select the eligible As new ; In step 10, the architecture distillation loss and the generator loss are used to jointly optimize the supernet parameters so that the supernet gradually approaches the optimal performance; In step 11, the IS and FID indicators are combined to quantify the quality and diversity of the generated images, and a multi-objective optimization strategy is used to balance the image quality and the similarity of the distribution between the generated images and the real images. The subnet with the best performance in the optimal subnet set is selected as the final optimal network architecture; the generator network architecture searched from the supernet will be applied to the field of image generation, and the input random noise will be converted into a color picture through the generator.

10. A generative adversarial network architecture search system based on architecture distillation technology implemented by the method according to any one of claims 1 to 9, characterized in that: include: Construction module, used to build the generative adversarial network supernet, optimal subnet set and matching optimization network; The architecture distillation module is used to transfer the knowledge of the optimal subnet to the sampling subnet, improving the training efficiency of the supernet; The training module is used to train the network weights of the generator supernet using architecture distillation loss and adversarial loss to ensure that the final network architecture performance is optimal; The search module is used to search for the subnet architecture with the best performance in the supernet, perform multi-objective search in the optimal subnet set, and select the subnet with the best performance as the final network architecture.

Images