Adversarial network architecture search method for text-to-image generation task

Abstract

This invention discloses a generative adversarial network (GAN) architecture search method for text-guided image generation tasks. The method first trains a supernet based on a single-path weight-sharing mechanism, providing comparable performance benchmarks for different subnets in a unified adversarial evaluation environment. Then, a multi-objective evolutionary algorithm is used to perform a global search of the network architecture space, obtaining candidate high-quality subnets by constructing a Pareto front. Furthermore, a particle swarm optimization algorithm is combined to locally refine the continuous space of the elite subnets, achieving reasonable adjustment of network operation encodings through velocity updates, discretization operators, and boundary constraints. Finally, the optimal architecture obtained after multi-objective genetic algorithms significantly improves generation quality during independent retraining, achieving lower FID and higher text consistency. This invention can automatically obtain the generator structure of a GAN with low computational cost and is applicable to fields such as text-to-image synthesis and generation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision technology, and specifically relates to a generative adversarial network architecture search method for text-guided image generation tasks. Background Technology

[0002] Generative Adversarial Networks (GANs) have been widely used in tasks such as image generation and image editing. With the development of image-text alignment models, more and more studies are using text as a condition to guide the generator to output images that are semantically consistent with the text, that is, generating images that are semantically consistent with the input statement through "text-guided image generation".

[0003] Currently, generator and discriminator structures are mostly designed empirically, typically based on fixed residual networks or U-shaped structures. Repeated parameter tuning is required for different datasets and resolutions, lacking systematic structural optimization methods and making it difficult to guarantee generation quality on fine-grained datasets. To ensure semantic consistency, image-text alignment loss is often superimposed on adversarial loss. However, when the image-text alignment loss is too strong in the early stages of training, the generator tends to sacrifice low-level visual quality, satisfying semantic constraints only through speculative means, resulting in blurred image structures, obvious color blocks, and high Fréchet Inception Distance (FID). Existing GAN Neural Architecture Search (NAS) methods mostly use image quality metrics such as Inception Score (IS) and FID as optimization targets, failing to explicitly incorporate text-image semantic consistency into the search objective, leading to limited performance of the searched structures in text-guided scenarios. In typical NAS frameworks, lightweight discriminators, shorter training epochs, and fewer evaluation samples are often used in the search phase to save computation; while larger models and more stringent evaluation configurations are used in the retraining phase. The inconsistency between the loss form, regularization strategy, and text loss weights in the two stages makes it difficult to reproduce the performance of the "optimal architecture" evaluated in the search stage during retraining, and the FID may even increase significantly.

[0004] Therefore, there is an urgent need for a generative adversarial network architecture search method that simultaneously considers image quality and text semantic consistency, and maintains a consistent training environment during the search and retraining phases, especially for text-guided image generation tasks. Summary of the Invention

[0005] Purpose of the invention: To address the problems existing in text-guided image generation tasks, such as network structure dependence on manual design, unstable training under text conditions, and inconsistent environments between the architecture search phase and the retraining phase, this invention proposes a generative adversarial network architecture search method for text-guided image generation tasks.

[0006] Technical solution: A generative adversarial network architecture search method for text-guided image generation tasks, comprising the following steps:

[0007] Step 1: Construct a text conditional generative adversarial network supernet: Under the constraints of a given search space, construct a text conditional generative adversarial network containing a generator supernet and a discriminator;

[0008] Step 2, Weight-sharing training in the search phase: In the search phase, the generator supernet and discriminator are jointly trained using a weight-sharing approach. The pre-trained text-image alignment model is used to perform semantic alignment constraints on the generated images and text, resulting in image-text alignment loss.

[0009] Step 3: Architecture evaluation based on multi-objective indicators: During the search process or at the end of each generation of genetic operations, multi-objective evaluation is performed on the candidate architectures, multi-objective optimization vectors are constructed, and non-dominated sorting and crowding distance calculation are performed on the candidate architectures to obtain the Pareto optimal solution set.

[0010] Step 4: Architecture Search Based on Multi-Objective Evolutionary Algorithm: An iterative search for the architecture code is performed using a multi-objective evolutionary algorithm, including: performing a multi-objective evolutionary search for the architecture code using a non-dominated sorting genetic algorithm; selecting parent individuals from the current population based on the non-dominated sorting results and crowding distance; performing crossover and mutation operations on the parent individuals to generate offspring architecture codes; merging parent and offspring individuals and recalculating the non-dominated sorting and crowding distance to select the next generation population; repeating the iteration until the preset number of generations or performance convergence conditions are met.

[0011] Step 5: Adversarial Retraining of the Optimal Architecture: In the retraining phase, at least one target architecture is selected from the Pareto optimal solution set. Under the condition of a fixed architecture, the generator and discriminator parameters are reinitialized, and adversarial retraining is performed using a stable adversarial training strategy, including: using the adversarial loss in the form of Logistic Regression; introducing Lazy R1 gradient regularization and instance noise into the discriminator; introducing exponential moving average parameter updates into the generator; and, in the text-conditional scenario, jointly optimizing the generator by combining image-text alignment loss. Finally, the retrained text-conditional generative adversarial network model is obtained.

[0012] As a preferred embodiment, in step 1: the generator hypernet is represented as The discriminator is represented as ;in, Architecture encoding representing the combination of network topology and evolution operators; This is the noise vector; This is a text description.

[0013] use This indicates that the text feature vector is obtained through the CLIP text encoder, and the generator supernet is used to obtain the text feature vector. Control the connection methods and candidate operators between layers, including operations such as convolution, upsampling, and skip connections, and... It is incorporated into the multi-layer features as a priori condition.

[0014] As a preferred option, in each iteration of step 2:

[0015] Step 2.1: Sample an architecture code from the search space based on the current population. This yields a subnetwork.

[0016] Step 2.2, Sampling noise vector With text description The image is generated by the sub-network and input into the discriminator. Conduct competitive training;

[0017] Step 2.3: Use a pre-trained image-text alignment model to generate the image. With text description Semantic alignment constraints are applied to obtain the image-text alignment loss.

[0018] As a preferred option, in step 3, during the search process or at the end of each genetic generation, the candidate architectures are evaluated. A multi-objective evaluation should include at least: calculating the IS index based on the generated samples. Calculate the FID index based on the statistical measures of the generated samples and the real data. And using a text-image alignment model to calculate the text-image alignment loss between the generated image and the text description. .

[0019] Therefore, a multi-objective optimization vector is constructed:

[0020] ;

[0021] in, , representing the FID metric, is a measure of the quality and diversity of generated images by calculating the statistical distance between generated samples and real data; , indicates a negative value of the IS index. Since the goal of this optimization problem is to minimize, and the IS index itself is better the higher it is, a negative sign is added to transform it into a minimization problem. , representing the text-image alignment loss, is the degree of matching between the generated image and the given text description calculated using a text-image alignment model. The smaller the value, the higher the degree of alignment.

[0022] The candidate architectures are then sorted by non-dominated order and crowded distance is calculated to obtain the Pareto optimal solution set.

[0023] Furthermore, the IS index is calculated according to the following formula:

[0024] ;

[0025] in, The distribution of the generator output samples. This indicates the distribution of the generator output samples. Below, the average value of the KL divergence term; To sample The category conditional probability distribution obtained after inputting into the pre-trained classification network Inception; To generate the marginal class distribution on the samples; The Kullback-Leibler divergence is an index that maximizes the expectation of the above KL divergence, which makes the semantic distribution of the class distribution of a single generated image clear, and also makes the class distribution of the overall generated samples highly diverse.

[0026] The FID index is calculated using the following formula:

[0027] ;

[0028] in, The mean and covariance of the real data after feature extraction by Inception; The mean and covariance of the generated data after Inception feature extraction; (.) represents the operation on the trace of a matrix. For the true distribution and the generated distribution, FID measures the "distance" between these two multidimensional Gaussian distributions.

[0029] Image and text alignment loss Due to global alignment loss Local alignment loss Weighted composition is defined as:

[0030] ;

[0031] in, and These are non-negative weighting coefficients.

[0032] Furthermore, global alignment loss Defined as:

[0033] ;

[0034] in, and These are pre-trained image-text alignment models for images. and text Extracted global feature vector, For parameters The cosine similarity.

[0035] Local alignment loss Defined as:

[0036] ;

[0037] in, For the first Feature vectors of local regions of an image This is the feature vector of the text substring or word corresponding to the semantics of this local region. The number of text pairs representing regions participating in local alignment.

[0038] As a preferred embodiment, in step 4, a multi-objective evolutionary algorithm is used to iteratively search the architecture encoding. The multi-objective evolutionary algorithm includes:

[0039] Step 4.1: Select parent individuals from the current population based on the non-dominated ordination results and crowding distance;

[0040] Step 4.2: Perform crossover and mutation operations on the parent generation individuals to generate the offspring architecture code;

[0041] Step 4.3: Merge parent and offspring individuals and recalculate non-dominated sorting and crowding distance to select the next generation of the population;

[0042] Step 4.4: Repeat steps 4.1 to 4.3 until the preset number of generations or performance convergence conditions are reached.

[0043] Specifically, a non-dominated sorting genetic algorithm is used to perform multi-objective evolutionary search on the architecture encoding, as follows:

[0044] For each candidate architecture individual First, calculate its multi-objective function value. Based on dominance relationships, the population is divided into several non-dominant frontiers;

[0045] Within each non-dominated frontier, to maintain the diversity of solutions, for individuals Calculate crowd distance :

[0046] ;

[0047] in, and They were respectively in the second Individuals in each goal dimension The objective function values ​​of two adjacent individuals and The first in this generation of the population The maximum and minimum values ​​of each objective; during the environmental selection phase, individuals with higher non-dominance levels and greater crowding distances are prioritized for inclusion in the next generation.

[0048] As a preferred embodiment, in step 5, at least one target architecture is selected from the Pareto optimal solution set. Under a fixed architecture, the generator and discriminator parameters are reinitialized, and a stable adversarial training strategy is used for retraining, ultimately resulting in a retrained text conditional generative adversarial network model.

[0049] In text-conditional scenarios, to mitigate the low-level visual quality degradation caused by excessively strong image-text alignment constraints in the early stages of training, a preheating strategy is adopted for the weight coefficients of the image-text alignment loss: [The strategy is implemented in the preceding text]. The weights are increased linearly from 0 to the target weights within each training cycle. ,Right now:

[0050] ;

[0051] in, Number the current training cycle The number of warm-up cycles is used to improve training stability and visual quality of generated images while ensuring semantic consistency.

[0052] Furthermore, adversarial retraining employs Logistic adversarial loss and Lazy R1 gradient regularization, specifically including:

[0053] The discriminator loss is defined as:

[0054] ;

[0055] in, This represents the true data distribution. This represents the expected value of the actual data sample. The Softplus activation function is a smoothed ReLU function, defined as follows: In adversarial loss, it is often used to transform the output of the discriminator into a loss in the form of Logistic. For the discriminator to distinguish real images The predicted score; For the discriminator to generate the image The predicted score; The expected value for generating samples; These are the weighting coefficients for the R1 regularization term; R1 gradient regularization is a penalty term introduced to stabilize GAN training. It is usually used to constrain the gradient norm of the real data to prevent the discriminator gradient from exploding or converging too quickly, which would lead to unstable training.

[0056] The R1 gradient regularization term Defined as:

[0057] ;

[0058] in, The gradient operator represents the discriminator. For input samples The vector of partial derivatives;

[0059] The generator loss is defined as:

[0060] ;

[0061] in, The weights for the image-text alignment loss are... This represents the loss due to text-image alignment issues.

[0062] Furthermore, an exponential moving average mechanism is introduced for the generator parameters to obtain smoother and more stable generator weights, specifically including:

[0063] Let the parameters of the generator for the current training step be... The parameters of the exponential moving average are Then each update step satisfies:

[0064] ;

[0065] in, The attenuation coefficient is used when evaluating FID and IS indices. The corresponding generator serves as the evaluation model.

[0066] On the other hand, during the discriminator training phase, instance noise is introduced into both real and generated samples to improve training stability. Specifically, this includes:

[0067] Set the noise standard deviation at the beginning of training. After several training cycles, the noise decays to 0.

[0068] The input samples to the discriminator are transformed as follows:

[0069] ;

[0070] ;

[0071] in, , It follows a normal distribution with zero mean and unit covariance. The noise intensity during the current training phase. These are the original real samples and the original generated samples, which are the basic image data of the model before any perturbation is added. These are the real sample with added noise and the generated sample with added noise, respectively. They are the transformed data that are finally input into the discriminator for judgment.

[0072] Furthermore, the search phase and the retraining phase are decoupled from each other, specifically as follows:

[0073] During the search phase, to shorten the evaluation time, only a small number of generated samples are used for computation. , and And quickly estimate architecture performance within a short training period;

[0074] During the retraining phase, the selected target architecture By training from scratch, using a longer number of training epochs and more generated samples, the FID and IS can be accurately evaluated, thereby improving the generation quality and text consistency of the final model while ensuring search efficiency.

[0075] The present invention also provides: a generative adversarial network architecture search system based on multi-objective evolutionary search and text semantic alignment, for implementing the aforementioned method, comprising:

[0076] Supernet Construction Module: Constructs a text-image generative adversarial network supernet within a predefined search space, including a generator that controls the topology and candidate operators with architecture encoding, and a discriminator for distinguishing between real and generated images;

[0077] Search training module: During the search phase, the generator supernet and discriminator are jointly trained using a weight-sharing method to generate generated samples corresponding to different architecture sub-networks;

[0078] Image-text alignment evaluation module: Calls a pre-trained image-text alignment model to calculate the similarity between the generated image and the text description, and obtains the image-text alignment loss;

[0079] Multi-objective evaluation and evolution module: Calculates the FID index, IS index and image-text alignment loss of each candidate architecture, and iteratively searches the architecture encoding based on the multi-objective evolution algorithm of non-dominated ranking and crowding distance to obtain the Pareto optimal architecture set.

[0080] Retraining module: Select a target architecture from the Pareto optimal architecture set, retrain the target architecture from scratch based on Logistic adversarial loss, Lazy R1 gradient regularization, instance noise and image-text alignment loss, and obtain the final generator weights through the exponential moving average mechanism;

[0081] Output module: Outputs the retrained text conditional generative adversarial network model and the corresponding FID and IS metrics.

[0082] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0083] 1. This invention addresses the text-guided image generation task by constructing a supernet search space that integrates generator convolutional units, upsampling paths, and text-image semantic alignment constraints. It introduces a multi-objective fitness function composed of FID, IS, and CLIP text-image similarity and combines multi-objective evolutionary search and local fine search to automatically explore the text conditional generator architecture within a larger structural space, avoiding local optima caused by manual and experience-based selection.

[0084] 2. This invention evaluates candidate structures individually by sharing supernet weights and employs a consistent discriminator structure and regularization strategy during the search and retraining phases. Combined with instance noise, R1 gradient penalty, and other stable training techniques, it reduces performance degradation caused by inconsistencies in the search-retraining environment, and improves the reproducibility and stability of the results. While ensuring the generation quality indicators such as FID, compared with traditional NAS-GAN methods that only target FID / IS, this invention can significantly improve the semantic consistency between text description and generated image. Attached Figure Description

[0085] Figure 1 This is a schematic diagram of the overall process of the text-guided image generation task of the present invention;

[0086] Figure 2 This is a schematic diagram of the generator supernet and its genotype integer encoding in this invention;

[0087] Figure 3 This is a schematic diagram of the text condition and CLIP semantic alignment module of the present invention;

[0088] Figure 4 This is a flowchart illustrating the architecture of the present invention, which combines the multi-objective genetic algorithm (NSGA-II) with elite local search.

[0089] Figure 5 This is a graph showing the changes in FID and IS indices of a representative architecture during the retraining phase of the search phase of this invention as a function of the training process. Detailed Implementation

[0090] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0091] Example 1: A generative adversarial network architecture search method for text-guided image generation is proposed.

[0092] like Figure 1 As shown in this embodiment, the generative adversarial network architecture search method for text-guided image generation mainly includes the following steps:

[0093] Step 1: Data and text preprocessing.

[0094] The training images and their corresponding text descriptions are obtained from the target task to form image-text pairs. Image samples are normalized in size and pixels, and text descriptions are preprocessed such as word segmentation and case normalization. Training and validation sets are then constructed.

[0095] Step 2: Construct a text conditional generation adversarial supernet.

[0096] A generator supernetwork was built on the existing NAS-GAN framework. With discriminator ,like Figure 2 As shown; where, These are the network structure parameters to be searched.

[0097] Generator hypernetwork structure such as Figure 2 As shown in (a), the bottom layer uses a random noise vector. As input, the feature map resolution is progressively increased through a series of bottom-up cascaded searchable units (upsampling), represented as Block1 to BlockL, to finally output a synthesized image. .

[0098] To incorporate textual conditions, a text modulation channel is reserved within the generator to convert the text vector obtained through the CLIP text encoder. As conditional information, it is injected into several searchable units using methods such as conditional normalization and feature concatenation, so that the generated features are simultaneously modulated by noise vectors and text semantics.

[0099] like Figure 2 As shown in (b) of the diagram, this embodiment uses discrete integer genotypes to encode the generator architecture. Wherein, the symbol... This represents the base number of channels in the convolutional layer. As the generator network depth increases, the number of channels in each search unit follows a preset doubling or reduction rule to control computational complexity while maintaining feature representation capability. (Symbol) The specific values ​​can be flexibly configured based on the target image resolution and computing resources. The specific internal structure of each search unit (Cell) is uniquely determined by its architecture encoding, i.e., its genotype. The genotype is a fixed-length integer vector, where each bit corresponds to a specific operator in the search space. For the ... Each block is represented by a vector: operator type kernel size Number of input channels Number of output channels And whether to receive text conditional flags Structural parameters, discriminator Using a fixed structure, for real images With synthetic images The adversarial scores are output separately, which are used to construct the adversarial loss of the GAN and provide performance evaluation for subsequent architecture search.

[0100] Step 3: Text encoding and CLIP semantic alignment constraints.

[0101] like Figure 3 As shown, this embodiment uses a pre-trained CLIP model as a common encoder for text and images.

[0102] Text description Text encoder Mapped to text vector Real images With synthetic images Vision encoder Mapped to image vectors respectively , .

[0103] Define text-image cosine similarity in a shared embedding space. :

[0104] ;

[0105] And construct semantic alignment loss :

[0106] ;

[0107] During training, the CLIP model parameters are kept constant, except for the generator. The loss is backpropagated to constrain the generated image by the semantics of the text.

[0108] Step 4: Adversarial training and multi-objective fitness assessment during the search phase.

[0109] Under the premise of weight sharing in supernets, for a given architecture parameter The subnetwork undergoes several steps of adversarial training. In this embodiment, taking the hinge form as an example, the discriminator loss... Defined as:

[0110] ;

[0111] in, For the discriminator to distinguish real images The predicted score; , This is the expected value.

[0112] Generators against loss for:

[0113] ;

[0114] Combining CLIP semantic alignment constraints, the generator's total loss is:

[0115] ;

[0116] in, >0 represents a weight coefficient. After several rounds of training, a batch of synthetic samples is generated from this sub-network, and its FID, IS, and text-image relevance metrics are calculated to measure generation quality, diversity, and semantic consistency. Let the _th_... The three metrics for each candidate architecture are as follows: , , Then construct a multi-objective fitness vector. Minimize simultaneously in a multi-objective context ,maximize and .

[0117] Step 5: Multi-objective evolutionary search and elite local search.

[0118] like Figure 4 As shown, this embodiment uses a non-dominated sorting multi-objective genetic algorithm (NSGA-II) for architecture encoding. A global search is performed. First, a number of architecture individuals are randomly initialized to form an initial population. Step 4 is performed on each individual to obtain a fitness vector. Then, Pareto front individuals are selected using non-dominated sorting and crowding distance, and the next generation of candidate architectures are generated through crossover, mutation, and other operations.

[0119] To further refine the optimal architectures, several elite individuals are selected in each generation, and local search operators such as particle swarm optimization are used to continuously or discretely update key structural parameters in their neighborhoods, resulting in new candidate individuals and re-evaluating their fitness. This process is repeated until a predetermined number of generations or fitness convergence is reached, thus obtaining a set of optimal architectures that satisfy different preference trade-offs.

[0120] Step 6: Retraining after fixing the optimal architecture.

[0121] The target generator architecture is selected based on a comprehensive ranking of FID, IS, and text relevance from the Pareto front. During the retraining phase, a fixed structure is used. The generator and discriminator parameters are then re-initialized randomly, and the entire training is performed on the target dataset. During training, stabilization strategies such as discriminator gradient regularization and instance noise can be further introduced to ultimately obtain the optimal generative adversarial network model for text-guided image generation tasks.

[0122] Example 2: Application of the method of the present invention on the CUB text-image dataset.

[0123] This embodiment demonstrates the specific application of the method of the present invention on the Caltech-UCSD Birds 200-2011 (CUB) image-text dataset, which is used to verify the effectiveness of the present invention in fine-grained text-guided image generation tasks.

[0124] Step 1: Prepare the CUB dataset.

[0125] The CUB dataset contains approximately 12,000 bird images, each accompanied by multiple natural language descriptions. This embodiment selects a training subset as the training set for search and retraining, and a validation subset for performance evaluation. First, all images are center-cropped and scaled to unify the resolution. The pixels are normalized; for each image with multiple text descriptions, one is randomly sampled during each training session as the text conditional input for the current image.

[0126] Step 2: Search phase configuration.

[0127] On the CUB dataset, construct as follows Figure 2 The text condition generator, hypernet, and discriminator are shown. The size of the hypernet's bottom-layer feature map is set to... Noise vector dimension Parameters such as the number of candidate convolution channels, kernel size, and upsampling connection method are represented by architecture encoding.

[0128] During the search phase, a medium-sized population and a number of generations are used. For each candidate architecture, several rounds of weight-sharing training are performed according to step 4, and a fixed number of sample images are generated for metric evaluation.

[0129] The FID metric uses feature statistics pre-calculated based on the CUB training set, while the IS metric and text-image similarity metric are calculated based on the pre-trained Inception network and CLIP model, respectively. Through multi-objective evolution and elite local search, a set of Pareto-optimal generator architectures with different trade-offs among FID, IS, and semantic alignment capabilities are finally obtained.

[0130] Step 3: Optimal architecture selection and retraining.

[0131] From the Pareto fronts obtained in step 2, select one or more architectures that perform well in terms of FID and text relevance metrics as the target architecture. .

[0132] During the retraining phase, fix Without changing anything, reinitialize the generator and discriminator parameters, and perform end-to-end training on the CUB dataset using uniform optimized hyperparameters.

[0133] Adversarial losses can be expressed in either the hinge form or the logistic form, as follows: :

[0134] ;

[0135] ;

[0136] Regularization terms such as R1 gradient penalty and instance noise can be used to improve training stability. The generator's total loss still includes adversarial terms. semantic alignment with CLIP :

[0137] .

[0138] Step 4: Performance evaluation and sample generation analysis.

[0139] The performance verification experiments of this embodiment were conducted on the CIFAR-10 Natural Image Dataset and the CUB Bird Image-Text Dataset to evaluate the effectiveness of the text-guided architecture search method in different scenarios.

[0140] Record the FID and IS curves of representative architectures during the search phase as a function of training epochs during the retraining phase, such as... Figure 5 As shown in (a) and (b), the FID index shows an overall downward trend, while the IS index shows an overall upward trend, indicating that the search process can steadily improve the generation quality and sample diversity under the guidance of the multi-objective fitness function. Furthermore, the method of this invention gradually converges on the CUB dataset, with continuously improving generation quality and diversity. For a given bird text description, the generator obtained and retrained using this invention can generate bird images with clear outlines and fine textures, whose colors, poses, and backgrounds are highly consistent with the text description; compared with manually designed architectures, it has significant advantages in both visual realism and semantic consistency.

[0141] Based on quantitative results such as FID, IS, and text-image similarity, this embodiment demonstrates that the generative adversarial network architecture search method for text-guided image generation tasks proposed in this invention can significantly improve the quality of fine-grained image generation and outperform existing NAS-GAN and traditional hand-designed architectures in terms of semantic alignment capability.

[0142] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A generative adversarial network architecture search method for text-guided image generation tasks, characterized in that, Includes the following steps: Step 1: Under the constraints of a given search space, construct a text conditional generative adversarial network containing a generator supernet and a discriminator; Step 2: In the search phase, the generator supernet and discriminator are jointly trained using a weight-sharing approach. The pre-trained text-image alignment model is used to perform semantic alignment constraints on the generated images and text, resulting in image-text alignment loss. Step 3: During the search process or at the end of each generation of genetic operations, perform multi-objective evaluation on the candidate architecture, construct multi-objective optimization vectors, and perform non-dominated sorting and crowding distance calculation on the candidate architecture to obtain the Pareto optimal solution set. Step 4: Iteratively search the architecture code using a multi-objective evolutionary algorithm, including: performing a multi-objective evolutionary search on the architecture code using a non-dominated sorting genetic algorithm; selecting parent individuals from the current population based on the non-dominated sorting results and crowding distance; performing crossover and mutation operations on the parent individuals to generate offspring architecture codes; merging parent and offspring individuals, and recalculating the non-dominated sorting and crowding distance to select the next generation population; repeating the iteration until the preset number of generations or performance convergence conditions are met. Step 5: In the retraining phase, select at least one target architecture from the Pareto optimal solution set, reinitialize the generator and discriminator parameters under the condition of fixed architecture, and perform adversarial retraining, including: adopting the adversarial loss in the form of Logistic, introducing Lazy R1 gradient regularization and instance noise to the discriminator; introducing exponential moving average parameter updates to the generator; and jointly optimizing the generator in the text-conditional scenario by combining image-text alignment loss to obtain the retrained text-conditional generative adversarial network model.

2. The generative adversarial network architecture search method according to claim 1, characterized in that, In step 1, the generator supernet is represented as follows: The discriminator is represented as ;in, Architecture encoding representing the combination of network topology and evolution operators; This is the noise vector; For text description; The generator supernet is encoded via architecture. Controlling the connection methods and candidate operators between layers, including convolution, upsampling, and skip connections, and converting the text feature vectors obtained through the CLIP text encoder... It is incorporated into the multi-layer features as a priori condition.

3. The generative adversarial network architecture search method according to claim 2, characterized in that, In step 2, each iteration method includes: Step 2.1: Based on the current population, sample an architecture code from the search space. This yields a subnetwork. Step 2.2: Sample noise and text, generate an image through the sub-network, and input it into the discriminator for adversarial training; Step 2.3: Use the pre-trained text-image alignment model to perform semantic alignment constraints on the generated images and text, and obtain the image-text alignment loss.

4. The generative adversarial network architecture search method according to claim 3, characterized in that, In step 3, the multi-objective evaluation includes: calculating the IS index based on the generated samples. Calculate the FID index based on the statistical measures of the generated samples and the real data. The text-image alignment model is used to calculate the image-text alignment loss between the generated image and the text description. ; Construct the multi-objective optimization vector , is represented as: ； in, , representing the FID indicator; , indicating a negative value of the IS index; , indicating the loss of alignment between text and image.

5. The generative adversarial network architecture search method according to claim 4, characterized in that, The IS index is calculated by exponentially taking the expected value: ； in, The distribution of the generator's output samples; This indicates the distribution of the generator output samples. The average value of the lower KL divergence term; To sample The category conditional probability distribution obtained after inputting into the pre-trained classification network Inception; To generate the marginal class distribution on the samples; Let KL divergence be a metric. The function represents an exponential function; The FID metric is calculated by measuring the distance between the true distribution and the generated distribution: ； in, The mean and covariance of the real data after feature extraction by Inception; The mean and covariance of the generated data after Inception feature extraction; (.) represents the matrix trace operation; For the true distribution and the generated distribution; The image-text alignment loss Due to global alignment loss Local alignment loss Weighted composition is defined as: ； in, and These are non-negative weighting coefficients.

6. The generative adversarial network architecture search method according to claim 4, characterized in that, In step 4, a multi-objective evolutionary search is performed on the architecture encoding using a non-dominated sorting genetic algorithm. The method includes: For each candidate architecture individual Calculate its multi-objective function value Based on dominance relationships, the population is divided into several non-dominant frontiers; Within each non-dominated frontier, to maintain the diversity of solutions, for individuals Calculate crowd distance : ； in, and They were respectively in the second Individuals in each goal dimension The objective function values ​​of two adjacent individuals and The first in this generation of the population The maximum and minimum values ​​of each objective; during the environmental selection phase, individuals with high non-dominance levels and large crowding distances are given priority to be retained for the next generation.

7. The generative adversarial network architecture search method according to claim 4, characterized in that, In step 5, an adversarial loss in the form of Logistic regression is used to introduce Lazy R1 gradient regularization and instance noise into the discriminator, specifically including: Define discriminator loss : ； in, This represents the true data distribution. This represents the expected value of the actual data sample. For activation functions; For the discriminator to distinguish real images The predicted score; For the discriminator to generate the image The predicted score; The expected value for generating samples; These are the weighting coefficients for the R1 regularization term; This is the R1 gradient regularization term; Define generator loss : ； in, The weights for the image-text alignment loss are... This is due to the loss in image-text alignment. To improve training stability, instance noise is introduced into both real and generated samples. A noise standard deviation is set at the beginning of training, and after several training cycles, the noise decays to zero. The input samples to the discriminator are then transformed as follows: ； ； in, , It follows a normal distribution with zero mean and unit covariance. The noise intensity during the current training phase. These are the original real samples and the original generated samples, respectively. These are real samples with added noise and generated samples with added noise, respectively.

8. The generative adversarial network architecture search method according to claim 7, characterized in that, In step 5, an exponential moving average mechanism is introduced for the generator parameters, setting the generator parameters for the current training step to... The parameters of the exponential moving average are Then each update step satisfies: ； in, The attenuation coefficient is used when evaluating the FID and IS indices. The corresponding generator serves as the evaluation model.

9. The generative adversarial network architecture search method according to claim 7, characterized in that, The search phase and the retraining phase are decoupled from each other, specifically: During the search phase, calculation , and The number of generated samples used in the indexing phase is less than the number of generated samples used in the retraining phase; and the number of iterations for adversarial training of each candidate architecture in the search phase is less than the number of iterations for adversarial retraining in the retraining phase. During the retraining phase, the selected target architecture is encoded. Perform training from scratch, using a longer number of training epochs and a larger number of generated samples than the search phase. , The evaluation of metrics aims to improve the quality of the final model generation and text consistency while ensuring search efficiency.

10. A generative adversarial network architecture search system based on multi-objective evolutionary search and text semantic alignment, used to implement the method according to any one of claims 1 to 9, characterized in that, include: Supernet Construction Module: Constructs a text-image generative adversarial network supernet within a predefined search space, including a generator that controls the topology and candidate operators with architecture encoding, and a discriminator for distinguishing between real and generated images; Search training module: During the search phase, the generator supernet and discriminator are jointly trained using a weight-sharing method to generate generated samples corresponding to different architecture sub-networks; Image-text alignment evaluation module: Calls a pre-trained image-text alignment model to calculate the similarity between the generated image and the text description, and obtains the image-text alignment loss; Multi-objective evaluation and evolution module: Calculates the FID index, IS index and image-text alignment loss of each candidate architecture, and iteratively searches the architecture encoding based on the multi-objective evolution algorithm of non-dominated ranking and crowding distance to obtain the Pareto optimal architecture set. Retraining module: Select a target architecture from the Pareto optimal architecture set, retrain the target architecture from scratch based on Logistic adversarial loss, Lazy R1 gradient regularization, instance noise and image-text alignment loss, and obtain the final generator weights through the exponential moving average mechanism; Output module: Outputs the retrained text conditional generative adversarial network model and the corresponding FID and IS metrics.

Images