Conditional stylegan-based small sample image generation method

Abstract

The application discloses a kind of small sample image generation methods based on conditional StyleGAN, comprising: 1) the image of data set is preprocessed according to category;2) using visible category pretraining conditional StyleGAN;3) using k invisible class image optimization conditional StyleGAN, its optimization process includes two stages: latent space positioning stage and latent space fine-tuning stage, wherein latent space positioning stage uses invisible class image subset as anchor point, and locates the subspace of invisible class, and latent space fine-tuning stage further aligns the generation distribution of conditional generator and the real distribution of invisible class by antagonistic loss and anchor point stability loss;4) generate the picture of invisible class by random sampling noise.The application has the characteristics of deep learning technology, and through a series of optimization objectives, the generator can be optimized by k invisible class reference pictures, and has the generation ability of invisible class images.

Description

Technical Field

[0001] This invention relates to the field of image generation technology, and in particular to a few-sample image generation method based on Conditional StyleGAN. Background Technology

[0002] With the continuous development of deep learning technology, the application scope of generative adversarial networks (GANs) has become increasingly wide, and the generation effect and application scenarios have been greatly improved. However, training GANs often relies on a large amount of training data, consuming a significant amount of training time and cost. In many real-world scenarios, we cannot obtain large amounts of training data. For example, images of rare and protected animals are often difficult to obtain, and the image quality is often poor; due to privacy protection, images of some pathological tissues are often difficult to obtain; and for some deceased celebrities, the image data they left behind is very limited. Traditional image generation models based on large datasets have many limitations in these scenarios and cannot be widely promoted and applied. Therefore, image generation with small sample sizes is a problem that urgently needs to be solved.

[0003] Existing few-shot image generation methods can be mainly divided into three categories: 1) Optimization-based methods: These methods employ the idea of ​​meta-learning. During pre-training, a large number of few-shot image generation tasks are constructed to find a set of generalizable initialization parameters for the generator and discriminator. During the testing phase, the original generator and discriminator are fine-tuned using a few images of invisible categories. 2) Fusion-based methods: During pre-training, the generator is trained to learn an image fusion strategy to fuse input images into a new image. Given images of invisible categories, the fusion strategy is used to synthesize a new image. 3) Transformation-based methods: During pre-training, the generator is trained to learn category-independent transformation methods. When an image needs to be generated, category-independent operations are used to perform category-independent transformations on the input images of invisible categories, thereby generating a new image.

[0004] Existing few-sample image generation methods have limited quality, specifically: 1) Numerous generation defects, resulting in many false generated images that differ significantly in quality from real images. 2) Inaccurate semantic categories, with visual discrepancies between the generated image's category and the unseen category. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies and propose a few-shot image generation method based on Conditional StyleGAN. This method solves the generation defects caused by the large difference between invisible categories and visible categories used for pre-training in traditional few-shot image generation methods. This invention fully utilizes the continuity and semantic decoupling of the latent space of the pre-trained Conditional StyleGAN, and uses images of a few invisible categories to fine-tune the pre-trained Conditional StyleGAN. This not only effectively utilizes prior knowledge to perform basic modeling of the distribution, but also makes full use of the small number of images containing very low information content from the invisible categories.

[0006] To achieve the above objectives, the technical solution provided by this invention is as follows: a few-shot image generation method based on Conditional StyleGAN. This method proposes a two-stage optimization method suitable for few-shot scenarios based on Conditional StyleGAN. Through joint optimization of the proposed anchor loss and adversarial loss, and with the assistance of semantic regularization, it effectively avoids the overfitting phenomenon of neural networks due to insufficient sample size, and realizes image generation in few-shot scenarios. It only requires a very small number of images of the given invisible category as references to generate high-quality and diverse images for that invisible category. The steps include:

[0007] 1) Data Acquisition:

[0008] Divide the given dataset X into a set of visible categories C in the category space. s and the set of invisible categories C u Subsequently, based on the division of the category set, the images in the given dataset X are divided into a visible category image set X. s and the set of invisible category images X u The objective is: given a certain invisible category c u a subset of k images For the invisible category c u Generate diverse and high-quality images, where c u ∈C u k≤5, Represents a subset of images of the invisible category. The image in the image, where i represents the index of the image subset;

[0009] 2) Pre-trained StyleGAN:

[0010] Using the visible category image set X s Pre-training a conditional StyleGAN yields a pre-trained conditional generator G. s Conditional discriminator D s ;

[0011] 3) Optimize Conditional StyleGAN using Invisible Class Images:

[0012] Using a subset of images of a given invisible category For G obtained in step 2), s and D s Fine-tuning is performed to obtain the optimized condition generator G. u Conditional discriminator D u This includes two phases: the latent space localization phase and the latent space fine-tuning phase.

[0013] The latent space localization stage uses a subset X of k invisible category images. cu The image in the image serves as the anchor point, and the pre-trained conditional generator G... s Locating a given invisible category c in the latent space u The subspace of c is obtained. u The corresponding invisible category feature vector f u To obtain c u The initial data distribution, and optimization of G s Mapping network The optimized condition generator G is obtained u ;

[0014] The latent space fine-tuning stage is based on the invisible category c obtained from the latent space localization stage. u The initial data distribution is used to further fine-tune the condition generator G obtained during the latent space localization stage. u This makes G u Fully learn the invisible category c u Features of the alignment condition generator G u Distribution of generated images and invisible category c u The image distribution is optimized, while further optimizing the invisible category c. u Invisible category feature vector f u To make it compatible with G u In fine-tuning G u During the process, the pre-trained conditional discriminator D is expanded. s D u And at the same time for D u Make fine adjustments;

[0015] Both the latent space localization stage and the latent space fine-tuning stage use semantic regularization terms to ensure the stability of the optimization process;

[0016] 4) Randomly sample noise to generate an image:

[0017] Randomly sample Gaussian noise z, and compare it with the invisible category c obtained in step 3). u Category feature vector f uInput together the optimized condition generator G obtained in step 3) u Generate the invisible category c u The image.

[0018] Furthermore, in step 1), the given dataset X is divided into a set of visible categories in the category space. and invisible category set in and N represents the visible category and the invisible category, respectively. s and N u These represent the number of visible categories and the number of invisible categories, respectively; then, based on the category division, the images in the given dataset X are divided into a set of visible category images X. s and the set of invisible category images X u Visible category image set X s Includes all corresponding categories c belonging to the set of visible categories C. s Image x s Each visible category contains n images, and the invisible category contains a set X of images. u This includes all corresponding categories c belonging to the set of invisible categories C. u Image x u .

[0019] Furthermore, in step 2), the set of visible category images X is used. s Pre-train the conditional StyleGAN to obtain the conditional generator G. s Conditional Discriminator D s During pre-training, the condition generator G s Conditional discriminator D s Iterate and update alternately until convergence.

[0020] Furthermore, in step 3), the latent space localization stage uses the anchor point approximation loss L. app and anchor point strength constraint L mgt Two optimization objectives for the invisible class feature vector f u Trainable anchor noise set Φ opt and pre-trained condition generator G s Mapping network Optimize, a i The noise is the trainable anchor point, where the condition generator in the optimization process is denoted as G. u The latent space localization stage includes the following steps:

[0021] 1) Obtain the target anchor point set Θ, where, Target anchor point w iSubset of invisible category images Invisible category images Using the pre-trained condition generator G s The inverse mapping I(·,·) yields:

[0022]

[0023] 2) Randomly initialize the invisible category feature vector f u and the set of trainable anchor noise Φ opt ;

[0024] 3) Obtain the set of trainable anchor points Among them, trainable anchor points From the invisible category feature vector f u Trainable anchor noise set Φ opt Through condition generator G u Mapping network Mapping to the latent space yields:

[0025]

[0026] 4) Use anchor points to approximate the loss L app and anchor point strength constraint L mgt Optimize:

[0027] The anchor point approximation loss L app Constrain k trainable anchor points and target anchor point w i The L2 loss aims to obtain the feature vector f of the invisible class. u Its definition is as follows:

[0028]

[0029] The anchor point strength constraint L mgt The goal is to limit the set of trainable anchor noise Φ opt Trainable anchor noise a i The size of a prevents a i Overfitting is defined as follows:

[0030]

[0031] Target loss L during latent space localization phase loc Defined as:

[0032] L loc =L app +L mgt

[0033] During the training process of the latent space localization phase, the condition generator G...u Mapping network Invisible category feature vector f u and trainable anchor noise a i Alternating optimization; where the condition generator G u Mapping network The learning rate used is the invisible class feature vector f u and the set of trainable anchor noise Φ opt The learning rate used is λ times the learning rate, where λ ≤ 1.

[0034] Furthermore, in step 3), the latent space fine-tuning stage uses adversarial loss L. adv And anchor point stability loss L stb The invisible category feature vector f obtained during the latent space localization stage u Trainable anchor noise set Φ opt Condition generator G u Conditional discriminator D s Optimizations were performed, including:

[0035] Condition discriminator D u Embedded network D embed The input dimension is expanded from n dimensions to n+1 dimensions to support additional invisible categories c. u The extended condition discriminator is denoted as D. u ;

[0036] The resistance loss L adv The goal is to align the generated image distribution with the invisible category c. u The distribution of real images is defined as follows:

[0037]

[0038] In the formula, E represents the expected value. Represents a subset of images of the invisible category. The image in the image, y u Indicates the invisible category c u The category label, z, indicates that it is from P. Z Gaussian noise from random sampling, P Z G represents a standard Gaussian distribution. u and D u These represent the optimized condition generator and condition discriminator, respectively.

[0039] The anchor point stability loss L stb The goal is to create a stable condition generator G. u The semantic information of the generated distribution is defined as follows:

[0040] L stb =Lperc +L mgt

[0041] In the formula, L perc Indicates perceived loss. Represents the invisible category feature vector f u and trainable anchor noise a i Through condition generator G u The obtained image is L mgt Indicates anchor point strength constraint;

[0042] The target loss L during the latent space fine-tuning stage ref Defined as:

[0043] L ref =L adv +L stb

[0044] Condition generator G u Conditional discriminator D u Alternating optimization, wherein the anchor point stability loss L stb Acting on G u The optimization phase.

[0045] Furthermore, in step 3), the regularization terms that maintain semantics include the regularization terms from the latent space localization stage. Regularization term in the latent space fine-tuning phase

[0046] Conditional generator mapping network for regularization term constraint optimization in the latent space localization stage Mapping network with pre-trained conditional generator The consistency between them is defined as follows:

[0047]

[0048] In the formula, z represents the Gaussian noise of the same random sample, and f s Indicates the visible category c s eigenvectors;

[0049] For the regularization term in the latent space fine-tuning stage, respectively constrain the generator G. u With G s Convolutional modules of the inter- and conditional discriminators and The consistency between them is defined as follows:

[0050]

[0051] In the formula, G represents the pre-trained condition generator.s Using randomly sampled Gaussian noise z and visible category feature vector f s The generated image, i.e. L perc (G u (z,f s ),G s (z,f s Regularization condition generator G u Condition generator G for constrained optimization u and pre-trained condition generator G s Visual discrepancies between generated images; Convolutional module of a conditional discriminator for constraint optimization and the convolutional module of the pre-trained conditional discriminator For the same generated image Consistency of output convolutional features;

[0052] The total loss L1 during the latent space localization stage is:

[0053]

[0054] The total loss L2 during the latent space fine-tuning stage is:

[0055]

[0056] Furthermore, in step 4), an invisible category image is generated by randomly sampling Gaussian noise. The generation process is as follows:

[0057] Randomly sample Gaussian noise z, and compare it with the invisible category feature vector f obtained in step 3). u Input together the optimized condition generator G obtained in step 3) u Generate invisible category c u Image

[0058]

[0059] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0060] 1. Image generation in low-sample scenarios is achieved through deep learning methods. Only a very small number of reference samples of the same category are needed to generate diverse and high-quality images for that category.

[0061] 2. It effectively combines prior knowledge and posterior information, enabling efficient learning of category features from a small number of images, thus solving the problem of neural network overfitting caused by low data volume.

[0062] 3. It retains the editable latent space property of Conditional StyleGAN, enabling various downstream tasks of small sample image generation, including image editing.

[0063] In summary, this invention possesses the characteristics of deep learning technology. Through a series of optimization objectives, the generator can generate images of invisible categories by optimizing k invisible category reference images. It has practical application value and is worth promoting. Attached Figure Description

[0064] Figure 1 This is a flowchart of the method of the present invention.

[0065] Figure 2 This is a schematic diagram of the two-stage optimization process of the present invention. Detailed Implementation

[0066] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0067] This embodiment discloses a few-shot image generation method based on Conditional StyleGAN. This method proposes a two-stage optimization approach suitable for few-shot scenarios based on Conditional StyleGAN. Through joint optimization of the proposed anchor loss and adversarial loss, and with the assistance of semantic regularization, it effectively avoids overfitting of the neural network due to insufficient sample size, achieving image generation in few-shot scenarios. It only requires a very small number of images of a given invisible category as reference to generate high-quality and diverse images for that invisible category; for example... Figure 1 As shown, this method involves data acquisition on an animal dataset, pre-training a Conditional StyleGAN, optimizing the Conditional StyleGAN using images of invisible categories, and generating images using randomly sampled noise. The steps include:

[0068] First, the dataset X is divided into a set of visible categories in the category space. and invisible category set Based on category classification, the images in the given dataset X are divided into a set of visible category images X. s and the set of invisible category images X u Given a dataset X with a total number of classes N c =N s +N u , where N s and N u These represent the number of visible categories and the number of invisible categories, respectively. Each category contains n images. In the wildlife dataset, N... c =149, N s =119, N u=30, n=100. The goal of this framework is: given an invisible category c u (c u ∈C u A subset of k (k≤5) images For the invisible category c u Generate diverse and high-quality images. Represents a subset of images of the invisible category. The image in the image.

[0069] Secondly, using the wildlife visible category image set X s Pre-trained conditional StyleGAN. Using a set of visible category images X. s All images are used to pre-train the Conditional StyleGAN. During pre-training, the conditional generator and conditional discriminator are iteratively updated alternately until training converges, resulting in the conditional generator G. s Conditional discriminator D s Among them, the condition generator G s Able to generate N s Images of visible categories.

[0070] Furthermore, k images of wildlife of invisible categories are used to train the pre-trained conditional generator G. s Conditional discriminator D s Fine-tuning is performed. The two-stage optimization is as follows: Figure 2 As shown.

[0071] In the localization phase of the latent space, this is achieved by using k (k≤5) invisible category images. The invisible category of image subset Locate the invisible category c u The location in the latent space. Specifically, the loss L is approximated using anchor points. app and anchor point strength constraint L mgt Two optimization objectives to find the invisible category c u The location in the hidden space. In this example implementation, k = 3, and the lion is selected as the invisible category c. u Image subset k samples in Figure 2 As shown.

[0072] Using anchor points to approximate the loss L app Constrained set of trainable anchor points and target anchor set The L2 loss aims to optimize the acquisition of the invisible category feature vector f. u Its definition is as follows:

[0073]

[0074] In the formula, the trainable anchor point From the invisible category feature vector f u and trainable anchor noise a i Through condition generator G u Mapping network The obtained hidden code Target anchor point w i Indicates invisible category images Using the pre-trained condition generator G s The hidden code obtained by the inverse mapping I(·,·)

[0075] The anchor point strength constraint L mgt The goal is to limit the noise of trainable anchor points. i The size of a is used to prevent optimization during the process. i Overfitting is defined as follows:

[0076]

[0077] During the latent space localization phase, the condition generator G is optimized simultaneously. u Mapping network To prevent interference from anomalous sample points, and to ensure the effectiveness of the semantic direction in the latent space of Conditional StyleGAN, the regularization term of the localization method for the latent space of Conditional StyleGAN is used:

[0078]

[0079] For the optimized condition generator G u Regularization is performed. In the formula, z represents the Gaussian noise from the same random sample, and f... s Indicates the visible category c s eigenvectors.

[0080] By locating the latent space, the optimized condition generator G was obtained. u Invisible category feature vector f u and trainable anchor noise a i The noise z and the invisible category feature vector f are obtained through random sampling. u The optimized condition generator G u It can generate invisible category c u The image was generated, but it only contained features from the pre-trained domain and failed to capture the invisible category c. u The unique characteristics lead to the condition generator G u The generation distribution and invisible category c uThe actual distribution is inconsistent.

[0081] During the fine-tuning phase of the latent space, in order to make the condition generator G... u The generated distribution is aligned with the invisible category c u The true distribution, using adversarial loss L adv And anchor point stability loss L stb Optimize for two objectives.

[0082] Using adversarial loss L adv :

[0083]

[0084] In the formula, E represents the expected value. Represents a subset of images of the invisible category. The image in the image, y u Indicates the invisible category c u The category label, z, indicates that it is from P. Z Gaussian noise from random sampling, P Z G represents a standard Gaussian distribution. u and D u Let D represent the optimized condition generator and condition discriminator, respectively. u With images and sparse category labels y u As input, the category label y u It needs to be conditionally discriminated by D first. u Embedded network D embed This is mapped to a dense set of category feature vectors. This is to support the invisible category c. u The training will embed the network D embed The input dimension has been reduced from the original N s Expand to N s +1. For the animal dataset example, the input dimension is expanded from 114 dimensions to 114 + 1 = 115 dimensions. Specifically, since the dense feature vector dimension is 512, the pre-trained discriminator is embedded into the network D. embed The sparse class labels are mapped to class feature vectors using a matrix W of dimensions [114, 512]. For invisible classes, an additional [1, 512] matrix is ​​concatenated onto the original matrix W to expand it into a matrix of dimensions [115, 512].

[0085] Using anchor point stabilization loss L stb The goal is to create a stable condition generator G. u The semantics of the generated distribution are defined as follows:

[0086] L stb =L perc+L mgt

[0087] In the formula, L perc Indicates perceived loss Represents the invisible category feature vector f u and trainable anchor noise a i Through condition generator G u The obtained image is L mgt This indicates the anchor point strength constraint.

[0088] To maintain the validity of the Conditional StyleGAN semantic space, a regularization term is used for the fine-tuning method of the Conditional StyleGAN latent space: To constrain condition generator G u Conditional Discriminator D u . G represents the pre-trained condition generator. s The noise z and the visible category feature vector f are obtained through random sampling. s The generated image (i.e.) ), where L perc (G u (z,f s ),G s (z,f s Regularization condition generator G u Condition generator G for constrained optimization u and pre-trained condition generator G s The visual difference between generated images, where Convolutional module of a conditional discriminator for constraint optimization and the convolutional module of the pre-trained conditional discriminator For the same generated image Consistency of output convolutional features. In the animal dataset, this is achieved by the convolutional module of the discriminator. The resulting convolutional feature range is relatively large, therefore, in terms of... and Before applying L2 loss constraints, max-min normalization is used to constrain the range of convolutional features to [0,1].

[0089] Finally, randomized Gaussian noise z and invisible category feature vector f are used. u To generate invisible category c u Image

[0090]

[0091] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A few-sample image generation method based on Conditional StyleGAN, characterized in that, Includes the following steps: 1) Data Acquisition: Divide the given dataset X into a set of visible categories C in the category space. s and the set of invisible categories C u Subsequently, based on the division of the category set, the images in the given dataset X are divided into a visible category image set X. s and the set of invisible category images X u The objective is: given a certain invisible category c u a subset of k images For the invisible category c u Generate diverse and high-quality images, where c u ∈C u k≤5, Represents a subset of images of the invisible category. The image in the image, where i represents the index of the image subset; 2) Pre-trained StyleGAN: Using the visible category image set X s Pre-training a conditional StyleGAN yields a pre-trained conditional generator G. s Conditional discriminator D s ; 3) Optimize Conditional StyleGAN using Invisible Class Images: Using a subset of images of a given invisible category For G obtained in step 2), s and D s Fine-tuning is performed to obtain the optimized condition generator G. u Conditional discriminator D u This includes two phases: the latent space localization phase and the latent space fine-tuning phase. The latent space localization stage uses a subset of k invisible category images. The image in the image serves as the anchor point, and the pre-trained conditional generator G... s Locating a given invisible category c in the latent space u The subspace of c is obtained. u The corresponding invisible category feature vector f u To obtain c u The initial data distribution, and optimization of G s Mapping network The optimized condition generator G is obtained u ; The latent space fine-tuning stage is based on the invisible category c obtained from the latent space localization stage. u The initial data distribution is used to further fine-tune the condition generator G obtained during the latent space localization stage. u This makes G u Fully learn the invisible category c u Features of the alignment condition generator G u Distribution of generated images and invisible category c u The image distribution is optimized, while further optimizing the invisible category c. u Invisible category feature vector f u To make it compatible with G u In fine-tuning G u During the process, the pre-trained conditional discriminator D is expanded. s D u And at the same time for D u Make fine adjustments; Both the latent space localization stage and the latent space fine-tuning stage use semantic regularization terms to ensure the stability of the optimization process; 4) Randomly sample noise to generate an image: Randomly sample Gaussian noise z, and compare it with the invisible category c obtained in step 3). u Category feature vector f u Input together the optimized condition generator G obtained in step 3) u Generate the invisible category c u The image.

2. The few-shot image generation method based on conditional StyleGAN according to claim 1, characterized in that: In step 1), the given dataset X is divided into a set of visible categories in the category space. and invisible category set in and N represents the visible category and the invisible category, respectively. s and N u These represent the number of visible categories and the number of invisible categories, respectively; then, based on the category division, the images in the given dataset X are divided into a set of visible category images X. s and the set of invisible category images X u Visible category image set X s Includes all corresponding categories c belonging to the set of visible categories C. s Image x s Each visible category contains n images, and the invisible category contains a set X of images. u This includes all corresponding categories c belonging to the set of invisible categories C. u Image x u .

3. The few-shot image generation method based on conditional StyleGAN according to claim 2, characterized in that: In step 2), the set of visible category images X is used. s Pre-train the conditional StyleGAN to obtain the conditional generator G. s Conditional Discriminator D s During pre-training, the condition generator G s Conditional discriminator D s Iterate and update alternately until convergence.

4. The few-shot image generation method based on conditional StyleGAN according to claim 3, characterized in that: In step 3), the latent space localization stage uses the anchor point approximation loss L. app and anchor point strength constraint L mgt Two optimization objectives for the invisible class feature vector f u Trainable anchor noise set Φ opt and pre-trained condition generator G s Mapping network Optimize, a i The noise is the trainable anchor point, where the condition generator in the optimization process is denoted as G. u The latent space localization stage includes the following steps: 1) Obtain the target anchor point set Θ, where, Target anchor point w i Subset of invisible category images Invisible category images Using the pre-trained condition generator G s The inverse mapping I(·,·) yields: 2) Randomly initialize the invisible category feature vector f u and the set of trainable anchor noise Φ opt ; 3) Obtain the set of trainable anchor points Among them, trainable anchor points From the invisible category feature vector f u Trainable anchor noise set Φ opt Through condition generator G u Mapping network Mapping to the latent space yields: 4) Use anchor points to approximate the loss L app and anchor point strength constraint L mgt Optimize: The anchor point approximation loss L app Constrain k trainable anchor points and target anchor point w i The L2 loss aims to obtain the feature vector f of the invisible class. u Its definition is as follows: The anchor point strength constraint L mgt The goal is to limit the set of trainable anchor noise Φ opt Trainable anchor noise a i The size of a prevents a i Overfitting is defined as follows: Target loss L during latent space localization phase loc Defined as: L loc =L app +L mgt During the training process of the latent space localization phase, the condition generator G... u Mapping network Invisible category feature vector f u and trainable anchor noise a i Alternating optimization; where the condition generator G u Mapping network The learning rate used is the invisible class feature vector f u and the set of trainable anchor noise Φ opt The learning rate used is λ times the learning rate, where λ ≤ 1.

5. The few-shot image generation method based on conditional StyleGAN according to claim 4, characterized in that: In step 3), the latent space fine-tuning stage uses adversarial loss L. adv And anchor point stability loss L stb The invisible category feature vector f obtained during the latent space localization stage u Trainable anchor noise set Φ opt Condition generator G u Conditional discriminator D s Optimizations were performed, including: Condition discriminator D s Embedded network D embed The input dimension is expanded from n dimensions to n+1 dimensions to support additional invisible categories c. u The extended condition discriminator is denoted as D. u ; The resistance loss L adv The goal is to align the generated image distribution with the invisible category c. u The distribution of real images is defined as follows: In the formula, E represents the expected value. Represents a subset of images of the invisible category. The image in the image, y u Indicates the invisible category c u The category label, z, indicates that it is from P. Z Gaussian noise from random sampling, P Z G represents a standard Gaussian distribution. u and D u These represent the optimized condition generator and condition discriminator, respectively. The anchor point stability loss L stb The goal is to create a stable condition generator G. u The semantic information of the generated distribution is defined as follows: L stb =L perc +L mgt In the formula, L perc Indicates perceived loss. Represents the invisible category feature vector f u and trainable anchor noise a i Through condition generator G u The obtained image is L mgt Indicates anchor point strength constraint; The target loss L during the latent space fine-tuning stage ref Defined as: L ref =L adv +L stb Condition generator G u Conditional discriminator D u Alternating optimization, wherein the anchor point stability loss L stb Acting on G u The optimization phase.

6. The few-shot image generation method based on conditional StyleGAN according to claim 5, characterized in that: In step 3), regularization terms that maintain semantics are included, such as those used in the latent space localization phase. Regularization term in the latent space fine-tuning phase Conditional generator mapping network for regularization term constraint optimization in the latent space localization stage Mapping network with pre-trained conditional generator The consistency between them is defined as follows: In the formula, z represents the Gaussian noise of the same random sample, and f s Indicates the visible category c s eigenvectors; For the regularization term in the latent space fine-tuning stage, respectively constrain the generator G. u With G s Convolutional modules of the inter- and conditional discriminators and The consistency between them is defined as follows: In the formula, G represents the pre-trained condition generator. s Using randomly sampled Gaussian noise z and visible category feature vector f s The generated image, i.e. L perc (G u (z,f s ),G s (z,f s Regularization condition generator G u Condition generator G for constrained optimization u and pre-trained condition generator G s Visual discrepancies between generated images; Convolutional module of a conditional discriminator for constraint optimization and the convolutional module of the pre-trained conditional discriminator For the same generated image Consistency of output convolutional features; The total loss L1 during the latent space localization stage is: The total loss L2 during the latent space fine-tuning stage is:

7. The few-shot image generation method based on conditional StyleGAN according to claim 6, characterized in that: In step 4), an invisible category image is generated by randomly sampling Gaussian noise. The generation process is as follows: Randomly sample Gaussian noise z, and compare it with the invisible category feature vector f obtained in step 3). u Input together the optimized condition generator G obtained in step 3) u Generate invisible category c u Image

Images