Small sample image generation method and device based on light model fine tuning and storage medium

Abstract

The application provides a small sample image generation method based on lightweight model fine-tuning, a device and a storage medium, and relates to the field of computer vision.The small sample image generation method based on lightweight GAN fine-tuning can relieve the GAN overfitting problem in the small sample scene, and meanwhile, the distillation strategy with key region constraint can better focus on the features of the region of interest of a user.The method comprises the following steps: a model compression method is used to obtain a lightweight model to relieve the overfitting in the model fine-tuning process; a mask operation is introduced to perform key region constraint; and a cross-domain channel self-correlation distillation loss is designed to process the small sample image generation task with deformation.The application proposes a cross-domain spatial self-correlation loss with key region constraint by combining the mask operation and the spatial self-correlation loss.Through the analysis of the experimental results of multiple data sets, it can be concluded that the small sample image generation method based on lightweight GAN model fine-tuning proposed by the application shows competitive results.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a method, apparatus and storage medium for generating few-sample images based on lightweight model fine-tuning. Background Technology

[0002] In recent years, with the rapid development of deep learning, Generative Adversarial Networks (GANs) have become the mainstream method for various computer vision tasks such as image generation, image editing, image super-resolution, image transformation, and video generation. Image generation is one of the main tasks in the field of computer vision. In recent years, researchers have proposed a series of deep learning-based image generation methods. Currently, GAN-based image generation methods suffer from the problems of huge image data requirements and high training costs. Training a generative adversarial network model requires a lot of manpower to collect image data and a large number of high-performance GPUs to complete the model training. In many real-world applications, the cost of obtaining training datasets is too high to support training models from scratch to complete image generation tasks. For example, in application scenarios where GAN models are trained using images of paintings created by artists, it is impractical for artists to create thousands of paintings in order for the GAN model to generate a large number of paintings with the artist's style. In scenarios lacking large-scale training datasets, the trained GAN models are prone to overfitting and mode collapse.

[0003] Because GAN models have a large number of parameters and the target domain dataset has limited image data, the generator is prone to model collapse, resulting in generated images lacking diversity (but not target domain consistency). Therefore, effectively maintaining the diversity of generated images and target domain consistency after generator fine-tuning is crucial for few-sample image generation tasks based on model fine-tuning. Summary of the Invention

[0004] Purpose of the invention: To propose a method, apparatus and storage medium for generating small sample images based on lightweight model fine-tuning, so as to solve the above-mentioned problems existing in the prior art.

[0005] Firstly, a few-sample image generation method based on lightweight model fine-tuning is proposed, with the following steps:

[0006] (1) Based on the image characteristics of the source and target domains in the AFSG (Adaptation-based Few-shot Image Generation) task, the dataset is divided into a spatial feature variation dataset and a spatial feature invariant dataset;

[0007] (2) Lightweight GAN models are obtained by performing structured pruning on GANs, and knowledge distillation is used to improve the generation performance of lightweight GAN models in the source domain.

[0008] (3) In the AFSG task, a content parsing network is introduced to parse key areas, and the resulting mask is used to eliminate the interference of the background on the subsequent fine-tuning process;

[0009] (4) Combining the masking operation in step (3), for the task of generating spatial feature invariant datasets, the image spatial structure is constrained to remain consistent while ignoring the interference of the background on the fine-tuning task by using cross-domain spatial autocorrelation loss of key regions during the fine-tuning process.

[0010] (5) Combining the masking operation in step (3), and considering the characteristics of the task of generating datasets with spatial feature changes, the spatial structure constraints are relaxed by using cross-domain channel autocorrelation loss of key regions during the fine-tuning process, while ignoring the interference of the background on the fine-tuning task.

[0011] (6) The above-mentioned cross-domain spatial autocorrelation loss and cross-domain channel autocorrelation loss of key regions are used for the small sample image generation task of spatial feature invariant dataset and spatial feature change dataset, respectively.

[0012] In a further embodiment of the first aspect, the present invention applies a lightweight GAN fine-tuning strategy to the AFSG task for the first time. In step (2), a lightweight GAN model is obtained by employing two deep learning model compression algorithms: structured pruning and knowledge distillation. The overfitting problem in small sample scenarios is alleviated by using lightweight GAN fine-tuning.

[0013] In a further embodiment of the first aspect, in step (3), to address the background interference problem in the AFSG task, this invention, for the first time, introduces a content parsing network to generate a mask to focus on key regions. In the few-sample image generation task, the source domain image is... The mask is obtained by feeding the pre-trained content parsing network. M h,w ∈{0,1}. Where M h,w =0 indicates that the spatial location (h,w) is the background region, M g,w =1 indicates that the spatial location (h,w) is the foreground region. The above operations classify the foreground and background regions of the image, allowing for focus on information in key areas during fine-tuning.

[0014] In a further embodiment of the first aspect, the key region cross-domain spatial autocorrelation loss constraint in step (4) differs from other AFSG methods in that this invention introduces cross-domain channel autocorrelation loss for the first time in the few-sample image generation task, constraining the diversity of the GAN fine-tuning process. This invention obtains the mask through a content parsing network. This is used to distinguish between foreground and background regions in an image, enabling a focus on the foreground region and neglecting background interference during fine-tuning of small sample image generation tasks. The distillation loss design during fine-tuning is shown in Equation (1). Based on the traditional spatial autocorrelation loss constraint, a masking operation is applied to ignore the spatial similarity calculation between the background and foreground regions, as well as the spatial similarity calculation between background regions. For the feature map of the generator... After merging the spatial dimensions and transposing them, and normalizing the channel dimensions, we obtain... for Merging spatial dimensions yields

[0015]

[0016] in, N = W*H represents the feature map of the l-th layer of the generator, s represents the source domain generator, and t represents the target domain generator. This indicates that the mask vector is being adjusted.

[0017] In a further embodiment of the first aspect, the key region cross-domain channel autocorrelation loss constraint in step (5) differs from other few-sample image generation tasks based on model fine-tuning in that, for the first time in the AFSG task, this invention relaxes the strong constraint of spatial autocorrelation on spatial structure by applying a cross-domain channel autocorrelation loss constraint. Based on the cross-domain channel autocorrelation constraint, a mask is then used... The foreground and background regions in the feature map are distinguished, background interference is ignored, and the foreground region is focused on. The calculation process of this invention is shown in formula (2). For the feature map of the generator... By merging spatial dimensions and normalizing, we obtain Similarly, for Merging spatial dimensions yields

[0018]

[0019] in, N = W*H represents the feature map of the l-th layer of the generator, s represents the source domain generator, and t represents the target domain generator.

[0020] Secondly, a few-sample image generation device based on lightweight model fine-tuning is proposed. This device mainly includes the following three modules:

[0021] ①Mask generation module;

[0022] ②Key area cross-domain spatial autocorrelation loss constraint module;

[0023] ③ Autocorrelation loss constraint module for cross-domain channels in key areas;

[0024] Module ① is mainly used to generate a mask to distinguish the foreground and background regions of an image; Module ② is designed for spatially non-deformable datasets and constrains key regions to maintain the consistency of their spatial autocorrelation matrices during fine-tuning, so that the GAN model can maintain its ability to generate diversity during fine-tuning; Module ③ is mainly used for spatially deformable datasets and constrains key regions to maintain the consistency of their channel autocorrelation matrices during fine-tuning, so that the GAN model can maintain its ability to generate diversity during fine-tuning.

[0025] Thirdly, an electronic device is proposed, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to execute the small sample image generation method as described in the first aspect.

[0026] Fourthly, a computer-readable storage medium is proposed, wherein at least one executable instruction is stored therein, which, when executed on an electronic device, causes the electronic device to perform the operation of the few-sample image generation method as described in the first aspect.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] (1) A strategy based on lightweight GAN fine-tuning was proposed for the first time in the AFSG task and its effectiveness was verified.

[0029] (2) For the first time, a masking operation focusing on key regions was introduced in the AFSG task to eliminate the interference of the background on the GAN fine-tuning process.

[0030] (3) Cross-domain channel autocorrelation loss was applied for the first time in the AFSG task, and channel autocorrelation loss with key region constraints was proposed by combining masking operation.

[0031] (4) A cross-domain spatial autocorrelation loss with key region constraints is proposed by combining masking operations and spatial autocorrelation loss. Analysis of experimental results on multiple datasets shows that the proposed few-sample image generation method based on lightweight GAN model fine-tuning exhibits competitive results. Attached Figure Description

[0032] Figure 1 This is a schematic diagram illustrating the small sample image generation algorithm of the present invention and its corresponding representative methods.

[0033] Figure 2 This is a diagram of the small sample image generation framework based on lightweight GAN fine-tuning of the present invention.

[0034] Figure 3 This invention relates to a lightweight model based on different parameter quantities for fine-tuning FID and Intra-LPIPS index plots.

[0035] Figure 4 This describes the generation effect of the method of this invention and other methods on the Sketches dataset. Detailed Implementation

[0036] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0037] In few-shot image generation tasks based on model fine-tuning, the main focus is on fine-tuning a pre-trained GAN model trained on a large-scale source domain dataset using a dataset with a small number of target domain images to generate images with both diversity and target domain consistency. However, due to the large number of parameters in the GAN model and the limited number of target domain images, the generator is prone to model collapse, resulting in generated images lacking diversity (target domain consistency). Therefore, effectively maintaining the diversity and target domain consistency of the fine-tuned generator is crucial for few-shot image generation tasks based on model fine-tuning. The main problems addressed in this invention patent are as follows:

[0038] Question (1): How to alleviate the overfitting problem caused by fine-tuning the model using a small number of target domain images;

[0039] Question (2): How to preserve the image diversity generated by the source domain generator during fine-tuning;

[0040] Question (3): How to focus on important foreground areas during fine-tuning;

[0041] The problem in question (1) stems from the mismatch between a small target domain dataset and a GAN model with a large number of parameters. Traditional methods primarily address overfitting by fine-tuning some model parameters. This invention, for the first time, employs a lightweight GAN fine-tuning method to solve this problem.

[0042] For problem (2), existing work mainly involves fine-tuning the model by applying constraints such as distillation loss. For example, CDC applies cross-domain consistency constraints to the distance between images before and after fine-tuning, and RSSA applies relaxed spatial structure alignment constraints to the images before and after fine-tuning. This invention differs from the above methods by using cross-domain channel autocorrelation loss constraints.

[0043] Regarding problem (3), current mainstream work on few-sample image generation tasks based on model fine-tuning does not consider the interference of background regions on the fine-tuning process. This invention is the first to use a content parsing network to generate a mask to distinguish between foreground and background regions. By combining the mask with distillation loss, the interference of background regions on the fine-tuning process is eliminated.

[0044] Example 1:

[0045] This embodiment designs a novel and effective few-shot image generation algorithm based on lightweight GAN fine-tuning. The core method mainly includes three key modules:

[0046] ①Mask generation module;

[0047] ②Key area cross-domain spatial autocorrelation loss constraint module;

[0048] ③ Autocorrelation loss constraint module for cross-domain channels in key areas;

[0049] Module ① is mainly used to generate a mask to distinguish the foreground and background regions of an image; Module ② is designed for spatially non-deformable datasets and constrains key regions to maintain the consistency of their spatial autocorrelation matrices during fine-tuning, so that the GAN model can maintain its ability to generate diversity during fine-tuning; Module ③ is mainly used for spatially deformable datasets and constrains key regions to maintain the consistency of their channel autocorrelation matrices during fine-tuning, so that the GAN model can maintain its ability to generate diversity during fine-tuning.

[0050] The following is a detailed explanation of the three modules mentioned above:

[0051] Mask generation module

[0052] To eliminate the interference of background regions on the fine-tuning process in the AFSG task, this invention introduces a masking operation to distinguish between foreground and background regions. By combining masking and distillation loss, the interference of background regions on the fine-tuning process is eliminated. In specific few-sample image generation tasks, the focus is on the content of the foreground region, and existing methods have not designed targeted processing strategies for this situation.

[0053] The source domain generator G is used in the AFSG task. s and target domain generator G t During the fine-tuning process, this module will... s generated image The content is fed into a content parsing network to obtain the mask. M h,w ∈{0,1}. Where M h,w =0 indicates that the spatial location (h,w) is the background region, M h,w=1 indicates that the spatial location (h,w) is the foreground region, i.e. the focus region.

[0054] Key Area Cross-Domain Channel Autocorrelation Constraint Module

[0055] To maintain the generator's ability to generate diverse images during GAN fine-tuning, this invention introduces key cross-domain channel autocorrelation constraints during the fine-tuning process. During fine-tuning, these constraints are applied to the input noise. Obtain the source domain generator G s Generate G from the target domain t Original feature map and By merging spatial dimensions and normalizing, we obtain Where N = W * H. The cross-domain channel autocorrelation loss is calculated by using cosine similarity to determine the pairwise similarity between channels. Therefore, the cross-domain channel autocorrelation loss constraint can be defined as shown in formula (3):

[0056]

[0057] Where l represents the l-th level in the generator.

[0058] The method of this invention differs from classical methods. The CDC method constrains the inter-sample distance to remain consistent before and after fine-tuning, and for batch noise of type B, it simultaneously inputs to the source domain generator G. s Generate G from the target domain t The obtained intermediate raw feature map and By merging the spatial dimension and the channel dimension and normalizing, we obtain And M = C*W*H. CDC calculates the pairwise similarity between samples using cosine similarity and obtains a B×B similarity matrix. It applies a relatively weak distance constraint by constraining the distance between samples before and after fine-tuning to maintain consistency. The RSSA method maintains the structural information of the image by constraining the spatial structure consistency of samples before and after fine-tuning. For the same noise input to the source domain generator G... s Generate G from the target domain t The obtained intermediate raw feature map and By merging spatial dimensions, transposing and normalizing, we obtain... And N = W*H. RSSA calculates the pairwise similarity between spatial locations in the image using cosine similarity and obtains an N×N similarity matrix, constraining the structural consistency of the image before and after fine-tuning, thus imposing a strong structural consistency constraint. While RSSA's spatial structural consistency can handle spatially invariant datasets well, for datasets with changing spatial features, the excessively strong spatial structural constraint makes it difficult for the GAN model to learn deformed spatial features during fine-tuning. Therefore, the channel autocorrelation loss constraint proposed in this invention achieves a good trade-off between the sample distance consistency constraint of CDC and the spatial structural consistency constraint of RSSA, imposing a stronger constraint than CDC while avoiding a strong constraint on spatial structure.

[0059] This invention, based on channel autocorrelation constraints, combines masking... Distinguish between foreground and background regions in the feature map, ignoring background interference and focusing on the foreground region. The final calculation process is shown in formula (2). For the generator's feature map... By merging spatial dimensions and normalizing, we obtain Similarly, for Merging spatial dimensions yields

[0060]

[0061] in, N = W*H represents the feature map of the l-th layer of the generator, s represents the source domain generator, and t represents the target domain generator.

[0062] Key Area Cross-Domain Spatial Autocorrelation Constraint Module

[0063] To maintain the generator's ability to generate diverse images during GAN fine-tuning, this invention introduces key cross-domain spatial autocorrelation constraints during the fine-tuning process. During fine-tuning, for the same noise input to the source domain generator G... s Generate G from the target domain t The obtained intermediate raw feature map and By merging spatial dimensions, transposing and normalizing, we obtain... And N = W * H. Cross-domain spatial autocorrelation loss is calculated using cosine similarity to determine the pairwise similarity between different spatial locations. This invention, based on spatial autocorrelation constraints, combines masking... Distinguish between foreground and background regions in the feature map, and eliminate similarity calculation operations between background and foreground regions, and between background regions. The final calculation process is shown in formula (1). Merging spatial dimensions yields

[0064]

[0065] Where l represents the feature map of the l-th layer of the generator, s represents the source domain generator, and t represents the target domain generator. For regions with a value of 0, the two corresponding positions do not need to have their similarity calculated. The similarity between two locations corresponding to a region with a value of 1 needs to be calculated.

[0066] The structure of the present invention is as follows Figure 1 As shown, it mainly consists of two generators, a mask generation module, and a cross-domain channel autocorrelation constraint module. Both generators are given the same noise vector as input. Features with size 64×64, 128×128, and 256×256 from the two generators are used to calculate the distillation loss. Furthermore, the generated image from the source domain generator is fed into a content parsing network to obtain a mask, which is then applied to the obtained feature map.

[0067] Example 2:

[0068] like Figure 2 Therefore, this invention provides a few-shot image generation algorithm based on lightweight GAN fine-tuning. The model training phase includes the following steps:

[0069] (1) Input the noise z into the source domain generator G s and target domain generator G t Generate image I s and I t ;

[0070] (2) Put I s The result is sent to the mask generation module to obtain the mask M;

[0071] (3) In the cross-domain channel autocorrelation loss constraint module (cross-domain spatial autocorrelation loss constraint module), obtain the feature maps of the generator with sizes of 64×64, 128×128, and 256×256, and calculate the loss by combining the mask to eliminate the influence of the background area.

[0072] (4) Will I t and the target domain real image The data is fed into the discriminator to calculate and generate adversarial loss;

[0073] (5) Add the generation adversarial loss and distillation loss and backpropagate them only to the target domain generator G. t Update the parameters accordingly;

[0074] (6) To the target domain generator G t Input noise z ′ Generate image I t ′;

[0075] (7) Will I t ′ and the target domain real image The data is fed into the discriminator to calculate and generate adversarial loss;

[0076] (8) Backpropagate the generated adversarial loss and update the parameters of the discriminator D only;

[0077] (9) Repeat steps (1) to (8) until generator G is generated. t The discriminator D is trained until convergence.

[0078] This invention first verifies the correctness of the view that "fine-tuning a lightweight model can achieve better small sample image generation results" through experiments. Figure 3 This paper demonstrates the generation of quantitative evaluation metrics using small sample images on the Vincent van Gogh dataset with the original model and pruning rates of 30%, 40%, 50%, 60%, 70%, and 80% (the number of channels removed by the generator G according to the corresponding proportion). The experiments are based on the TGAN model fine-tuning method, updating the parameters of the generated GAN using only the generative adversarial network loss. Figure 3 The paper presents the FID and Intra-LPIPS metrics for different training epochs. FID evaluates the distance between the generated image data distribution and the target domain image data distribution; a smaller FID indicates a closer proximity between the two distributions. Intra-LPIPS evaluates the diversity of generated images; a larger Intra-LPIPS indicates higher diversity. Figure 3 It can be observed that during fine-tuning, the FID metric does not reveal the superiority of lightweight models for generators with different parameter sets; the performance differences between models with different parameter sets are not significant at different training stages. However, through... Figure 3 It is evident that during fine-tuning, for generators with different parameter counts, models with fewer parameters perform better in preserving generative diversity, and Intra-LPIPS performs even better. These experiments fully validate the correctness of the viewpoint that "fine-tuning based on lightweight models can achieve better few-sample image generation results."

[0079] To evaluate the performance of the few-shot image generation algorithm proposed in this invention, our method was experimentally compared with the CDC method, currently the best-performing model-based fine-tuned few-shot image generation method, on five datasets. Based on the presence of "spatial feature misalignment" or "feature mismatch" between the source and target domains, the five datasets were divided into two categories: two spatial feature-invariant datasets, Sketches and Vincent van Gogh, and three spatial feature-transformable datasets, FFHQ-sunglasses, FFHQ-babies, and Amedeo. The proposed cross-domain channel autocorrelation loss is mainly used for spatial feature-transformable datasets, achieving excellent generation performance when combined with a masking mechanism. On spatial feature-invariant data, this invention combines the cross-domain spatial autocorrelation proposed in RSSA with the masking mechanism used in this invention. Tables 1 and 2 show the FID and Intra-LPIPS metrics of the proposed method on the FFHQ-sunglasses, FFHQ-babies, and Amedeo datasets, respectively. The second column is the reproducibility metric of the CDC method, and the third column is the metric of the proposed method (lightweight model + masking + cross-domain channel autocorrelation loss). On the FFHQ-babies dataset, although the FID metric of the CDC method is significantly better than that of the method proposed in this invention, the baby faces generated by the CDC method on this dataset contain obvious signs of forgery.

[0080] Table 1: FID metrics of the proposed method on the FFHQ-sunglasses, FFHQ-babies, and Amedeo datasets.

[0081]

[0082] Table 2: Intra-LPIPS metrics of the method on the FFHQ-sunglasses, FFHQ-babies, and Amedeo datasets

[0083]

[0084] Tables 3 and 4 show the FID and Intra-LPIPS metrics of the proposed method on the Sketches and Vincent van Gogh datasets, respectively. The second column is the reproducibility metric of the CDC method, and the third column is the metric of the proposed method (lightweight model + mask + cross-domain spatial autocorrelation loss).

[0085] Table 3: FID metrics of this method on the Sketches and Vincent van Gogh datasets

[0086]

[0087] Table 4: Intra-LPIPS metrics of this method on the Sketches and Vincent van Gogh datasets

[0088]

[0089] Figure 4 The proposed method is shown to generate images on the Sketches and FFHQ-sunglasses datasets. The left figure shows the 10 target domain images used in the fine-tuning process. The first row of the left figure shows the generation effect of the FreezeD method, the second row shows the generation effect of the CDC method, the third row shows the generation effect of the RSSA method, the fourth row shows the generation effect of the proposed method, and the fifth row shows the image generation effect after feeding the same noise into the source domain generator.

[0090] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A few-sample image generation method based on lightweight model fine-tuning, characterized in that, Includes the following steps: S1. Based on the image characteristics of the source and target domains in the small sample image generation task, the dataset is divided into spatial feature variation dataset and spatial feature invariant dataset. S2. Lightweight GAN models are obtained by performing structured pruning on GANs, and knowledge distillation is used to improve the generation performance of lightweight GAN models in the source domain. S3. In the few-sample image generation task, a content parsing network is introduced to parse key regions and transform the source image. The mask is obtained by feeding the pre-trained content parsing network. , ;in Indicates spatial location It is the background area. Indicates spatial location The foreground region is used to obtain a mask to eliminate the interference of the background on the subsequent fine-tuning process. The foreground and background regions of the image are classified through the above operations to focus on the information of key regions during the fine-tuning process. S4. Combining the masking operation of S3, for the task of generating spatially invariant datasets, during the fine-tuning process, cross-domain spatial autocorrelation loss of key regions is used to constrain the spatial structure of the image to remain consistent while ignoring the interference of the background on the fine-tuning task. Specifically, this includes: S4-1. Based on the traditional spatial autocorrelation loss constraint, a masking operation is applied to ignore the spatial similarity calculation between the background region and the foreground region, as well as the spatial similarity calculation between background regions. S4-2, Feature Map of the Generator After merging and transposing the spatial dimensions, and normalizing the channel dimensions, we obtain the following: ,for By merging spatial dimensions, we obtain ; The distillation loss during the fine-tuning process is designed as follows: ； in, Represents the generator's first Feature map of the layer Represents the source domain generator. Represents the target domain generator. This indicates an adjustment to the mask vector; S5. Combining the masking operation of S3, and considering the characteristics of the dataset generation task with spatial feature changes, the fine-tuning process uses cross-domain channel autocorrelation loss of key regions to relax spatial structure constraints while ignoring the interference of the background on the fine-tuning task. S6. The above-mentioned cross-domain spatial autocorrelation loss and cross-domain channel autocorrelation loss of key regions are used for the small sample image generation task of spatial feature invariant dataset and spatial feature change dataset, respectively.

2. The method for generating small sample images according to claim 1, characterized in that, Step S5 further includes: Based on the autocorrelation constraints of cross-domain channels, by combining a mask Distinguish between foreground and background regions in the feature map, ignore background interference, and focus on the foreground region.

3. The method for generating small sample images according to claim 2, characterized in that, For the feature map of the generator By merging spatial dimensions and normalizing, we obtain... ;for By merging spatial dimensions, we obtain ; ； in, Represents the generator's first Feature map of the layer Represents the source domain generator. This represents the target domain generator.

4. A few-shot image generation apparatus based on lightweight model fine-tuning, used to execute the few-shot image generation method according to any one of claims 1 to 3, characterized in that, include: The mask generation module is used to perform masking operations, distinguishing between foreground and background regions. By combining masking with distillation loss, it eliminates the interference of background regions on the fine-tuning process. The key region cross-domain channel autocorrelation constraint module, for spatially invariant datasets, constrains the key regions to maintain the consistency of the spatial autocorrelation matrix during fine-tuning, so that the GAN model can maintain its generative diversity during fine-tuning; The key region cross-domain spatial autocorrelation constraint module is used for spatial deformation datasets to constrain the channel autocorrelation matrix of key regions to remain consistent during fine-tuning, so that the GAN model can maintain its ability to generate diversity during fine-tuning.

5. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the few-sample image generation method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the operation of the small sample image generation method as described in any one of claims 1-3.

Images