Image generation method and system based on limited data set

Abstract

The present invention provides a method and system for generating images based on a limited dataset. The method comprises: constructing an encoder model using limited image data as the dataset; pre-training the encoder model through contrastive learning, KL divergence learning, and classification learning; training a conditional diffusion model using the dataset based on the conditional vectors output by the pre-trained encoder model; and, when a user inputs an image, generating an image similar to the input image using the trained conditional diffusion model based on the conditional vectors output by the pre-trained encoder model. This solution not only ensures the accuracy and reliability of the diffusion model but also effectively improves the quality of the generated images.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to an image generation method and system based on a limited data set. Background Art

[0002] Image generation is a key technology in computer vision. Its goal is to generate new images based on given input data or manipulate existing images to produce desired visual results. In recent years, deep learning has gradually enabled significant breakthroughs in image generation research. In particular, diffusion models have demonstrated their ability to generate high-quality and realistic images.

[0003] However, training diffusion models, whether conditional or unconditional, typically requires a large amount of training data. However, in some domains or application scenarios, collecting sufficient training data is often impractical. For example, in the field of medical imaging, concerns about patient privacy and legal constraints limit the collection and use of large-scale training data. Consequently, only a limited number of available training examples can be used.

[0004] Limited datasets are often insufficient to support the model training and validation process. For image generation models, limited training data hinders the diffusion model's ability to learn the complex features and underlying structure of images, resulting in reduced generalization and robustness. Currently, methods such as data augmentation and transfer learning are commonly used to address the issue of limited datasets. Data augmentation involves performing transformations such as rotation, translation, and scaling on existing training data to generate additional samples and increase the size of the dataset. Transfer learning involves fine-tuning a pre-trained model trained on large-scale training data on a limited dataset.

[0005] However, data augmentation using methods such as rotation, translation, and scaling may cause the model to fail to learn objective object relationships, such as objects that only differ in size, while eliminating color and arbitrary cropping may cause the key attributes of the objects to be lost, all of which will lead to a decrease in model accuracy; transfer learning depends largely on whether the pre-trained training data and the target data have similar characteristics. If the pre-trained training data and the target data have significant differences and cannot be generalized well, the accuracy of the model will be questionable.

[0006] Obviously, the current methods of data enhancement and transfer learning generate poor quality images, which can no longer meet the needs of practical applications. It is necessary to propose a new diffusion model that can achieve high-quality image generation under limited data training. Summary of the Invention

[0007] In view of this, an embodiment of the present invention provides an image generation method and system based on a limited data set, which is used to solve the problem that the existing diffusion model generates images with poor quality and low accuracy under a limited data set.

[0008] In a first aspect of an embodiment of the present invention, a method for generating an image based on a limited data set is provided, comprising:

[0009] Using limited image data as a dataset, constructing an encoder model, and pre-training the encoder model through contrastive learning, KL divergence learning, and classification learning;

[0010] Training a conditional diffusion model using the dataset based on the conditional vector output by the pre-trained post-encoder model;

[0011] When the user inputs an image, an image similar to the input image is generated through the trained conditional diffusion model based on the conditional vector output by the pre-trained post-encoder model.

[0012] In a second aspect of an embodiment of the present invention, a system for generating an image based on a limited data set is provided, comprising:

[0013] An encoder model construction module is used to construct an encoder model using limited image data as a dataset, and pre-train the encoder model through contrastive learning, KL divergence learning, and classification learning;

[0014] A diffusion model construction module, configured to train a conditional diffusion model using the dataset based on the conditional vector output by the pre-trained post-encoder model;

[0015] The image generation module is used to generate an image similar to the input image through the trained conditional diffusion model based on the conditional vector output by the pre-trained post-encoder model after the user inputs an image.

[0016] In a third aspect of an embodiment of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the steps of the method described in the first aspect of the embodiment of the present invention when executing the computer program.

[0017] In a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method provided in the first aspect of the embodiment of the present invention are implemented.

[0018] In an embodiment of the present invention, the encoder model is pre-trained by integrating pre-training methods such as contrastive learning, KL divergence learning, and classification learning, so that the encoder model can extract image features from limited image data as conditional vectors to guide the training and image generation of the conditional diffusion model. This not only enables image generation of the conditional diffusion model under small samples, but also ensures the accuracy and reliability of the conditional diffusion model, effectively improving the quality of images generated based on the conditional diffusion model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic flow chart of an image generation method based on a limited data set provided by one embodiment of the present invention;

[0021] Figure 2 A diagram showing the model architecture of an image generation method based on a limited data set provided by one embodiment of the present invention;

[0022] Figure 3 A schematic structural diagram of an image generation system based on a limited data set provided by one embodiment of the present invention;

[0023] Figure 4 The present invention provides a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] It should be understood that the terms "including" and similar expressions in the specification, claims, and drawings of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, or apparatus comprising a series of steps or units is not limited to the listed steps or units. Furthermore, the terms "first" and "second" are used to distinguish between different objects and are not intended to describe a specific order.

[0026] It is understandable that the unconditional diffusion model is difficult to control the images it generates, so it is necessary to integrate the conditional diffusion model and control the generation results of the diffusion model through conditions; the conditional diffusion model requires appropriate conditions to complete the image generation work under the premise of limited data. This condition cannot be learned with a small number of samples relying solely on a single type of information. The generated images lack image content and cannot be denoised. Therefore, it is necessary to propose a method of image generation based on a limited data set using an encoder to guide the conditional diffusion model.

[0027] See also Figure 1 , a flowchart of an image generation method based on a limited data set provided by an embodiment of the present invention includes:

[0028] S101, using limited image data as a data set, constructing an encoder model, and pre-training the encoder model through contrastive learning, KL divergence learning, and classification learning;

[0029] A training data set is constructed based on valid images to train the encoder model, which uses contrastive learning, KL divergence learning and classification learning to obtain a conditional vector of specific image features.

[0030] The encoder model includes a noise adding unit and a conditional enhancement unit. The noise adding unit is used to add different degrees of noise to the image. The conditional enhancement unit is used to extract and enhance the image features after adding noise, and sample supplementary feature information in the sample feature space.

[0031] The noise adding unit adds noise of the same degree n to the input image samples x and x', and adds noise of different degrees n+r to the sample x to obtain three different images x after adding noise. n , x′ n and x n+r , so that the conditional vector generated by the encoder can be in the same noise environment as the conditional diffusion model;

[0032] The conditional enhancement unit enhances the condition of the encoder Enc(·) pre-training, and adds noise to the image x n , x′ n and x n+r The feature information of the sample is extracted and enhanced, and the enhanced and supplemented feature information is sampled in the potential feature space of the sample, and the output is the conditional vector C(x n ), C(x′ n ) and C(x n+r ).

[0033] It can be understood that the condition enhancement unit will n , x′ n and xn+r The initial condition vector encoding C(x n ), C(x′ n ) and C(x n+r ), and then randomly sample latent variables from independent Gaussian distributions and Sampling is performed, where the mean μ(x n ), μ(x′ n ), μ(x n+r ) and the diagonal covariance matrix σ(x n ),σ(x′ n ),σ(x n+r ) is applied to the image x n , x′ n and x n+r function.

[0034] Furthermore, the noise adding unit adds noise in the following manner:

[0035]

[0036]

[0037] In the formula, x0 represents the original image to which noise is added, 0 represents the original image, that is, 0 steps of noise are added, t represents the diffusion step, and x t represents the original image with t-step noise added, represents a scalar parameter, a t =1-β t , β t is the linear variance table, z represents random noise, represents the normal distribution, I represents the variance of the normal distribution, n represents the number of steps of adding noise for the first time when training the encoder, n end represents the maximum value of the range of the initial noise addition, r represents the number of steps of adding noise again when training the encoder, r end Indicates the maximum value of the range where noise is added again;

[0038] As the number of diffusion steps t increases, the added noise will be closer to the Gaussian distribution. n and r are used to control the degree of noise addition in the early stage, so that the image features extracted by the encoder are based on the image distribution rather than the Gaussian distribution of the noise.

[0039] Preferably, β t β1=10 -4 and β T = 0.2 linear series, diffusion step T = 500, n end =80, r end=20, these parameters can ensure that the noise level contained in the conditional vector extracted by the encoder is in the early stage of the diffusion process of the diffusion model and has the same noise distribution as the diffusion model.

[0040] Specifically, based on contrastive learning, the image feature information is distinguished so that the conditional vectors output by sample images with the same content are in a close feature space, while the conditional vectors output by sample images with different content are in a farther feature space.

[0041] Among them, the contrastive learning loss function is expressed as:

[0042]

[0043] Where, represents the contrastive learning loss, x n 、x' n Both represent sample images, label is the label, and label∈{0,1}, when x n and x' n When the image is of the same type, label = 0, otherwise, label ≠ 0, C(x n ), C(x′ n ) represents the conditional vector, D(·) represents the conditional vector C(x n ) and C(x′ n ) is calculated using the Euclidean distance, and m represents the margin that controls the degree of distinction between similar and dissimilar pairs during contrastive learning.

[0044] Contrastive learning enables the encoder to distinguish or bring feature information closer. The encoder model can map images of the same type closer in space and images of different types farther apart, making it easier to distinguish image features. This helps learn key feature representations from limited data sets to generate high-quality images with smooth transitions and fine details.

[0045] Based on KL divergence learning, the image feature information obtained from the same sample image under different noise levels is processed similarly, so that the image feature information obtained from encoding the same image with different noise levels is as similar as possible;

[0046] Among them, the loss function of KL divergence learning is expressed as:

[0047]

[0048] Where, represents the KL divergence learning loss, w n+r represents the weight term, x n 、x n+r Both represent sample images, C(x n )、C(xn+r ) represents the conditional vector, C(x n ) i 、C(x n+r ) i Represent the conditional vector C(x n )、C(x n+r ) in the i-th element, D KL Represents the KL divergence between the conditional vectors of the image. The smaller the value, the closer the two distributions are. N is the total number of elements in the conditional vector.

[0049] w n+r The weight will decrease as the noise increases. As the noise level increases, the image gradually approaches the Gaussian distribution. In order to prevent the encoder from biasing the features toward the Gaussian distribution, the conditional vector C(x n ) to guide C(x n+r ), ensuring that they all belong to the same distribution, which helps the model mitigate the negative impact caused by noisy input and improve the quality of generated images;

[0050] Based on classification learning, the extracted image feature information is classified to ensure that the extracted image feature information can distinguish different sample contents;

[0051] Among them, the loss function of classification learning is expressed as:

[0052]

[0053]

[0054] Where, represents the classification learning loss, S(x n ) represents the type vector classified by the encoder, l x Indicates the type label, M indicates the number of types, Represents the mth element in the type vector, represents the mth type label, Indicates the classification result.

[0055] Classification learning further classifies the feature information (conditional vector) extracted by the encoder to ensure that the extracted feature information can distinguish different sample contents and guide the encoding of the conditional vector.

[0056] S102, training a conditional diffusion model using the data set based on the conditional vector output by the pre-trained encoder model;

[0057] The conditional diffusion model can be trained based on a data set constructed based on limited image data and a conditional vector output by an encoder model. The trained conditional diffusion model can generate similar images based on the input image and the conditional vector.

[0058] The backbone of the conditional diffusion model is the Unet architecture. Before entering the UNet, the conditional vector C(·) obtained from the encoder model is embedded through two fully connected (FC) layers to convert it into a conditional embedding Cemd(·). The conditional embedding is then sent to the Unet. After downsampling and upsampling stages, the diffusion model measures the noise ∈ t .

[0059] In the process of training the conditional diffusion model, it is necessary to provide the conditional diffusion model with a noise-added image x t , time step t and the condition of embedding Cemd(x), the objective function of the conditional diffusion model is:

[0060]

[0061] Where, Represents the training target of the conditional diffusion model, and z is random noise.

[0062] In order to enhance generalization and prevent overfitting during training, the conditional diffusion model has a certain probability (p) that no condition is provided. In this case, the objective function is:

[0063]

[0064] After the conditional diffusion model is trained, it generates an image through a back-diffusion process based on the conditional vector. To control the use of the conditional vector, the back-diffusion process is guided to generate an image with the desired characteristics. The noise of the model during the back-diffusion process can be described as:

[0065] ∈ t =(1-S)∈ θ (x t ,t)+S·∈ θ (x t ,t,Cemd(x0))

[0066] Where s represents the conditional scale used by the control diffusion model in image generation. A larger s indicates a preference for using conditional noise. Preferably, s>0.6.

[0067] Conditional Diffusion Model∈ θ Predictions are added to image x t The noise in t From image x t Subtract the noise ∈ t , we can get the image x of the previous step t-1 , we get the following expression:

[0068]

[0069] In the last step, the output image x0 can be calculated based on x1 and noise∈1, and the noise z in the last step is 0.

[0070] S103. When the user inputs an image, an image similar to the input image is generated through the trained conditional diffusion model based on the conditional vector output by the pre-trained encoder model.

[0071] For the image input by the user, after the encoder model outputs a conditional vector with image features, the conditional diffusion model generates an image with similar content to the input image but different background, subject attributes or subject position.

[0072] In some embodiments, as Figure 2 As shown, during the image generation process, the input image x is converted into a conditional vector C(x) by the encoder and encoded as Cemd(x0) after user editing (optional). The diffusion step number t is input into the conditional diffusion model ∈ θ In the prediction, the noise ∈ t , and repeat until the output image x0 is calculated.

[0073] In this embodiment, contrastive learning and KL divergence are used to pre-train the encoder, and then classification is combined so that the image features provided by the encoder model can assist the diffusion model training and generate high-quality images.

[0074] Using limited image data as input, this generative model provides richer feature information and greater interpretability than images generated using only type-conditional or unconditional diffusion models. The model is trained on limited data, generating images with similar content but distinct details. Not only does the generated images have high quality, rich diversity, and high clarity, but the overall model accuracy and reliability are also high, making them suitable for practical applications.

[0075] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a specific order of execution; the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0076] Figure 3 A schematic diagram of the structure of an image generation system based on a limited data set provided by an embodiment of the present invention, the system comprising:

[0077] An encoder model construction module 310 is configured to construct an encoder model using limited image data as a dataset, and to pre-train the encoder model through contrastive learning, KL divergence learning, and classification learning;

[0078] The encoder model includes a noise adding unit and a conditional enhancement unit. The noise adding unit is used to add different degrees of noise to the image. The conditional enhancement unit is used to extract and enhance the image features after adding noise, and sample supplementary feature information in the sample feature space.

[0079] Furthermore, the noise adding unit adds noise in the following manner:

[0080]

[0081]

[0082] In the formula, x0 represents the original image to which noise is added, t represents the diffusion step, and x t represents the original image with t-step noise added, represents a scalar parameter, a t =1-β t , β t is the linear variance table, z represents random noise, represents the normal distribution, I represents the variance of the normal distribution, n represents the number of steps of adding noise for the first time when training the encoder, n end represents the maximum value of the range of the initial noise addition, r represents the number of steps of adding noise again when training the encoder, r end Indicates the maximum value of the range where noise is added again;

[0083] As the number of diffusion steps t increases, the added noise is closer to the Gaussian distribution. n and r are used to control the degree of added noise in the early stage, so that the image features extracted by the encoder are based on the image distribution rather than the Gaussian distribution of the noise.

[0084] Specifically, based on contrastive learning, the image feature information is distinguished so that the conditional vectors output by sample images with the same content are in a close feature space, while the conditional vectors output by sample images with different content are in a farther feature space.

[0085] Among them, the contrastive learning loss function is expressed as:

[0086]

[0087] Where, represents the contrastive learning loss, x n 、x' n Both represent sample images, label is the label, and label∈{0,1}, when x n and x' n When the image is of the same type, label = 0, otherwise, label ≠ 0, C(x n ), C(x′ n) represents the conditional vector, D(·) represents the conditional vector C(x n ) and C(x′ n ) is calculated using the Euclidean distance, and m represents the margin that controls the degree of distinction between similar and dissimilar pairs during contrastive learning.

[0088] Based on KL divergence learning, the image feature information obtained from the same sample image under different noise levels is processed similarly, so that the image feature information obtained from encoding the same image with different noise levels is as similar as possible;

[0089] Among them, the loss function of KL divergence learning is expressed as:

[0090]

[0091] Where, represents the KL divergence learning loss, w n+r represents the weight term, x n 、x n+r Both represent sample images, C(x n )、C(x n+r ) represents the conditional vector, C(x n ) i 、C(x n+r ) i Represent the conditional vector C(x n )、C(x n+r ) in the i-th element, D KL Represents the KL divergence between the conditional vectors of the image. The smaller the value, the closer the two distributions are. N is the total number of elements in the conditional vector.

[0092] Based on classification learning, the extracted image feature information is classified to ensure that the extracted image feature information can distinguish different sample contents;

[0093] Among them, the loss function of classification learning is expressed as:

[0094]

[0095]

[0096] Where, represents the classification learning loss, S(x n ) represents the type vector classified by the encoder, l x Indicates the type label, M indicates the number of types, Represents the mth element in the type vector, represents the mth type label, Indicates the classification result.

[0097] a diffusion model construction module 320 for training a conditional diffusion model using the dataset based on the conditional vector output by the pre-trained post-encoder model;

[0098] Specifically, the conditional vector output by the encoder model is embedded through a fully connected layer, the conditional vector is converted into a conditional embedding, and the conditional embedding is transmitted to the conditional diffusion model;

[0099] The conditional diffusion model predicts the noise at each diffusion step based on the original noisy image, the number of diffusion steps, and the conditional embedding.

[0100] Preferably, the user edits the conditional vector output by the encoder model to control the generation of the image.

[0101] The image generation module 330 is used to generate an image similar to the input image through the trained conditional diffusion model based on the conditional vector output by the pre-trained post-encoder model after the user inputs an image.

[0102] Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0103] Figure 4 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device is used for image generation. Figure 4 As shown, the electronic device 4 of this embodiment includes: a memory 410, a processor 420 and a system bus 430, and the memory 410 includes an executable program 4101 stored thereon. It can be understood by those skilled in the art that Figure 4 The electronic device structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0104] The following combination Figure 4 A detailed introduction to the various components of electronic equipment:

[0105] The memory 410 can be used to store software programs and modules. The processor 420 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 410. The memory 410 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as cached data), etc. In addition, the memory 410 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0106] An executable program 4101 for a network request method is included in the memory 410. The executable program 4101 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 410 and executed by the processor 420 to generate high-quality images, etc. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the executable program 4101 in the electronic device 4. For example, the executable program 4101 can be divided into functional modules such as an encoder model construction module, a diffusion model construction module, and an image generation module.

[0107] Processor 420 is the control center of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in memory 410 and accessing data stored in memory 410, it performs various functions of the electronic device and processes data, thereby monitoring the overall status of the electronic device. Optionally, processor 420 may include one or more processing units; preferably, processor 420 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, application programs, etc., and the modem processor primarily handles wireless communications. It is understood that the modem processor described above may not be integrated into processor 420.

[0108] The system bus 430 connects the various functional components within the computer and can transmit data, address information, and control information. It can be a PCI bus, an ISA bus, a CAN bus, or the like. Instructions from the processor 420 are transmitted to the memory 410 via the bus, and the memory 410 feeds data back to the processor 420. The system bus 430 facilitates the exchange of data and instructions between the processor 420 and the memory 410. Of course, the system bus 430 can also connect to other devices, such as network interfaces and display devices.

[0109] In an embodiment of the present invention, the executable program executed by the processor 420 included in the electronic device includes:

[0110] Using limited image data as a dataset, constructing an encoder model, and pre-training the encoder model through contrastive learning, KL divergence learning, and classification learning;

[0111] Training a conditional diffusion model using the dataset based on the conditional vector output by the pre-trained post-encoder model;

[0112] When the user inputs an image, an image similar to the input image is generated through the trained conditional diffusion model based on the conditional vector output by the pre-trained post-encoder model.

[0113] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0115] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating an image based on a finite data set, characterized in that: include: Using limited image data as a dataset, constructing an encoder model, and pre-training the encoder model through contrastive learning, KL divergence learning, and classification learning; The encoder model uses contrastive learning, KL divergence learning and classification learning to obtain the conditional vector of specific image features; Among them, based on contrastive learning, the image feature information is distinguished, so that the conditional vectors output by sample images with the same content are in a close feature space, and the conditional vectors output by sample images with different content are in a farther feature space; Based on KL divergence learning, the image feature information obtained from the same sample image under different noise levels is processed similarly, so that the image feature information obtained from encoding the same image with different noise levels is as similar as possible; Based on classification learning, the extracted image feature information is classified to ensure that the extracted image feature information can distinguish different sample contents; Training a conditional diffusion model using the dataset based on the conditional vector output by the pre-trained post-encoder model; The conditional vector output by the encoder model is embedded through a fully connected layer, the conditional vector is converted into a conditional embedding, and the conditional embedding is transmitted to the conditional diffusion model; The conditional diffusion model predicts the noise of each diffusion step based on the original noise image, the number of diffusion steps and the conditional embedding; The user edits the conditional vector output by the encoder model to control image generation; When the user inputs an image, an image similar to the input image is generated through the trained conditional diffusion model based on the conditional vector output by the pre-trained post-encoder model.

2. The method according to claim 1, characterized in that The encoder model includes a noise adding unit and a conditional enhancement unit. The noise adding unit is used to add noise of different degrees to the image. The conditional enhancement unit is used to extract and enhance the image features after the noise is added, and to sample supplementary feature information in the sample feature space.

3. The method according to claim 2, characterized in that The adding of different degrees of noise to the image comprises: The noise adding unit adds noise in the following manner: In the formula, x0 represents the original image to which noise is added, t represents the diffusion step, and x t represents the original image with t-step noise added, represents a scalar parameter, a t =1-β t , β t is the linear variance table, z represents random noise, represents the normal distribution, I represents the variance of the normal distribution, n represents the number of steps of adding noise for the first time when training the encoder, n end represents the maximum value of the range of the initial noise addition, r represents the number of steps of adding noise again when training the encoder, r end Indicates the maximum value of the range where noise is added again; As the number of diffusion steps t increases, the added noise is closer to the Gaussian distribution. n and r are used to control the degree of added noise in the early stage, so that the image features extracted by the encoder are based on the image distribution rather than the Gaussian distribution of the noise.

4. The method according to claim 1, wherein The pre-training of the encoder model by contrastive learning, KL divergence learning and classification learning includes: The contrastive learning loss function is expressed as: Where, represents the contrastive learning loss, x n 、x' n Both represent sample images, label is the label, and label∈{0,1}, when x n and x' n When the image is of the same type, label = 0, otherwise, label ≠ 0, C(x n )、C(x' n ) represents the conditional vector, D(·) represents the conditional vector C(x n ) and C(x' n ) is calculated using the Euclidean distance, and m represents the margin that controls the degree of distinction between similar and dissimilar pairs during contrastive learning. The loss function of KL divergence learning is expressed as: Where, represents the KL divergence learning loss, w n+r represents the weight term, x n 、x n+r Both represent sample images, C(x n )、C(x n+r ) represents the conditional vector, C(x n ) i 、C(x n+r ) i Represent the conditional vector C(x n )、C(x n+r ) in the i-th element, D KL Represents the KL divergence between the conditional vectors of the image. The smaller the value, the closer the two distributions are. N is the total number of elements in the conditional vector. The loss function of classification learning is expressed as: Where, represents the classification learning loss, S(x n ) represents the type vector classified by the encoder, l x Indicates the type label, M indicates the number of types, Represents the mth element in the type vector, represents the mth type label, Indicates the classification result.

5. An image generation system based on a finite data set, characterized in that: include: An encoder model construction module is used to construct an encoder model using limited image data as a dataset, and pre-train the encoder model through contrastive learning, KL divergence learning, and classification learning; The encoder model uses contrastive learning, KL divergence learning and classification learning to obtain the conditional vector of specific image features; Among them, based on contrastive learning, the image feature information is distinguished, so that the conditional vectors output by sample images with the same content are in a close feature space, and the conditional vectors output by sample images with different content are in a farther feature space; Based on KL divergence learning, the image feature information obtained from the same sample image under different noise levels is processed similarly, so that the image feature information obtained from encoding the same image with different noise levels is as similar as possible; Based on classification learning, the extracted image feature information is classified to ensure that the extracted image feature information can distinguish different sample contents; A diffusion model construction module, configured to train a conditional diffusion model using the dataset based on the conditional vector output by the pre-trained post-encoder model; The conditional vector output by the encoder model is embedded through a fully connected layer, the conditional vector is converted into a conditional embedding, and the conditional embedding is transmitted to the conditional diffusion model; The conditional diffusion model predicts the noise of each diffusion step based on the original noise image, the number of diffusion steps and the conditional embedding; The user edits the conditional vector output by the encoder model to control image generation; The image generation module is used to generate an image similar to the input image through the trained conditional diffusion model based on the conditional vector output by the pre-trained post-encoder model after the user inputs an image.

6. The system according to claim 5, characterized in that The encoder model includes a noise adding unit and a conditional enhancement unit. The noise adding unit is used to add noise of different degrees to the image. The conditional enhancement unit is used to extract and enhance the image features after the noise is added, and to sample supplementary feature information in the sample feature space.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the image generation method based on a finite data set as described in any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the steps of the image generation method based on a finite data set as claimed in any one of claims 1 to 4 are implemented.

Images