Model training method and device, image generation method and device, electronic equipment and medium

Abstract

The invention provides a model training method and device, an image generation method and device, electronic equipment and a medium, and the method comprises the steps: obtaining an abnormal image sample, and a marked abnormal position feature and an abnormal type feature corresponding to the abnormal image sample; adding original noise into the abnormal image sample to obtain a first noise image; inputting the first noise image, the marked abnormal position feature and the abnormal type feature into a to-be-trained diffusion model to obtain predicted noise at the abnormal position; and based on the predicted noise and the original noise, adjusting the diffusion model to be trained until a trained diffusion model is obtained. According to the embodiment of the invention, the diffusion model learns the abnormal type representation and the abnormal position representation of the abnormal pixel information, the trained diffusion model can generate the abnormal image of the specified abnormal type at the specified abnormal position, the diversity and controllability of the abnormal images are realized, and the abnormal images can be generated in batches.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a model training method, an image generation method, a device, an electronic device and a medium. Background Art

[0002] With the development of technology, the model has been widely used in many fields. For example, in industrial quality inspection scenarios, after taking a photo of the product, the abnormal image detection model can be used to detect the product photo.

[0003] In the training phase of the abnormal image detection model, a large number of normal image samples and a large number of abnormal image samples are needed to train the model. However, among the collected image samples, most are normal image samples, and defective image samples account for only a small part. In the case of a small number of abnormal image samples, it is impossible to train an accurate abnormal image detection model. Therefore, how to obtain a large number of abnormal image samples is a problem worthy of attention. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a model training method, an image generation method, an apparatus, an electronic device, and a medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a model training method, comprising: Acquire an abnormal image sample and a marked abnormal position feature and an abnormal type feature corresponding to the abnormal image sample; the marked abnormal position feature is used to indicate the abnormal position where abnormal pixel information exists in the abnormal image sample; the abnormal type feature is used to indicate the abnormal type of the abnormal pixel information; Adding original noise to the abnormal image sample to obtain a first noise image; Inputting the first noise image, the anomaly position feature and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position; Based on the predicted noise and the original noise, the diffusion model to be trained is adjusted until a trained diffusion model is obtained; the trained diffusion model can output predicted noise for an input normal image, and the predicted noise is used to generate an abnormal image.

[0006] In an optional implementation, the first noise image, the annotated abnormal position feature and the abnormal type feature are input into a diffusion model to be trained to obtain the predicted noise at the abnormal position, including: The anomaly position feature and the anomaly type feature are combined to obtain a first anomaly feature; The first noise image and the first abnormal feature are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

[0007] In an optional implementation, before inputting the first noise image, the annotated abnormal position feature and the abnormal type feature into the image generation model to be trained to obtain the predicted noise at the abnormal position, the method further includes: Obtain normal image samples and randomly set abnormal position features; Add the original noise to the normal image sample to obtain a second noisy image; Inputting the first noise image, the anomaly position feature and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position includes: The first noise image, the second noise image, the randomly set abnormal position features, the annotated abnormal position features and the abnormal type features are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

[0008] In an optional implementation, the first noise image, the second noise image, the randomly set abnormal position feature, the annotated abnormal position feature and the abnormal type feature are input into a diffusion model to be trained to obtain the predicted noise at the abnormal position, including: The anomaly position feature and the anomaly type feature are combined to obtain a second anomaly feature; The randomly set abnormal position feature and the abnormal type feature are combined to obtain a third abnormal feature; The first noise image, the second noise image, the second abnormal feature and the third abnormal feature are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

[0009] In an optional implementation, adjusting the diffusion model to be trained based on the predicted noise and the original noise includes: Determining a loss function value based on the predicted noise and the original noise; Based on the loss function value, the diffusion model is updated.

[0010] In an optional implementation, determining a loss function value based on the predicted noise and the original noise includes: A mean square error value between the predicted noise and the original noise is determined, and the mean square error value is used as the loss function value.

[0011] In a second aspect, the present disclosure also provides an image generation method, including: Acquire a normal image, as well as preset abnormal position features and preset abnormal type features; The normal image, random noise, the abnormal position feature and the abnormal type feature are input into a trained diffusion model to obtain a predicted noise at the abnormal position represented by the abnormal position feature, and an abnormal image is obtained based on the predicted noise; in the abnormal image, abnormal pixel information under the abnormal type represented by the abnormal type feature exists at the abnormal position represented by the abnormal position feature; The diffusion model is trained by using the steps in the first aspect or any possible implementation of the first aspect.

[0012] In an optional implementation, the normal image, random noise, the abnormal position feature and the abnormal type feature are input into a trained diffusion model to obtain a predicted noise at the abnormal position represented by the abnormal position feature, and based on the predicted noise, an abnormal image is obtained, including: In a time step T, random noise is added to the normal image to obtain a noise image corresponding to the time step T; the noise image corresponding to the time step T, the abnormal position feature and the abnormal type feature are input into the trained diffusion model to obtain the predicted noise at the abnormal position; T is a positive integer greater than or equal to 1; Based on the predicted noise and the abnormal position, denoising the noise image corresponding to the time step T to obtain a denoised image corresponding to the time step T; In time step N, random noise is added to the denoised image obtained in time step N+1, except for the abnormal position represented by the abnormal position feature, to obtain a noise image corresponding to the time step N; N is an integer greater than 1 or equal to 0 and less than T; Inputting the noise image corresponding to the time step N, the abnormal position feature and the abnormal type feature into the trained diffusion model to obtain the predicted noise at the abnormal position; Based on the predicted noise and the abnormal position, the noise image is denoised to obtain a denoised image corresponding to the time step N, and the above steps are repeated until the time step 0 is reached, and the denoised image obtained at the time step 0 is used as the abnormal image.

[0013] In an optional implementation manner, after obtaining the abnormal image, the method further includes: Extracting the abnormal type feature at the abnormal position represented by the abnormal position feature in the abnormal image; Comparing the extracted abnormal type feature with the preset abnormal type feature to obtain a similarity comparison result; When the similarity comparison result is less than a set threshold, the abnormal image is retained.

[0014] In a third aspect, the present disclosure also provides a model training device, including: An acquisition module is used to acquire an abnormal image sample and a marked abnormal position feature and an abnormal type feature corresponding to the abnormal image sample; the marked abnormal position feature is used to indicate an abnormal position where abnormal pixel information exists in the abnormal image sample; the abnormal type feature is used to indicate an abnormal type of the abnormal pixel information; An adding module, configured to add original noise to the abnormal image sample to obtain a first noise image; An input module, used for inputting the first noise image, the anomaly position feature and the anomaly type feature into a diffusion model to be trained to obtain a predicted noise at the anomaly position; An adjustment module is used to adjust the diffusion model to be trained based on the predicted noise and the original noise until a trained diffusion model is obtained; the trained diffusion model can output predicted noise for an input normal image, and the predicted noise is used to generate an abnormal image.

[0015] In a fourth aspect, the present disclosure also provides an image generating device, including: An acquisition module, used to acquire a normal image, as well as preset abnormal position features and preset abnormal type features; An input module, used for inputting the normal image, random noise, the abnormal position feature and the abnormal type feature into a trained diffusion model, obtaining a predicted noise at the abnormal position represented by the abnormal position feature, and obtaining an abnormal image based on the predicted noise; in the abnormal image, abnormal pixel information under the abnormal type represented by the abnormal type feature exists at the abnormal position represented by the abnormal position feature; The diffusion model is trained by using the steps in the first aspect or any possible implementation of the first aspect.

[0016] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect, are executed, or the above-mentioned second aspect, or the steps of any possible implementation of the second aspect, are executed.

[0017] In a sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computer program executes the steps of the above-mentioned first aspect, or any possible implementation of the first aspect, or executes the above-mentioned second aspect, or any possible implementation of the second aspect.

[0018] The present disclosure provides a model training method, which trains a diffusion model by using abnormal image samples and abnormal position features and abnormal type features annotated on the abnormal image samples, so that the diffusion model can learn the abnormal type representation and abnormal position representation of abnormal pixel information. Therefore, when abnormal images are generated using the trained diffusion model, abnormal images of specified abnormal types can be generated at specified abnormal positions, thereby achieving diversity and controllability of abnormal images. In addition, the present disclosure can generate abnormal images in batches through a small number of abnormal image samples, thereby quickly increasing the number of abnormal image samples.

[0019] For a description of the effects of the above-mentioned device, electronic device, and computer-readable storage medium, please refer to the description of the above-mentioned method, which will not be repeated here.

[0020] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic diagram of a flow chart of a model training method provided by an embodiment of the present disclosure is shown; Figure 2 A flow chart of another model training method provided by an embodiment of the present disclosure is shown; Figure 3 A flow chart of another model training method provided by an embodiment of the present disclosure is shown; Figure 4 A schematic diagram showing a flow chart of an image generating method provided by an embodiment of the present disclosure; Figure 5A schematic diagram showing a flow chart of another image generating method provided by an embodiment of the present disclosure; Figure 6 A schematic diagram of the process of generating and applying abnormal image samples provided by an embodiment of the present disclosure is shown; Figure 7 A schematic diagram of the structure of a model training device provided by an embodiment of the present disclosure is shown; Figure 8 A schematic diagram of the structure of an image generating device provided by an embodiment of the present disclosure is shown; Fig. 9 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0024] With the development of technology, the model has been widely used in many fields. For example, in industrial quality inspection scenarios, after taking a photo of the product, the abnormal image detection model can be used to detect the product photo.

[0025] Research has found that in the training stage of the abnormal image detection model, a large number of normal image samples and a large number of abnormal image samples are needed to train the model. However, among the collected image samples, most are normal image samples, and defective image samples account for only a small part. When the number of abnormal image samples is small, the trained abnormal image detection model cannot accurately detect the abnormalities in the image. In addition, in the current method of generating abnormal image samples, the abnormal type (label) is usually directly input into the model as a guiding condition to generate abnormal images. This causes the abnormal position of the generated abnormal image to be consistent with the abnormal position of the input abnormal image, lacking diversity and controllability. Therefore, how to obtain a large number of diverse abnormal image samples is a problem worthy of attention.

[0026] Based on this, the present disclosure provides a model training method, which trains a diffusion model by using abnormal image samples, as well as abnormal position features and abnormal type features annotated on the abnormal image samples, so that the diffusion model can learn the abnormal type representation and abnormal position representation of the abnormal pixel information. Therefore, when generating abnormal images using the trained diffusion model, abnormal images of the specified abnormal type can be generated at the specified abnormal position, thereby achieving the diversity and controllability of the abnormal images; in addition, the present disclosure can generate abnormal images in batches through a small number of abnormal image samples, thereby quickly increasing the number of abnormal image samples.

[0027] The defects existing in the above schemes and the proposed solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0029] To facilitate understanding of this embodiment, a model training method disclosed in the embodiment of the present disclosure is first introduced in detail. The executor of the model training method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities.

[0030] See also Figure 1 FIG. 1 is a flow chart of a model training method provided in an embodiment of the present disclosure, wherein the method includes S101 to S104, wherein: S101: Acquire abnormal image samples and annotated abnormal position features and abnormal type features corresponding to the abnormal image samples; the annotated abnormal position features are used to indicate the abnormal position where abnormal pixel information exists in the abnormal image samples; the abnormal type features are used to indicate the abnormal type of the abnormal pixel information.

[0031] S102: Add original noise to the abnormal image sample to obtain a first noise image.

[0032] S103: Inputting the first noise image, the anomaly position feature and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position.

[0033] S104: Based on the predicted noise and the original noise, the diffusion model to be trained is adjusted until a trained diffusion model is obtained; the trained diffusion model can output predicted noise for an input normal image, and the predicted noise is used to generate an abnormal image.

[0034] In the disclosed embodiment, some abnormal image samples may be collected in advance. In order to improve the prediction accuracy of the diffusion model, in one embodiment, the number of abnormal image samples may be increased by data enhancement. The data enhancement method may include but is not limited to splicing, deforming, cutting, changing the color of the abnormal image samples, etc.

[0035] The abnormal image samples may be labeled to obtain the labeled abnormal position information and abnormal type information of the abnormal image samples.

[0036] The anomaly position information may indicate the anomaly position of the abnormal pixel information in the abnormal image sample. The anomaly type information may indicate the anomaly type of the abnormal pixel information (also known as an abnormal label), such as cracks, gaps, missing characters, etc.

[0037] The anomaly location information can be expressed in the form of a mask binary image, and the anomaly type information can be expressed in the form of text.

[0038] Figure 2 A flowchart of another model training method provided in an embodiment of the present disclosure.

[0039] like Figure 2 As shown, after obtaining the anomaly position information, the anomaly position encoder can be used to encode the anomaly position information to obtain an anomaly position feature. In one embodiment, the anomaly position encoder can be a convolutional neural network (CNN), and the structure of the anomaly position encoder can include a backbone network: a residual network (Residual Network, ResNet), a feature pyramid network (Feature Pyramid Networks, FPN) and a projection layer: a multilayer perceptron (Multilayer Perceptron, MLP). The anomaly position information in the form of a masked binary image is input into the anomaly position encoder, and an anomaly position feature map can be obtained after feature extraction by the ResNet backbone network and the FPN feature pyramid network. After the anomaly position feature map is projected by the MLP projection layer, it can be converted into a one-dimensional vector as an abnormal position representation code, that is, an anomaly position feature. The obtained anomaly position feature can represent the abnormal position where abnormal pixel information exists in the abnormal image sample.

[0040] After obtaining the abnormal type information, the abnormal type encoder can be used to encode the abnormal type information to obtain the abnormal type representation code. That is, the abnormal type feature. In one embodiment, the abnormal type encoder can be a text encoder based on a transformer encoder, and the abnormal type information represented in text form can be input into the abnormal type encoder and converted into a word embedding vector. In a specific implementation, a contrastive language-image pretraining (CLIP) encoder can be used. During the training phase, all parameters of the layers except the input embedding conversion layer in the CLIP encoder can be frozen. Based on the textual inversion technology, the embedding representation of the abnormal type is obtained by training. The obtained word embedding vector can be used as an abnormal type representation code, that is, an abnormal type feature. The abnormal type feature can be used to represent the abnormal type of abnormal pixel information present in the abnormal image sample.

[0041] In the embodiment of the present disclosure, the number of network channels for representing the exception type and the number of network channels for representing the exception position can be made consistent by designing a suitable number of network channels, for example, setting the number of network channels to 128.

[0042] The original noise added to the abnormal image sample may be random noise. In one implementation, the original noise may be Gaussian noise.

[0043] The diffusion model to be trained may be a pre-trained diffusion model. The pre-trained diffusion model may be a latent diffusion model trained using collected normal image samples, or a model trained based on a general open image set.

[0044] In one embodiment, inputting the first noise image, the annotated abnormal position feature, and the abnormal type feature into a diffusion model to be trained to obtain a predicted noise at the abnormal position may include: concatenating the annotated abnormal position feature and the abnormal type feature to obtain a first abnormal feature; inputting the first noise image and the first abnormal feature into a diffusion model to be trained to obtain a predicted noise at the abnormal position.

[0045] In the above implementation, the code representing the marked abnormal position and the code representing the abnormal type may be concatenated to obtain the abnormal feature code, that is, the first abnormal feature.

[0046] The image obtained by encoding the first noise image using the image encoder represents the encoding, that is, the image features corresponding to the first noise image.

[0047] The abnormal feature representation code (i.e., the first abnormal feature) can be used as a conditional guide and input into the diffusion model to be trained together with the image representation code obtained by encoding the first noise image (i.e., the image feature corresponding to the first noise image). After the first abnormal feature and the image feature are cross-attention calculated in the diffusion model, the diffusion model can output the predicted noise located at the abnormal position.

[0048] Based on the predicted noise and the original noise, the loss function value can be obtained. Based on the abnormal position feature, a local mask of the abnormal image sample is implemented, so the loss function value obtained here is a local loss function value. According to the local loss function value, the diffusion model to be trained can be adjusted until a trained diffusion model is obtained.

[0049] In order to enable the diffusion model to learn abnormal positions of more shapes, the diffusion model can be trained using normal image samples and abnormal image samples.

[0050] In one embodiment, before the first noise image, the annotated abnormal position features and the abnormal type features are input into the image generation model to be trained to obtain the predicted noise located at the abnormal position, the method also includes: obtaining normal image samples and randomly set abnormal position features; adding the original noise to the normal image samples to obtain a second noise image.

[0051] Inputting the first noise image, the annotated abnormal position feature, and the abnormal type feature into the diffusion model to be trained to obtain the predicted noise located at the abnormal position may include: inputting the first noise image, the second noise image, the randomly set abnormal position feature, the annotated abnormal position feature, and the abnormal type feature into the diffusion model to be trained to obtain the predicted noise located at the abnormal position.

[0052] Here, you can randomly set the abnormal position information (abnormal position mask) and set the abnormal type information to an empty string. By randomly setting the abnormal position information, the diffusion model can learn abnormal position masks of more shapes during the training phase.

[0053] Figure 3 A flowchart of another model training method provided in an embodiment of the present disclosure. Figure 3 The process of training the diffusion model using normal image samples and abnormal image samples is shown.

[0054] like Figure 3As shown, the randomly set abnormal position information is input into the abnormal position encoder for encoding to obtain the randomly set abnormal position representation code, that is, the randomly set abnormal position feature.

[0055] The original noise is added to the normal image sample to obtain the second noisy image.

[0056] During the training phase, the first noise image, the second noise image, the randomly set abnormal position features, the annotated abnormal position features and the abnormal type features can be input into the diffusion model to be trained to obtain the predicted noise at the abnormal position.

[0057] In one embodiment, the first noise image, the second noise image, the randomly set abnormal position feature, the annotated abnormal position feature and the abnormal type feature are input into the diffusion model to be trained to obtain the predicted noise located at the abnormal position, which may include: splicing the annotated abnormal position feature and the abnormal type feature to obtain the second abnormal feature; splicing the randomly set abnormal position feature and the abnormal type feature to obtain the third abnormal feature; inputting the first noise image, the second noise image, the second abnormal feature and the third abnormal feature into the diffusion model to be trained to obtain the predicted noise located at the abnormal position.

[0058] In the above implementation, the set empty string exception type information and the marked exception type information are input into the exception type encoder to obtain the exception type representation code, which may be the same as the exception type representation code obtained by encoding the marked exception type information.

[0059] The anomaly position representation code and the anomaly type representation code may be concatenated to obtain an anomaly feature representation code, namely the second anomaly feature. The randomly set anomaly position representation code and the anomaly type representation code may be concatenated to obtain another anomaly feature representation code, namely the third anomaly feature.

[0060] It should be noted here that the splicing process of annotated abnormal position features and abnormal type features and the splicing process of randomly set abnormal position features and abnormal type features do not occur simultaneously. In the case where the training samples contain normal image samples and abnormal image samples, if the diffusion model receives annotated abnormal position features at the current moment, the annotated abnormal position features and abnormal type features are spliced; if the diffusion model receives randomly set abnormal position features at the current moment, the randomly set abnormal position features and abnormal type features are spliced. The final abnormal features can include two abnormal features: the second abnormal feature and the third abnormal feature.

[0061] The image obtained by encoding the first noise image using the image encoder represents the encoding, that is, the image features corresponding to the first noise image. The image obtained by encoding the second noise image using the image encoder represents the encoding, that is, the image features corresponding to the second noise image.

[0062] The aforementioned abnormal feature representation codes (i.e., the second abnormal feature and the third abnormal feature) can be used as conditional guides and input into the diffusion model to be trained together with the image representation code obtained by encoding the first noise image (i.e., the image features corresponding to the first noise image) and the image representation code obtained by encoding the second noise image (i.e., the image features corresponding to the second noise image).

[0063] Based on the predicted noise and the original noise, a loss function value can be obtained. Based on the loss function value, the diffusion model to be trained can be adjusted until a trained diffusion model is obtained.

[0064] In one embodiment, adjusting the diffusion model to be trained based on the predicted noise and the original noise includes: determining a loss function value based on the predicted noise and the original noise; and updating the diffusion model based on the loss function value.

[0065] Based on the predicted noise and the original noise, the loss function value can be obtained. Based on the abnormal position feature, a local mask of the abnormal image sample is implemented, so the loss function value obtained here is a local loss function value. According to the local loss function value, the diffusion model to be trained can be adjusted until a trained diffusion model is obtained.

[0066] In one implementation, determining a loss function value based on the predicted noise and the original noise includes: determining a mean square error value between the predicted noise and the original noise, and using the mean square error value as the loss function value.

[0067] The above loss function value can be used for back propagation to adjust the diffusion model.

[0068] In the disclosed embodiment, the abnormal position encoder and the abnormal type encoder can be jointly trained with the diffusion model. The gradients of the abnormal position encoder and the abnormal type encoder can be updated by using the loss function value obtained above, thereby realizing the training of the abnormal position encoder and the abnormal type encoder.

[0069] In the application stage, using the trained diffusion model, an abnormal image with abnormal pixel information under the abnormal type represented by the abnormal type feature at the abnormal position represented by the abnormal position feature can be obtained according to the randomly input normal image, abnormal position feature and abnormal type feature. The specific process can refer to the steps of an image generation method provided in an embodiment of the present disclosure. The execution subject of the image generation method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities.

[0070] See also Figure 4 FIG. 4 is a flow chart of an image generation method provided by an embodiment of the present disclosure, wherein the method includes S401 to S402, wherein: S401: Acquire a normal image, as well as preset abnormal position features and preset abnormal type features.

[0071] S402: Input the normal image, random noise, the abnormal position feature and the abnormal type feature into a trained diffusion model to obtain the predicted noise at the abnormal position represented by the abnormal position feature, and obtain the abnormal image based on the predicted noise; in the abnormal image, at the abnormal position represented by the abnormal position feature, there is abnormal pixel information of the abnormal type represented by the abnormal type feature; the diffusion model is trained using the above-mentioned model training method.

[0072] In the disclosed embodiment, preset abnormal position information and preset abnormal type information can be obtained. The preset abnormal position information can be represented by a mask binary image. The preset abnormal type information can be represented in the form of text. Here, the abnormal position information and the abnormal type information can be preset according to the use scenario requirements. In the training stage, the diffusion model learns the abnormal features of various abnormal positions and various abnormal types. Therefore, in the application stage, by presetting the abnormal position information and the abnormal type, an abnormal image of a preset abnormal type can be generated at the preset abnormal position.

[0073] Figure 5 A flowchart of another image generation method provided in an embodiment of the present disclosure. Figure 5 The process of generating abnormal images using normal images is shown in FIG. Figure 5 As shown, the abnormal position encoder is used to encode the preset abnormal position feature to obtain the abnormal position representation code, that is, the preset abnormal position feature. The abnormal type encoder is used to encode the preset abnormal type feature to obtain the abnormal type representation code, that is, the preset abnormal type feature.

[0074] Then, the abnormal position representation code (i.e., abnormal position feature) and the abnormal type representation code (i.e., abnormal type feature) can be concatenated to obtain the abnormal feature representation code, i.e., the abnormal feature.

[0075] The normal image can be encoded using an image encoder to obtain an image representation code, that is, the image features corresponding to the normal image.

[0076] The image features, random noise and abnormal features corresponding to the normal image are input into the diffusion model trained according to the above method for sampling to obtain an abnormal image. In the obtained abnormal image, abnormal pixel information under the abnormal type represented by the abnormal type feature exists at the abnormal position represented by the abnormal position feature.

[0077] In the disclosed embodiment, the number of sampling steps can be set. During sampling, the abnormal position can be denoised in each time step until the preset number of steps is reached. Specifically, in one embodiment, the normal image, random noise, the abnormal position feature and the abnormal type feature are input into a trained diffusion model to obtain the predicted noise at the abnormal position represented by the abnormal position feature, and based on the predicted noise, the abnormal image is obtained, which may include: adding random noise to the normal image in time step T to obtain the noise image corresponding to the time step T; inputting the noise image corresponding to the time step T, the abnormal position feature and the abnormal type feature into the trained diffusion model to obtain the predicted noise at the abnormal position; T is a positive integer greater than or equal to 1; based on the predicted noise and the abnormal position, the noise image corresponding to the time step T is denoised. Processing is performed to obtain a denoised image corresponding to the time step T; within time step N, random noise is added to the denoised image obtained at time step N+1, except for the abnormal position represented by the abnormal position feature, to obtain a noise image corresponding to the time step N; N is an integer greater than or equal to 0 and less than T; the noise image corresponding to the time step N, the abnormal position feature and the abnormal type feature are input into the trained diffusion model to obtain a predicted noise at the abnormal position; based on the predicted noise and the abnormal position, the noise image is denoised to obtain a denoised image corresponding to the time step N, and the above steps are repeated until time step 0 is reached, and the denoised image obtained at time step 0 is used as the abnormal image.

[0078] In the above implementation, for time step T, random noise can be added to the normal image within time step T to obtain the noise image corresponding to time step T; the noise image corresponding to time step T, the abnormal position feature and the abnormal type feature are input into the trained diffusion model to obtain the predicted noise at the abnormal position.

[0079] Then, based on the predicted noise, the abnormal position in the noise image corresponding to time step T can be denoised, and the area other than the abnormal position in the denoised image corresponding to time step T still retains the original pixel information, thereby obtaining the denoised image corresponding to time step T.

[0080] Starting from the next time step, for the current time step (i.e., time step N, where N is an integer greater than or equal to 0 and less than T), the input in the diffusion model is the noise image with random noise added to the area other than the abnormal position in the first denoised image obtained in the previous time step (N+1), the abnormal position feature and the abnormal type feature, and the output is the predicted noise at the abnormal position. That is, for the abnormal position, the input in each time step is the pixel information after denoising in the previous time step, and for the area other than the abnormal position, the input in each time step is the pixel information with random noise added to the original normal image. Through this local redrawing mask method, conditional guided generation can be achieved at the abnormal position, while the original pixel information remains unchanged in the area outside the abnormal position.

[0081] After obtaining the predicted noise at the abnormal position, the abnormal position of the noise image can be denoised, and the pixel information of the area outside the abnormal position still maintains the original pixel information, and the denoised image corresponding to the current time step is obtained. In the denoised image corresponding to the current time step, the pixel information at the abnormal position is more different from the denoised image corresponding to the previous time step, and the pixel information of the area outside the abnormal position still maintains the original pixel information. After multiple iterations, in the denoised image obtained at time step 0, abnormal pixel information under the abnormal type can appear at the abnormal position.

[0082] According to the above method, a preset number of abnormal images can be obtained.

[0083] After obtaining the abnormal image, the quality of the abnormal image can be evaluated. In one embodiment, the abnormal type feature at the abnormal position represented by the abnormal position feature in the abnormal image can be extracted; the extracted abnormal type feature is compared with the preset abnormal type feature to obtain a similarity comparison result; if the similarity comparison result is less than a set threshold, the abnormal image is retained.

[0084] In the above implementation, the abnormal type information at the abnormal position can be input into the abnormal type encoder to obtain an embedded vector corresponding to the abnormal type feature, and the embedded vector is compared with the input vector corresponding to the preset abnormal type feature. In one implementation, the cosine distance between the embedded vector and the input vector can be calculated. If the cosine distance is less than a set threshold, it can be considered that the generated abnormal image meets the requirements, otherwise, the abnormal image can be discarded.

[0085] The abnormal images obtained according to the above-mentioned image generation method can be used as abnormal image samples, and together with normal image samples, the abnormal image detection model is trained to obtain a trained abnormal image detection model, thereby realizing the detection of abnormal images.

[0086] Figure 6 A schematic diagram of the process of generating and applying abnormal image samples provided in an embodiment of the present disclosure.

[0087] First, as many normal image samples and abnormal image samples as possible can be collected.

[0088] The abnormal image samples can be annotated to obtain the anomaly position information and the anomaly type information. Then, according to the anomaly position encoder and the anomaly type encoder, the anomaly position features and the anomaly type features of the anomaly are obtained.

[0089] Then, the diffusion model to be trained is trained using normal image samples and abnormal image samples, as well as the annotated abnormal position features and abnormal type features, to obtain a trained diffusion model. The training process can refer to the process of the aforementioned model training method, which will not be repeated here. In a specific implementation, a method of jointly training normal image samples and abnormal image samples can be adopted, or a method of training abnormal image samples alone can be adopted.

[0090] By inputting normal image samples, random or on-demand (i.e., preset) abnormal position features and abnormal type features into the trained diffusion model, abnormal image samples can be generated. The process of generating abnormal image samples can refer to the process of the aforementioned image generation method, which will not be repeated here.

[0091] Then, it can be determined whether the number of abnormal image samples reaches a set threshold. Here, the number of abnormal image samples can be the number of generated abnormal image samples, or the number of generated abnormal image samples plus the number of collected abnormal image samples.

[0092] If the set threshold is not reached, abnormal image samples will continue to be generated.

[0093] If the set threshold is reached, the abnormal image detection model is trained using the obtained abnormal image samples and the original normal image samples. The abnormal image samples here may include only the generated abnormal image samples, or may include the generated abnormal image samples and the collected abnormal image samples.

[0094] Finally, the trained abnormal image detection model can be used to perform the corresponding abnormal image detection function.

[0095] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0096] Based on the same inventive concept, a model training device corresponding to the model training method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned model training method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0097] Reference Figure 7 FIG. 1 is a schematic diagram of a structure of a model training device provided by an embodiment of the present disclosure, wherein the device comprises: The first acquisition module 701 is used to acquire an abnormal image sample and a marked abnormal position feature and an abnormal type feature corresponding to the abnormal image sample; the marked abnormal position feature is used to indicate the abnormal position where the abnormal pixel information exists in the abnormal image sample; the abnormal type feature is used to indicate the abnormal type of the abnormal pixel information; A first adding module 702, configured to add original noise to the abnormal image sample to obtain a first noise image; An input module 703 is used to input the first noise image, the anomaly position feature and the anomaly type feature into the diffusion model to be trained to obtain the predicted noise at the anomaly position; The adjustment module 704 is used to adjust the diffusion model to be trained based on the predicted noise and the original noise until a trained diffusion model is obtained; the trained diffusion model can output predicted noise for an input normal image, and the predicted noise is used to generate an abnormal image.

[0098] In an optional implementation, the input module 703 is specifically configured to: The anomaly position feature and the anomaly type feature are combined to obtain a first anomaly feature; The first noise image and the first abnormal feature are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

[0099] In an optional embodiment, before the input module 703 executes the step of inputting the first noise image, the annotated abnormal position feature and the abnormal type feature into the image generation model to be trained to obtain the predicted noise at the abnormal position, the device further includes: The second acquisition module is used to acquire normal image samples and randomly set abnormal position features; A second adding module, used for adding the original noise to the normal image sample to obtain a second noise image; The input module 703 is specifically used for: The first noise image, the second noise image, the randomly set abnormal position features, the annotated abnormal position features and the abnormal type features are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

[0100] In an optional implementation, the input module 703 is specifically configured to: The anomaly position feature and the anomaly type feature are combined to obtain a second anomaly feature; The randomly set abnormal position feature and the abnormal type feature are combined to obtain a third abnormal feature; The first noise image, the second noise image, the second abnormal feature and the third abnormal feature are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

[0101] In an optional implementation, the adjustment module 704 is specifically configured to: Determining a loss function value based on the predicted noise and the original noise; Based on the loss function value, the diffusion model is updated.

[0102] In an optional implementation, the adjustment module 704 is specifically configured to: A mean square error value between the predicted noise and the original noise is determined, and the mean square error value is used as the loss function value.

[0103] Based on the same inventive concept, an image generating device corresponding to the image generating method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image generating method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0104] Reference Figure 8 FIG. 1 is a schematic diagram of a structure of an image generating device provided by an embodiment of the present disclosure, wherein the device comprises: An acquisition module 801 is used to acquire a normal image, a preset abnormal position feature, and a preset abnormal type feature; An input module 802 is used to input the normal image, random noise, the abnormal position feature and the abnormal type feature into a trained diffusion model to obtain a predicted noise at the abnormal position represented by the abnormal position feature, and obtain an abnormal image based on the predicted noise; in the abnormal image, abnormal pixel information of the abnormal type represented by the abnormal type feature exists at the abnormal position represented by the abnormal position feature; The diffusion model is trained using the aforementioned model training method.

[0105] In an optional implementation, the input module 802 is specifically configured to: In a time step T, random noise is added to the normal image to obtain a noise image corresponding to the time step T; the noise image corresponding to the time step T, the abnormal position feature and the abnormal type feature are input into the trained diffusion model to obtain the predicted noise at the abnormal position; T is a positive integer greater than or equal to 1; Based on the predicted noise and the abnormal position, denoising the noise image corresponding to the time step T to obtain a denoised image corresponding to the time step T; In time step N, random noise is added to the denoised image obtained in time step N+1, except for the abnormal position represented by the abnormal position feature, to obtain a noise image corresponding to the time step N; N is an integer greater than or equal to 0 and less than T; Inputting the noise image corresponding to the time step N, the abnormal position feature and the abnormal type feature into the trained diffusion model to obtain the predicted noise at the abnormal position; Based on the predicted noise and the abnormal position, the noise image is denoised to obtain a denoised image corresponding to the time step N, and the above steps are repeated until the time step 0 is reached, and the denoised image obtained at the time step 0 is used as the abnormal image.

[0106] In an optional embodiment, the device further comprises: An extraction module, used for extracting the abnormal type feature at the abnormal position represented by the abnormal position feature in the abnormal image; A comparison module, used to compare the extracted abnormal type feature with the preset abnormal type feature to obtain a similarity comparison result; The retaining module is used to retain the abnormal image when the similarity comparison result is less than a set threshold.

[0107] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0108] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Fig. 9 As shown, it is a schematic diagram of the structure of the electronic device 900 provided in the embodiment of the present disclosure, including a processor 901, a memory 902, and a bus 903. Among them, the memory 902 is used to store execution instructions, including a memory 9021 and an external memory 9022; the memory 9021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 901, and the data exchanged with the external memory 9022 such as a hard disk. The processor 901 exchanges data with the external memory 9022 through the memory 9021. When the electronic device 900 is running, the processor 901 communicates with the memory 902 through the bus 903, so that the processor 901 executes the following instructions: Acquire an abnormal image sample and a marked abnormal position feature and an abnormal type feature corresponding to the abnormal image sample; the marked abnormal position feature is used to indicate the abnormal position where abnormal pixel information exists in the abnormal image sample; the abnormal type feature is used to indicate the abnormal type of the abnormal pixel information; Adding original noise to the abnormal image sample to obtain a first noise image; Inputting the first noise image, the anomaly position feature and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position; Based on the predicted noise and the original noise, the diffusion model to be trained is adjusted until a trained diffusion model is obtained; the trained diffusion model can output predicted noise for an input normal image, and the predicted noise is used to generate an abnormal image.

[0109] Alternatively, the processor 901 exchanges data with the external memory 9022 through the memory 9021. When the electronic device 900 is running, the processor 901 communicates with the memory 902 through the bus 903, so that the processor 901 executes the following instructions: Acquire a normal image, as well as preset abnormal position features and preset abnormal type features; The normal image, random noise, the abnormal position feature and the abnormal type feature are input into a trained diffusion model to obtain a predicted noise at the abnormal position represented by the abnormal position feature, and an abnormal image is obtained based on the predicted noise; in the abnormal image, abnormal pixel information under the abnormal type represented by the abnormal type feature exists at the abnormal position represented by the abnormal position feature; The diffusion model is trained using the model training method described above.

[0110] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the model training method or the steps of the image generation method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0111] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the model training method or the steps of the image generation method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0112] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0113] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0116] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0117] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. Model training method, characterized in that, include: Obtaining abnormal image samples and annotated abnormal position features and abnormal type features corresponding to the abnormal image samples; The anomaly position feature is used to indicate the anomaly position where the abnormal pixel information exists in the abnormal image sample; the anomaly type feature is used to indicate the anomaly type of the abnormal pixel information; Adding original noise to the abnormal image sample to obtain a first noise image; Inputting the first noise image, the anomaly position feature and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position; Based on the predicted noise and the original noise, the diffusion model to be trained is adjusted until a trained diffusion model is obtained; the trained diffusion model can output predicted noise for an input normal image, and the predicted noise is used to generate an abnormal image.

2. The method according to claim 1, characterized in that Inputting the first noise image, the anomaly position feature, and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position includes: splicing the anomaly position feature and the anomaly type feature to obtain a first anomaly feature; The first noise image and the first abnormality feature are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

3. The method according to claim 1, characterized in that Before inputting the first noise image, the anomaly position feature, and the anomaly type feature into the image generation model to be trained to obtain the predicted noise at the anomaly position, the method further includes: Obtain normal image samples and randomly set abnormal position features; Adding the original noise to the normal image sample to obtain a second noisy image; Inputting the first noise image, the anomaly position feature, and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position includes: The first noise image, the second noise image, the randomly set abnormal position features, the annotated abnormal position features and the abnormal type features are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

4. The method according to claim 3, characterized in that Inputting the first noise image, the second noise image, the randomly set abnormal position features, the annotated abnormal position features, and the abnormal type features into a diffusion model to be trained to obtain predicted noise at the abnormal position, including: splicing the anomaly position feature and the anomaly type feature to obtain a second anomaly feature; splicing the randomly set abnormal position feature and the abnormal type feature to obtain a third abnormal feature; The first noise image, the second noise image, the second abnormal feature, and the third abnormal feature are input into a diffusion model to be trained to obtain predicted noise at the abnormal position.

5. The method according to any one of claims 1 to 4, characterized in that Adjusting the diffusion model to be trained based on the predicted noise and the original noise includes: Determining a loss function value based on the predicted noise and the original noise; The diffusion model is updated based on the loss function value.

6. The method according to claim 5, characterized in that Determining a loss function value based on the predicted noise and the original noise includes: A mean square error value between the predicted noise and the original noise is determined, and the mean square error value is used as the loss function value.

7. An image generation method, characterized in that include: Acquire a normal image, as well as preset abnormal position features and preset abnormal type features; Inputting the normal image, random noise, the abnormal position feature, and the abnormal type feature into a trained diffusion model to obtain predicted noise at the abnormal position represented by the abnormal position feature, and obtaining an abnormal image based on the predicted noise; in the abnormal image, abnormal pixel information of the abnormal type represented by the abnormal type feature exists at the abnormal position represented by the abnormal position feature; The diffusion model is trained using the model training method according to any one of claims 1 to 6.

8. The method according to claim 7, characterized in that Inputting the normal image, random noise, the abnormal position feature, and the abnormal type feature into a trained diffusion model to obtain predicted noise at the abnormal position represented by the abnormal position feature, and obtaining an abnormal image based on the predicted noise, including: In time step T, random noise is added to the normal image to obtain a noise image corresponding to the time step T; the noise image corresponding to the time step T, the abnormal position feature, and the abnormal type feature are input into the trained diffusion model to obtain the predicted noise at the abnormal position; T is a positive integer greater than or equal to 1; Based on the predicted noise and the abnormal position, denoising the noise image corresponding to the time step T to obtain a denoised image corresponding to the time step T; In time step N, random noise is added to the denoised image obtained in time step N+1, except for the abnormal position represented by the abnormal position feature, to obtain a noisy image corresponding to the time step N; where N is an integer greater than or equal to 0 and less than T; Inputting the noise image corresponding to the time step N, the abnormal position feature, and the abnormal type feature into the trained diffusion model to obtain the predicted noise at the abnormal position; Based on the predicted noise and the abnormal position, the noise image is denoised to obtain a denoised image corresponding to the time step N. The above steps are repeated until time step 0 is reached, and the denoised image obtained at time step 0 is used as the abnormal image.

9. The method according to claim 7 or 8, characterized in that After obtaining the abnormal image, the method further includes: Extracting the abnormal type feature at the abnormal position represented by the abnormal position feature in the abnormal image; Comparing the extracted abnormality type feature with the preset abnormality type feature to obtain a similarity comparison result; When the similarity comparison result is less than a set threshold, the abnormal image is retained.

10. Model training device, characterized in that, include: A first acquisition module is used to acquire abnormal image samples and annotated abnormal position features and abnormal type features corresponding to the abnormal image samples; The anomaly position feature is used to indicate the anomaly position where the abnormal pixel information exists in the abnormal image sample; the anomaly type feature is used to indicate the anomaly type of the abnormal pixel information; A first adding module, configured to add original noise to the abnormal image sample to obtain a first noise image; An input module, configured to input the first noise image, the anomaly position feature, and the anomaly type feature into a diffusion model to be trained to obtain predicted noise at the anomaly position; An adjustment module is used to adjust the diffusion model to be trained based on the predicted noise and the original noise until a trained diffusion model is obtained; the trained diffusion model can output predicted noise for an input normal image, and the predicted noise is used to generate an abnormal image.

11. An image generating device, characterized in that include: An acquisition module, used to acquire a normal image, as well as preset abnormal position features and preset abnormal type features; An input module is configured to input the normal image, random noise, the abnormal position feature, and the abnormal type feature into a trained diffusion model to obtain a predicted noise at the abnormal position represented by the abnormal position feature, and obtain an abnormal image based on the predicted noise; in the abnormal image, abnormal pixel information of the abnormal type represented by the abnormal type feature exists at the abnormal position represented by the abnormal position feature; The diffusion model is trained using the model training method according to any one of claims 1 to 6.

12. An electronic device, characterized in that include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus. When the machine-readable instructions are executed by the processor, the steps of the model training method according to any one of claims 1 to 6 are performed, or the steps of the image generation method according to any one of claims 7 to 9 are performed.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the model training method according to any one of claims 1 to 6, or executes the steps of the image generation method according to any one of claims 7 to 9.

Images