Image generation method and device, electronic equipment and storage medium
By using the generative model and the extraction model to generate sample images with target masks, the problem of lack of labels in the prior art is solved, and automatic labeling and improved production efficiency are achieved.
Patent Information
- Application Number
- CN202510012857.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-03
- Publication Date
- 2025-04-11
AI Technical Summary
The defective sample images generated in the prior art lack accurate labeling and require manual annotation, resulting in waste of time and low productivity.
By processing the category labels with the first generative model, the mask label is generated, the target features of the target image are extracted using the extraction model, and the sample image with the target mask is generated in combination with the second generative model, so that automatic annotation is achieved.
No manual annotation is required, which improves the efficiency of image generation and thus improves the production efficiency of industrial products.
Smart Images
Figure CN119810591A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure belongs to the field of computer technology, and specifically relates to an image generation method and device, an electronic device, and a storage medium. Background Art
[0002] With the development of computer technology, image generation models such as diffusion model (DM) have emerged. In the defect detection of industrial products, the diffusion model can be used to generate sample images with defects to improve the accuracy of defect detection of industrial products.
[0003] However, the sample images with defects currently generated do not have accurate labels to mark the types of defects, and need to be labeled manually, which greatly wastes time and is not conducive to improving production efficiency. Summary of the invention
[0004] The present disclosure aims to solve at least one of the technical problems existing in the prior art, and provides an image generation method and device, an electronic device, and a storage medium.
[0005] In a first aspect, an embodiment of the present disclosure provides an image generation method, the image generation method comprising:
[0006] Using the first generative model, the category labels are processed to generate mask labels;
[0007] Processing the target image using the extraction model to extract target features of the target image;
[0008] The mask label, the target feature and the original sample image are processed by using a second generation model to generate a sample image with a target mask.
[0009] In some embodiments, the method further comprises: processing the category label by using the first generation model to generate the mask label; and then:
[0010] The mask label is processed by using a control network to generate feature information, and the feature information is input into the second generation model.
[0011] In some embodiments, the step of processing the category label using the first generation model to generate the mask label includes:
[0012] Using a drawing method, a mask image corresponding to each of the category labels is generated;
[0013] Annotating defects on the mask image to generate the mask label;
[0014] Using the category label and the mask label, training the first generation model;
[0015] The first generation model is used to predict and generate the mask label according to the category label.
[0016] In some embodiments, the step of training the first generation model using the category and the mask label includes:
[0017] Performing text encoding processing on the category labels to generate text features;
[0018] Performing noise processing on the mask label to generate a noisy mask label;
[0019] Generate prediction noise according to the text feature and the noise mask label;
[0020] Using the diffusion model, calculate the loss value of the predicted noise and the original added noise;
[0021] Using the loss value, gradient back propagation is performed to optimize the first generation model.
[0022] In some embodiments, using the first generation model to predict and generate the mask label according to the category label includes:
[0023] Inputting the category label and random noise into an image segmentation model to predict the noise at the current moment;
[0024] Inputting the noise at the current moment into a diffusion model to obtain the noise at the previous moment;
[0025] The mask label is predicted and generated according to the noise at the current moment and the noise at the previous moment.
[0026] In some embodiments, the using of the second generation model to process the mask label, the target feature and the original sample image to generate the sample image with the target mask also includes:
[0027] A drawing method is adopted to generate images containing multiple preset masks;
[0028] Inputting the preset mask image and the corresponding preset mask label into a third generation model, and using the output features of the third generation model as feature priors;
[0029] The original sample image, the original sample image after noise addition and the feature prior are input into the second generation model to train the second generation model.
[0030] In a second aspect, an embodiment of the present disclosure provides an image generating device, the image generating device comprising:
[0031] A first generation module is configured to process the category label using a first generation model to generate a mask label;
[0032] An extraction module is configured to process the target image using the extraction model to extract target features of the target image;
[0033] The second generation module is configured to use a second generation model to process the mask label, the target feature and the original sample image to generate a sample image with a target mask.
[0034] In some embodiments, the image generating device further comprises:
[0035] The control module is configured to process the mask label and generate feature information by using a control network, and input the feature information into the second generation model.
[0036] In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising:
[0037] at least one processor; and
[0038] a memory communicatively connected to the at least one processor; wherein,
[0039] The memory stores one or more computer programs that can be executed by the at least one processor. The one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the image generating method provided above.
[0040] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the image generation method provided above. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 A flowchart of an image generation method provided in an embodiment of the present disclosure.
[0042] Figure 2 Schematic diagram of data transmission during the image generation process.
[0043] Figure 3 A schematic diagram of a process for generating mask labels.
[0044] Figure 4 A schematic diagram of a mask image.
[0045] Figure 5 Schematic diagram of the training process of the first generative model.
[0046] Figure 6 Schematic diagram of data transmission during the first generation model training process.
[0047] Figure 7 Schematic diagram of the inference process of the first generative model.
[0048] Figure 8 Schematic diagram of data transmission during the inference process of the first generation model.
[0049] Fig. 9 Schematic diagram of the training process of the second generative model.
[0050] Fig.10 A schematic diagram of a preset mask image.
[0051] Fig.11 A schematic diagram of the structure of an image generating device provided in an embodiment of the present disclosure.
[0052] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure. In the absence of conflict, the various embodiments of the present disclosure and the various features in the embodiments can be combined with each other.
[0054] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantitative restrictions, but indicate that there is at least one. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0055] The "multiple or several" mentioned in this disclosure refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0056] In related technologies, in industrial defect detection, the training of detection models often requires a large number of sample images with defects. Currently, sample images with defects generated by the diffusion model (DM) do not have accurate labels to mark the types of defects, and manual labeling is required, which greatly wastes time and is not conducive to improving production efficiency.
[0057] In order to solve at least one of the above-mentioned technical problems, the embodiments of the present disclosure provide an image generation method and device, an electronic device, and a storage medium. The image generation method and device, electronic device, and storage medium provided by the embodiments of the present disclosure will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0058] In a first aspect, the present disclosure provides an image generation method. Figure 1 A flowchart of an image generation method provided by an embodiment of the present disclosure is shown in FIG. Figure 2 This is a schematic diagram of data transmission during the image generation process, such as Figure 1 and Figure 2 As shown, the image generating method includes the following steps S101 to S103.
[0059] S101, using the first generation model, processing the category label to generate a mask label.
[0060] In the above step S101, the first generation model may be a generation model based on a generative adversarial network (GAN).
[0061] In machine learning, category labels refer to the method of digitally representing the target names of classification problems. When performing classification tasks, it is usually impossible to directly use words or names to train the model, so these targets need to be encoded. Specifically, there are many categories of industrial products, such as display panels, circuit boards, etc.
[0062] In the task of distinguishing display panels from circuit boards, display panels and circuit boards are categories, and the above categories need to be converted into digital form for calculation. For example, the display panel can correspond to (1, 0), and the circuit board can correspond to (0, 1). Each category is represented by a different number or matrix, which is the corresponding category label.
[0063] The mask image is to use the selected image, graphic or object to block the processed image (all or part) to control the image processing area or processing process. The mask can be used to shield certain areas on the image so that they do not participate in the processing or the calculation of processing parameters, or only the shielded area can be processed or counted.
[0064] The mask label is a mask image marked with defect types, including defects of the display panel, defects of the circuit board, etc., such as scratches, black spots, holes, etc.
[0065] There is a corresponding relationship between the category label and the mask label. The category label can be input into the first generation model to generate a variety of mask labels corresponding to the category label.
[0066] S102, using the extraction model, processing the target image to extract target features of the target image.
[0067] In the above step S102, the target image is a specific image that needs to be identified, classified, tracked or processed. For example, a local image in a display panel. The target image is a defect-free image, that is, a good product image. The target features of the target image can be extracted using an extraction model.
[0068] S103, using the second generation model, processing the mask label, the target feature and the original sample image to generate a sample image with a target mask.
[0069] In the above step S103, the mask label, target feature and original sample image are input into the second generation model, and a sample image with a target mask is generated according to the mask label, target feature and original sample image, wherein the target mask is the marked defect type, such as scratches, black spots, holes, etc.
[0070] In the image generation method provided by the embodiment of the present disclosure, a mask label is generated using the first generation model, and a target feature of the target image is extracted using the extraction module. The mask label, the target feature and the original sample image are input into the second generation model to generate a sample image with a target mask, wherein the target mask is the marked defect type. In this way, it is no longer necessary to manually mark the defects on the generated sample image, thereby improving the efficiency of image generation and further improving the production efficiency of industrial products (display panels, circuit boards, etc.).
[0071] In some embodiments, Figure 1 and Figure 2As shown, the above step S101 uses the first generation model to process the category label to generate the mask label, and then also includes: step S101a, uses the control network to process the mask label and generate feature information, and inputs the feature information into the second generation model.
[0072] In the above step S101a, the generated sample image is controlled by using a control network branch, so that the position where the target mask is generated in the sample image can be accurately known, thereby further improving the accuracy of the generated sample image.
[0073] In some embodiments, Figure 3 A schematic diagram of a process for generating mask labels is shown in FIG. Figure 3 As shown, the above step S101 uses the first generation model to process the category label to generate a mask label, which specifically includes: steps S1011 to S1014.
[0074] S1011, using a drawing method, generating a mask image corresponding to each category label one by one.
[0075] In the above step S1011, a polygon drawing method is used to draw a polygon white area of the corresponding category on a completely black image of the same size as the original sample image for training. Each target image on the original sample image corresponds to a mask image (the mask image is as follows Figure 4 As shown in the figure), the position in the mask image corresponding to the target image is the white area.
[0076] S1012: Annotate the defects of the mask image and generate a mask label.
[0077] In the above step S1012, defects are annotated on the mask image, that is, text expressions are added, such as defect 1, defect 2, defect 3, which may specifically be scratches, black spots, holes, etc., to generate a mask label.
[0078] S1013: Train the first generation model using the category labels and mask labels.
[0079] In the above step S1013, the first generation model can be trained so that the first generation model learns the correspondence between the category labels and the mask labels, so that even if only the category labels are input into the first generation model, diverse mask labels can be obtained.
[0080] S1014: using the first generation model, predict and generate a mask label according to the category label.
[0081] In the above step S1014, for the trained first generation model, a category label can be input to perform model inference to predict and generate corresponding diversified mask labels.
[0082] In some embodiments, Figure 5 It is a flowchart of the training process of the first generation model. Figure 6 This is a schematic diagram of data transmission during the first generation model training process, as shown in Figure 5 and Figure 6 As shown, the above step S1013 uses categories and mask labels to train the first generation model, which specifically includes: steps S1013A to S1013E.
[0083] S1013A, performing text encoding processing on the category label to generate text features.
[0084] In the above step S1013A, after the category label is input, it is encoded by a tokenizer and then by a text encoder to obtain the encoded text features.
[0085] S1013B, performing noise processing on the mask label to generate a noisy mask label.
[0086] In the above step S1013B, the mask label is an image label, and a noisy mask label can be obtained by adding randomly generated noise.
[0087] S1013C, generates prediction noise based on text features and noisy mask labels.
[0088] In the above step S1013D, the encoded text features and the noise mask labels are input into the image segmentation model (Unet), and the image segmentation model (Unet) can output the predicted noise.
[0089] S1013D, using the diffusion model, calculate the loss value of the predicted noise and the original added noise.
[0090] S1013E, using the loss value, perform gradient back propagation to optimize the first generation model.
[0091] In the above steps S1013D and S1013E, if the time is t, the noise can be obtained at time t-1 through the diffusion model, and the noise loss value can be calculated for the predicted noise and the original noise. The noise loss value can be used to perform gradient back propagation to optimize the first generation model and realize the training of the first generation model.
[0092] In some embodiments, Figure 7 It is a flowchart of the reasoning process of the first generation model. Figure 8This is a schematic diagram of data transmission during the first generation model reasoning process, as shown in Figure 7 and Figure 8 As shown, the above step S1014, using the first generation model, predicts and generates a mask label according to the category label, specifically includes: step S1014A to step S1014C.
[0093] S1014A, input the category label and random noise into the image segmentation model to predict the noise at the current moment.
[0094] S1014B, input the noise at the current moment into the diffusion model to obtain the noise at the previous moment.
[0095] S1014C, predict and generate a mask label based on the noise at the current moment and the noise at the previous moment.
[0096] It should be noted that the reasoning process of the first generative model is basically similar to its training process. The difference is that during the training process, the first generative model has learned the correspondence between the category label and the mask label. During the reasoning process, it is only necessary to input the category label to the first generative model, and then add noise to the category label to predict the corresponding mask label. The original input only has the category label and random noise, which are input into the trained image segmentation model (Unet) to predict the noise at the current time t, and then the noise at time t-1 is obtained through the diffusion model. This cycle is repeated until t=0, and the predicted mask label is the required mask label.
[0097] In some embodiments, the above step S103, using the second generation model, processes the mask label, the target feature and the original sample image to generate a sample image with a target mask, and also includes the step of training the second generation model. Fig. 9 It is a flowchart of the training process of the second generation model, such as Fig. 9 As shown, the training process of the second model includes the following steps S901 to S903.
[0098] S901, generating a plurality of preset mask images by drawing.
[0099] In the above step S901, a polygon drawing method is adopted to assign a value of 255 to the background area (not assigned a value of 0 in order to distinguish it from the above actual mask image), and only one defect type is retained on the white background, and a value of 0 is assigned to the area to form a black area, wherein the black area is the actual target. The generated preset mask image is as follows: Fig.10 shown.
[0100] S902: Input a preset mask image and a corresponding preset mask label into a third generation model, and use output features of the third generation model as feature priors.
[0101] In the above step S902, the above preset mask image and the corresponding preset mask label are input as a data set into the third generation model (VQGAN model) for training. The trained third generation model can obtain the required sample image by inputting only noise, and the output features of its encoder are taken as feature priors.
[0102] S903, inputting the original sample image, the noisy original sample image and the feature prior into the second generation model to train the second generation model.
[0103] In the above step S903, the original sample image, the noisy original sample image and the feature prior are input into the second generation model. Due to the input of the feature prior, the accuracy of the generated target can be further improved, and the quality of the generated sample image can be improved.
[0104] In a second aspect, an embodiment of the present disclosure provides an image generating device, Fig.11 A schematic diagram of the structure of an image generating device provided by an embodiment of the present disclosure is shown in FIG. Fig.11 As shown, the image generating device includes: a first generating module 111 , an extracting module 112 and a second generating module 113 .
[0105] The first generation module 111 is configured to process the category label using the first generation model to generate a mask label. The extraction module 112 is configured to process the target image using the extraction model to extract the target feature of the target image. The second generation module 113 is configured to process the mask label, the target feature and the original sample image using the second generation model to generate a sample image with a target mask.
[0106] The image generation device further includes: a control module 114. The control module 114 is configured to process the mask label and generate feature information by using a control network, and input the feature information into the second generation model.
[0107] It should be noted that the various modules in the above-mentioned image generation device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules to implement the image generation method provided in any of the above-mentioned embodiments. Its implementation principle and beneficial effects are the same as the implementation principle of the image generation provided in any of the above-mentioned embodiments, and will not be repeated here.
[0108] In a third aspect, an embodiment of the present disclosure provides an electronic device, Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown in FIG. Fig.12 As shown, the electronic device provided by the embodiment of the present disclosure includes: at least one processor 121; at least one memory 122, and one or more I / O interfaces 123, which are connected between the processor 121 and the memory 122; wherein the memory 122 stores one or more computer programs that can be executed by at least one processor 121, and the one or more computer programs are executed by at least one processor 121 so that the at least one processor 121 can execute the image generation method provided by any of the above embodiments.
[0109] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module to implement the image generation method provided in any of the above embodiments.
[0110] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor / processing core, implements the image generation method provided in any of the above embodiments. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.
[0111] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).
[0112] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.
[0113] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0114] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0115] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.
[0116] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0117] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0118] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0119] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0120] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.
Claims
1. An image generation method, characterized in that: The image generation method comprises: Using the first generative model, the category labels are processed to generate mask labels; Processing the target image using the extraction model to extract target features of the target image; The mask label, the target feature and the original sample image are processed by using a second generation model to generate a sample image with a target mask.
2. The image generation method according to claim 1, characterized in that: The method uses the first generation model to process the category label to generate the mask label, and then further includes: The mask label is processed by using a control network to generate feature information, and the feature information is input into the second generation model.
3. The image generation method according to claim 1, characterized in that: The method of using the first generation model to process the category label to generate the mask label includes: Using a drawing method, a mask image corresponding to each of the category labels is generated; Annotating defects on the mask image to generate the mask label; Using the category label and the mask label, training the first generation model; The first generation model is used to predict and generate the mask label according to the category label.
4. The image generation method according to claim 3, characterized in that: The using the category and the mask label to train the first generation model includes: Performing text encoding processing on the category labels to generate text features; Performing noise processing on the mask label to generate a noisy mask label; Generate prediction noise according to the text feature and the noise mask label; Using the diffusion model, calculate the loss value of the predicted noise and the original added noise; Using the loss value, gradient back propagation is performed to optimize the first generation model.
5. The image generation method according to claim 3, characterized in that: The using the first generation model to predict and generate the mask label according to the category label includes: Inputting the category label and random noise into an image segmentation model to predict the noise at the current moment; Inputting the noise at the current moment into a diffusion model to obtain the noise at the previous moment; The mask label is predicted and generated according to the noise at the current moment and the noise at the previous moment.
6. The image generation method according to claim 3, characterized in that: The method further includes: using the second generation model to process the mask label, the target feature and the original sample image to generate a sample image with a target mask; Using a drawing method, a plurality of preset mask images are generated; Inputting the preset mask image and the corresponding preset mask label into a third generation model, and using the output features of the third generation model as feature priors; The original sample image, the original sample image after noise addition and the feature prior are input into the second generation model to train the second generation model.
7. An image generating device, characterized in that: The image generating device comprises: A first generation module is configured to process the category label using a first generation model to generate a mask label; An extraction module is configured to process the target image using the extraction model to extract target features of the target image; The second generation module is configured to use a second generation model to process the mask label, the target feature and the original sample image to generate a sample image with a target mask.
8. The image generating device according to claim 7, characterized in that: The image generating device further comprises: The control module is configured to process the mask label and generate feature information by using a control network, and input the feature information into the second generation model.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the image generating method according to any one of claims 1 to 6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the image generation method according to any one of claims 1 to 6.