Model training method and device for defect generation and electronic equipment
By separating the appearance features and spatial location of defects using diffusion models and text inversion techniques, the problem of insufficient accuracy in generating defects by deep learning models in small sample scenarios is solved, achieving efficient and low-cost defect image generation and detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU MEGAROBO TECH CO LTD
- Filing Date
- 2025-12-15
- Publication Date
- 2026-04-17
AI Technical Summary
Deep learning models exhibit a significant decline in defect generation performance in small sample scenarios, with insufficient spatial alignment accuracy between generated defects and their corresponding masks, resulting in low accuracy.
By employing a diffusion model combined with spatial anomaly embedding and text inversion techniques, the appearance features and spatial location information of defects are separated, and high-fidelity defect images are generated from a small number of defect samples.
It improves the accuracy and flexibility of defect image generation, reduces data acquisition and annotation costs, and generates defect images that highly match real-world scenes, thus enhancing the performance of subsequent detection tasks.
Smart Images
Figure CN121884028A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of defect detection technology, and more specifically, relates to a model training method, apparatus and electronic equipment for defect generation. Background Technology
[0002] With the rapid development of industrial automation and intelligent manufacturing, deep learning-based computer vision defect detection technology has become an indispensable key link in modern production processes. While these methods can significantly improve detection accuracy, they still face several challenges in practical applications: First, deep learning models typically require a large amount of labeled data for training, and data collection and labeling often consume significant time and manpower. Second, defect types in industrial scenarios are diverse and unevenly distributed, with a severe shortage of samples for some rare defects, making it difficult for the model to fully learn these features and ultimately affecting overall detection performance.
[0003] Among the related defect generation methods, Generative Adversarial Networks (GANs) rely on a large number of labeled samples for model training. In scenarios with small sample sizes, their performance drops significantly, and the spatial alignment accuracy between generated defects and corresponding masks is insufficient, resulting in low accuracy. Summary of the Invention
[0004] The purpose of this application is to provide a model training method, apparatus, and electronic device for defect generation, so as to improve the generation performance of defect images in small sample scenarios.
[0005] A first aspect of this application provides a model training method for defect generation, the method comprising: Obtain the defect sample image, the first mask corresponding to the defect sample image, and the initial defect appearance features of the defect sample image; The first mask is input into the initial model to obtain the initial spatial distribution features; The defect sample image, the first mask, the initial defect appearance features, and the initial spatial distribution features are input into the pre-trained diffusion model, so that the diffusion model outputs the generated image corresponding to the defect sample image through denoising. The initial defect appearance features and initial model are trained based on the generated images to obtain the target defect appearance features and target model, so as to generate defect images using the target defect appearance features and target model.
[0006] A second aspect of this application provides a method for generating defect images, including: Steps for obtaining the spatial distribution features of defects: acquire a defect-free sample image and a second mask; input the second mask into the target model to obtain the target spatial distribution features of defects in the target defect sample image to be generated; Steps for obtaining the appearance features of defects: Obtain the appearance features of the target defect in the target defect sample image; Defect image generation steps: Input the appearance features of the target defect, the spatial distribution features of the target, and the defect-free sample image into the pre-trained diffusion model to obtain the target defect sample image; The target defect appearance features and target model are obtained by training any model training method used for defect generation.
[0007] A third aspect of this application provides a model training apparatus for defect generation, comprising: The acquisition unit is used to acquire a defect sample image, a first mask corresponding to the defect sample image, and initial defect appearance features of the defect sample image; The input unit is used to input the first mask into the initial model to obtain the initial spatial distribution features; The input unit is also used to input the defect sample image, the first mask, the initial defect appearance features and the initial spatial distribution features into the pre-trained diffusion model, so that the diffusion model outputs the generated image corresponding to the defect sample image through denoising; The training unit is used to train the initial defect appearance features and the initial model based on the generated image to obtain the target defect appearance features and the target model, so as to generate a defect image using the target defect appearance features and the target model.
[0008] A fourth aspect of this application provides a defect image generation apparatus, comprising: The first acquisition unit is used to acquire a defect-free sample image and a second mask; the second mask is input into the target model to obtain the target spatial distribution features of defects in the target defect sample image to be generated; The second acquisition unit is used to acquire the appearance features of the target defect in the target defect sample image; The generation unit is used to input the appearance features of the target defect, the spatial distribution features of the target, and the defect-free sample image into the pre-trained diffusion model to obtain the target defect sample image. The target defect appearance features and target model are obtained by training any model training method used for defect generation.
[0009] A fifth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the methods described above.
[0010] A sixth aspect of the embodiments of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the methods described above.
[0011] In the solution provided in this application embodiment, the target defect appearance features and target model used for defect image generation are obtained based on defect sample images trained. The target model is used to obtain the target spatial distribution features of defects in the target defect sample image to be generated. The target spatial distribution features obtained by this application based on the trained target defect appearance features and target model can improve the accuracy of feature acquisition.
[0012] Furthermore, the separation of defect appearance features and spatial distribution features in this scheme allows for more flexible and precise control over the defect appearance features and spatial distribution features in the generated defect image during the defect image generation process, thereby improving the accuracy of defect image generation and enhancing the flexibility of the defect image generation process.
[0013] In addition, high-accuracy defect images can be achieved based on a small number of labeled samples, overcoming the technical problems of defect image generation methods in related technologies, such as significant performance degradation in small sample scenarios, insufficient spatial alignment accuracy between generated defects and corresponding masks, and low accuracy. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 This is a schematic flowchart of a model training method for defect generation provided in an embodiment of this application; Figure 2 A schematic diagram illustrating an image generation output process provided in an embodiment of this application; Figure 3 A schematic diagram illustrating a defect image generation method according to an embodiment of this application; Figure 4 A schematic diagram illustrating a defect image generation process according to an embodiment of this application; Figure 5 A schematic diagram of a defect generation model training device provided in an embodiment of this application; Figure 6 A schematic diagram of a defect image generation device provided in an embodiment of this application. Figure 7 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0016] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0017] Diffusion, as an emerging generative model, generates images through a stepwise denoising process, demonstrating great potential in terms of image generation quality and diversity.
[0018] In addition to the aforementioned problems, defect image generation methods in related technologies synthesize defect images by randomly cropping and pasting from existing defect or texture datasets. The resulting defect images have low realism and are difficult to match the defect distribution in real scenes.
[0019] This paper proposes a scheme that combines spatial anomaly embedding and textual inversion techniques to generate highly realistic defect images that are strictly aligned with the mask, even with a small number of defect samples, significantly improving the performance of subsequent detection tasks.
[0020] This application utilizes the concept of spatial anomaly embedding to separate the appearance features of defects from their spatial location information, thereby better controlling the generation process. It employs text inversion techniques to learn the defect concept, optimizes text embedding to capture the appearance features of defects, and enhances the realism and diversity of the generated data.
[0021] The proposed solution requires minimal data; only a small number of defect samples are needed to generate a large number of high-quality defect images, significantly reducing data acquisition and annotation costs. The generated defect images exhibit high realism; the defect data generated by the diffusion model combined with text inversion technology highly matches real-world scenes in appearance and spatial distribution. Furthermore, it can be applied to multiple fields; defect appearance features fine-tuned through text inversion in one scene can be applied to other scenes.
[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.
[0023] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a model training method for defect generation provided in an embodiment of this application. The method can be executed by any electronic device and may include the following steps: S101-S104: S101. Obtain the defect sample image, the first mask corresponding to the defect sample image, and the initial defect appearance features of the defect sample image; Optionally, the defect sample image can be a digital image captured by an imaging device that records abnormal areas of an industrial product; essentially, it is an image that deviates locally from a normal image. The abnormal area refers to the defective region of the industrial product.
[0024] In some optional embodiments of this application, the defect sample image includes at least one type of defect. The type can be: texture defect, dot / hole defect, or regional defect.
[0025] Texture defects include scratches or cracks, which appear as high-contrast fine line structures with sharp edges.
[0026] Point-like / pore defects include pores or pits, appearing as round or oval dark spots with high central darkness and gradually fading edges. Point-like / pore defects can also include inclusions, appearing as irregular bright spots (metallic inclusions) or dark spots (non-metallic inclusions), with significant differences in grayscale from the background.
[0027] Regional defects include delamination or peeling defects, which appear as patchy shadows with blurred edges in layered materials. Regional defects can also include stains / oxidation defects, which appear as color spots or texture variations without a fixed shape.
[0028] The defect sample image includes at least one type of defect, which allows the generated image of the diffusion model to also include at least one type of defect, improving the efficiency of image generation, making the generated image more realistic, and further improving the ability and comprehensiveness of the subsequent defect detection model to identify defects.
[0029] The defect sample image can be a random image from an existing defect image library.
[0030] A mask is a binary or probabilistic matrix used for labeling and locating regions. Its core function is to distinguish different regions in an image, such as defects and normal areas, through numerical labels. It typically refers to manually labeled or algorithmically generated binary / probabilistic defect region labels, which can be in the form of an H×W matrix, where 0 represents the background and 1 represents the defect region.
[0031] The aforementioned first mask can refer to the mask used to label and locate the defective region and the normal region in the defective sample image.
[0032] Defect appearance features characterize the texture, color, material, and other appearance properties of defects. They are the visual representation of abnormal areas in a product, and are essentially a mapping of localized physical or chemical inhomogeneities in the material within an image. They generally include texture defects, point / hole defects, and regional defects.
[0033] The aforementioned initial defect appearance feature refers to a vector describing the defect type and defect morphology. In some optional embodiments of this application, the initial defect appearance feature is determined by random defect description text information, and the parameters of the initial model are random values.
[0034] The initial defect appearance features are determined by random defect description text information, which can provide different starting points for the initial defect appearance features. This allows for a more comprehensive coverage of the diversity of initial defect appearance features during training, thereby improving the realism and accuracy of the trained target defect appearance features.
[0035] The parameters of the initial model are random values, which can prevent the network in the initial model from degenerating into a single feature extractor. This ensures that the initial model can learn different features and improves the accuracy of the trained target model when processing data.
[0036] The defect sample image can be one or multiple images.
[0037] S102. Input the first mask into the initial model to obtain the initial spatial distribution features; The initial model is used to map the first mask from the pixel space to the high-dimensional feature space to obtain the initial spatial distribution features.
[0038] S103. Input the defect sample image, the first mask, the initial defect appearance features and the initial spatial distribution features into the pre-trained diffusion model, so that the diffusion model outputs the generated image corresponding to the defect sample image through denoising. The initial defect appearance features and initial spatial distribution features can be used as conditional inputs for the diffusion model.
[0039] The generated image is essentially a generated defect image. This generated defect image integrates the spatial information from the first mask and the initial spatial distribution features, the initial defect appearance features, and the defect information from the defect sample image, resulting in a more realistic defect image.
[0040] The sampling method used in the process of generating the image corresponding to the defect sample image by the diffusion model is not limited in this application, such as DDPM (Denoising Diffusion Probabilistic Models), DDIM (Denoising Diffusion Implicit Models), and DLMS (Diffusion Langevin Monte Carlo Sampler).
[0041] S104. Based on the generated image, train the initial defect appearance features and the initial model to obtain the target defect appearance features and the target model, so as to generate a defect image using the target defect appearance features and the target model.
[0042] The target model is used to determine the target spatial distribution features of defects in the target defect sample image. Based on the target defect appearance features obtained by training and the target spatial distribution features obtained by the target model, this application can improve the accuracy of feature acquisition.
[0043] The separation of defect appearance features and spatial distribution features in this scheme allows for more flexible and precise control over the defect appearance features and spatial distribution features in the generated defect image during the defect image generation process. This further improves the accuracy of defect image generation and enhances the flexibility of the defect image generation process. Through the scheme of this application, high-accuracy defect images can be achieved based on a small number of labeled samples, overcoming the technical problems of significantly reduced performance, insufficient spatial alignment accuracy between generated defects and corresponding masks, and low accuracy in related defect image generation methods under small sample scenarios.
[0044] In some optional embodiments of this application, training the initial defect appearance features based on the generated image includes: training the initial defect appearance features based on the generated image using text inversion technology.
[0045] By using text inversion technology to train the initial defect appearance features based on the generated images, the determination of the initial defect appearance features can be more accurate and flexible during the training process, which can further enhance the realism and diversity of the subsequently generated defect images.
[0046] In some optional embodiments of this application, see [link to relevant documentation]. Figure 2 As shown, Figure 2 This is a schematic diagram illustrating an image generation output process provided in this application. Figure 2 In this context, S represents text description information including placeholders. This text description information is used to describe the defect type of the defect in the generated image, and the placeholders represent the defect type.
[0047] The word segmenter is used to segment text description information into discrete word units, and the word unit embedder is used to map the discrete word units into continuous vector representations, thereby capturing the semantic, syntactic and contextual information in the language.
[0048] In some optional embodiments of this application, in the aforementioned S104, training the initial defect appearance features and the initial model based on the generated image to obtain the target defect appearance features and the target model includes the following steps S1041-S1042: S1041. Determine the corresponding loss information based on the defect sample image and the generated image; In some optional embodiments of this application, in the aforementioned S1041, determining the corresponding loss information based on the defect sample image and the generated image includes: determining the corresponding loss information based on the defect region corresponding to the defect sample image and the generated image.
[0049] Optionally, the corresponding loss information can be determined based on the defect region corresponding to the defect sample image and the generated image, which can be achieved through the following formula:
[0050] in, This refers to loss information; mask can refer to the first mask, and y refers to the defect sample image.
[0051] To generate an image, where the generated image can be the image generated by the diffusion model at one sampling number during the image generation stage. In the process of image generation by the diffusion model, the sampling number refers to the number of iterations required for the model to progressively denoise the random noise and finally generate the image. The determination of the value depends on three parameters: the time step of the diffusion model, the noise seed, and the conditional input, that is, the aforementioned... refer to ,in, Here, t is the noise seed and t is the time step. As a conditional input, specifically, the loss information can be represented by the following formula:
[0052] In this application, only the loss information of the defective region needs to be calculated during the calculation, which ensures that the model only focuses on the defective region and can improve data processing efficiency.
[0053] S1042. Use the loss information to train the initial defect appearance features and the initial model to obtain the target defect appearance features and the target model.
[0054] Training the initial defect appearance features and initial model using loss information to obtain the target defect appearance features and target model means adjusting the parameters of the initial defect appearance features and initial model according to the principle of minimizing loss information, thereby obtaining the target defect appearance features and target model. Here, the target defect appearance features are the trained initial defect appearance features, and the target model is the trained initial model.
[0055] Optionally, training is completed when the loss information is less than a preset threshold, or when the number of iterations during training (i.e., the number of times the loss information is calculated) is greater than a preset number.
[0056] By using the loss information from defect sample images and generated images to train the target defect appearance features and target model, the training process of features and models can be transformed from an abstract goal, such as improving the accuracy of the trained features and models, into a specific mathematical optimization problem. This facilitates the automatic adjustment of parameters by the algorithm, ultimately achieving the directed evolution of the capabilities of features and models, and improving the training efficiency of features and models.
[0057] After training, we can obtain the target defect appearance features that can accurately describe the appearance features of the defect and the target model that can convert the defect mask into spatial location and morphological embedding, that is, spatial distribution features.
[0058] Please refer to Figure 3 , Figure 3 This diagram illustrates a defect image generation method according to an embodiment of this application. It can be executed by any electronic device. The defect image generation method can augment and expand existing limited defect data, improving the model's generalization ability and thus enhancing the final detection performance. The method may include the following steps: S301, Steps for obtaining spatial distribution characteristics of defects; Obtain a defect-free sample image and a second mask; input the second mask into the target model to obtain the target spatial distribution features of defects in the target defect sample image to be generated; S302, Steps for obtaining the visual characteristics of defects; Obtain the appearance features of the target defect in the target defect sample image; S303, Defect image generation steps; The target defect appearance features, target spatial distribution features, and defect-free sample images are input into a pre-trained diffusion model to obtain target defect sample images. The appearance features of the target defect and the target model are obtained by training using any of the methods mentioned above.
[0059] The target model is used to map the second mask from the pixel space to the high-dimensional feature space to obtain the target space distribution features.
[0060] The appearance features of a target defect characterize the texture, color, material, and other appearance characteristics of the defect in the target defect sample image. It is a visual representation of abnormal areas of the product, and its essence is the mapping of local physical or chemical inhomogeneities of the material in the image. It generally includes texture defects, point / hole defects, and regional defects.
[0061] A schematic diagram of the defect image generation steps can be found here. Figure 4 As shown.
[0062] Optionally, the method further includes performing the defect image generation step multiple times to obtain multiple different target defect sample images.
[0063] Optionally, the method further includes: using the target defect sample image as training data for the defect detection model, training the defect detection model to obtain the target defect detection model, the target defect detection model being used to identify defect information in the input defect image and output defect information, the defect information including the appearance information of the defect as well as the location and shape information of the defect.
[0064] Among them, the appearance information of the defect can refer to the appearance characteristics of the defect, and the location and shape information of the defect can refer to the spatial distribution characteristics of the defect.
[0065] In the solution provided in this application embodiment, the target defect appearance features and target model used for defect image generation are obtained based on defect sample images trained. The target model is used to obtain the target spatial distribution features of defects in the target defect sample image to be generated. The target spatial distribution features obtained by this application based on the trained target defect appearance features and target model can improve the accuracy of feature acquisition.
[0066] Furthermore, the separation of defect appearance features and spatial distribution features in this scheme allows for more flexible and precise control over the defect appearance features and spatial distribution features in the generated defect image during the defect image generation process, thereby improving the accuracy of defect image generation and enhancing the flexibility of the defect image generation process.
[0067] Furthermore, the separation of defect appearance features and spatial distribution features in this scheme allows for a more flexible defect image generation process. Based on a small number of labeled samples, high-accuracy defect images can be achieved, overcoming the technical problems of significant performance degradation, insufficient spatial alignment accuracy between generated defects and corresponding masks, and low accuracy in related defect image generation methods under small sample scenarios.
[0068] Corresponding to the model training method for defect generation in the above embodiment, Figure 5 This is a schematic diagram of a model training apparatus for defect generation provided in an embodiment of this application. The apparatus includes: The acquisition unit 51 is used to acquire a defect sample image, a first mask corresponding to the defect sample image, and initial defect appearance features of the defect sample image; Input unit 52 is used to input the first mask into the initial model to obtain the initial spatial distribution features; The input unit 52 is also used to input the defect sample image, the first mask, the initial defect appearance features and the initial spatial distribution features into the pre-trained diffusion model, so that the diffusion model outputs the generated image corresponding to the defect sample image through denoising; Training unit 53 is used to train the initial defect appearance features and the initial model based on the generated image to obtain the target defect appearance features and the target model, so as to generate a defect image using the target defect appearance features and the target model.
[0069] Optionally, when the aforementioned device is used to train the initial defect appearance features and the initial model based on the generated image to obtain the target defect appearance features and the target model, it is specifically used for: The corresponding loss information is determined based on the defective sample image and the generated image; The initial defect appearance features and the initial model are trained using the loss information to obtain the target defect appearance features and the target model.
[0070] Optionally, when the aforementioned device is used to determine the corresponding loss information based on the defect sample image and the generated image, it is specifically used for: The corresponding loss information is determined based on the defect region corresponding to the defect sample image and the generated image.
[0071] Optionally, the initial defect appearance features are determined by random defect description text information, and the parameters of the initial model are random values.
[0072] Optionally, the defect sample image includes defects of at least one category.
[0073] Optionally, when the aforementioned device is used to train the initial defect appearance features based on the generated image, it is specifically used to: train the initial defect appearance features based on the generated image using text inversion technology.
[0074] Corresponding to the defect image generation method in the above embodiments, Figure 6 This is a schematic diagram of a defect image generation apparatus according to an embodiment of the present application. The apparatus includes: The first acquisition unit 61 is used to acquire a defect-free sample image and a second mask; the second mask is input into the target model to obtain the target spatial distribution features of defects in the target defect sample image to be generated; The second acquisition unit 62 is used to acquire the appearance features of the target defect in the target defect sample image; The generation unit 63 is used to input the appearance features of the target defect, the spatial distribution features of the target, and the defect-free sample image into the pre-trained diffusion model to obtain the target defect sample image. The target defect appearance features and target model are obtained by training the model training method for defect generation in any embodiment.
[0075] Optionally, the aforementioned apparatus is further configured to: perform the defect image generation step multiple times to obtain multiple different target defect sample images.
[0076] Optionally, the aforementioned apparatus is further configured to: use the target defect sample image as training data for the defect detection model, train the defect detection model to obtain the target defect detection model, the target defect detection model is used to identify defect information in the input defect image, and output defect information, the defect information including the appearance information of the defect as well as the location and shape information of the defect.
[0077] See Figure 7 , Figure 7 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 7 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the units in the aforementioned device embodiments.
[0078] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0079] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.
[0080] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory.
[0081] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in the embodiments of this application, or they can execute the implementation methods of the electronic devices described in the embodiments of this application, which will not be repeated here.
[0082] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0083] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0084] This application provides a computer program product, which includes computer-executable instructions or a computer program. The computer-executable instructions or computer program are stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the methods described in the embodiments of this application.
[0085] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.
[0086] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0087] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.
[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.
[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0090] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A model training method for defect generation, characterized in that, The method includes: Acquire a defect sample image, a first mask corresponding to the defect sample image, and initial defect appearance features of the defect sample image; The first mask is input into the initial model to obtain the initial spatial distribution features; The defect sample image, the first mask, the initial defect appearance features, and the initial spatial distribution features are input into a pre-trained diffusion model, which then outputs a generated image corresponding to the defect sample image through denoising. The initial defect appearance features and the initial model are trained based on the generated image to obtain the target defect appearance features and the target model, so as to generate a defect image using the target defect appearance features and the target model.
2. The method as described in claim 1, characterized in that, The step of training the initial defect appearance features and the initial model based on the generated image to obtain the target defect appearance features and the target model includes: Based on the defective sample image and the generated image, the corresponding loss information is determined; The initial defect appearance features and the initial model are trained using the loss information to obtain the target defect appearance features and the target model.
3. The method as described in claim 2, characterized in that, The step of determining the corresponding loss information based on the defective sample image and the generated image includes: Based on the defective sample image and the corresponding defective region of the generated image, the corresponding loss information is determined.
4. The method as described in claim 1, characterized in that, The initial defect appearance features are determined by random defect description text information, and the parameters of the initial model are random values.
5. The method as described in claim 1, characterized in that, The defect sample images include defects of at least one category.
6. The method as described in claim 1, characterized in that, The training of the initial defect appearance features based on the generated image includes: The initial defect appearance features are trained based on the generated image using text inversion technology.
7. A method for generating defect images, characterized in that, include: Steps for obtaining spatial distribution features of defects: Obtain defect-free sample images and a second mask; The second mask is input into the target model to obtain the target spatial distribution features of the defects in the target defect sample image to be generated; Defect appearance feature acquisition step: Acquire the appearance features of the target defect in the target defect sample image; Defect image generation step: Input the appearance features of the target defect, the spatial distribution features of the target, and the defect-free sample image into a pre-trained diffusion model to obtain the target defect sample image; The appearance features of the target defect and the target model are obtained by training using the method of any one of claims 1-6.
8. The method as described in claim 7, characterized in that, The method further includes: By performing the defect image generation step multiple times, multiple different target defect sample images are obtained.
9. The method as described in claim 7 or 8, characterized in that, The method further includes: The target defect sample image is used as training data for the defect detection model. The defect detection model is trained to obtain the target defect detection model. The target defect detection model is used to identify defect information in the input defect image and output defect information. The defect information includes the appearance information of the defect as well as the location and shape information of the defect.
10. A model training device for defect generation, characterized in that, include: The acquisition unit is used to acquire a defect sample image, a first mask corresponding to the defect sample image, and initial defect appearance features of the defect sample image; The input unit is used to input the first mask into the initial model to obtain the initial spatial distribution features; The input unit is further configured to input the defect sample image, the first mask, the initial defect appearance features, and the initial spatial distribution features into a pre-trained diffusion model, so that the diffusion model outputs a generated image corresponding to the defect sample image through denoising; The training unit is used to train the initial defect appearance features and the initial model based on the generated image to obtain the target defect appearance features and the target model, so as to generate a defect image using the target defect appearance features and the target model.
11. A defect image generation apparatus, characterized in that, include: The first acquisition unit is used to acquire a defect-free sample image and a second mask; The second mask is input into the target model to obtain the target spatial distribution features of the defects in the target defect sample image to be generated; The second acquisition unit is used to acquire the appearance features of the target defect in the target defect sample image; The generation unit is used to input the appearance features of the target defect, the spatial distribution features of the target, and the defect-free sample image into a pre-trained diffusion model to obtain the target defect sample image; The appearance features of the target defect and the target model are obtained by training using the method of any one of claims 1-6.
12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6 or any one of claims 7 to 9.
13. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6 or any one of claims 7 to 9.