Method and apparatus for generating structured defects based on class and shape
By using a data generation framework based on a diffusion model, and leveraging mask constraints and category conditions, defect samples with reasonable structure, natural boundaries, and clear categories are generated. This solves the problem of insufficient defect data in industrial quality inspection and improves model training effectiveness and detection accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU TIANZHUN SOFTWARE CO LTD
- Filing Date
- 2026-04-01
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies suffer from limited defect data samples in industrial quality inspection, making data collection difficult. Furthermore, the generated defects differ significantly from real defects in terms of boundary morphology and texture fusion, and the background structure is prone to drift, resulting in poor model training performance, difficulty in distinguishing different types of defect features, and insufficient controllability and diversity of generated samples.
A data generation framework based on a diffusion model is adopted. By introducing mask constraints, category conditions and an improved loss function, the model is trained to focus on learning structural features in the defect region, suppress background changes, and generate defect samples with reasonable structure, natural boundaries and clear categories through mask boundary weighting and background consistency constraints.
It improves the quality of generated defect samples, reduces reliance on real defect data, expands the dataset size, enhances the effectiveness of model training and the accuracy of detection, and reduces data collection costs.
Smart Images
Figure CN122116047A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and data generation, and specifically relates to a method and apparatus for generating structured defects based on category and shape. Background Technology
[0002] In the field of industrial quality inspection, deep learning-based visual inspection models have been widely used. However, the performance of the models largely depends on high-quality defect data, while defect data in actual production environments usually has the following characteristics: (1) the number of defect samples is small, especially rare defect types are difficult to collect; (2) the distribution of defect categories is uneven, which can easily lead to the model being biased towards common defects; (3) the cost of collecting real defects is high and the cycle is long, and it is significantly limited by production conditions.
[0003] Existing data augmentation methods still have some problems in industrial scenarios: On the one hand, the generated defects differ from real defects in terms of boundary morphology and texture fusion, which can easily produce obvious boundary artifacts and affect the model training effect; on the other hand, some generation methods may cause unnecessary changes to the background region when generating defects, resulting in background structure drift and thus reducing the realism and usability of the generated images; in addition, existing methods often have difficulty in effectively distinguishing the features of different types of defects when generating defects, resulting in insufficient controllability and diversity of generated samples. Summary of the Invention
[0004] To overcome the shortcomings of the prior art, the present invention aims to provide a method and apparatus for generating structured defects based on category and shape, which can generate defect samples with reasonable structure, natural boundaries and controllable categories in a specified area of a normal workpiece image, thereby effectively solving the problems of small number of defect samples, difficulty in collection and uneven data distribution in industrial quality inspection projects.
[0005] Design Principles: This invention is based on a diffusion model-based data generation framework. By introducing mask constraints, category conditions, and an improved loss function, the model can focus on learning the structural features of defect regions during training, while suppressing unnecessary changes in the background region. During model training, a defect mask is used to spatially constrain the generated region, making the model more focused on reconstructing the defect region. By introducing defect category conditions, the model can distinguish the features of different defect categories, thereby improving the controllability and diversity of generated samples. Simultaneously, this invention applies weighted processing to the mask boundary region, making the model pay more attention to the defect edge structure during training, thus improving the transition effect between the generated defect and the background. Furthermore, a background consistency constraint is introduced into the loss function to ensure that the generated results outside the mask region are consistent with the original image, thereby reducing the risk of background structure drift. Through the above technical means, defect samples with reasonable structure, natural boundaries, and clear categories can be generated, effectively improving the data quality of industrial vision model training and reducing the dependence on real defect data collection. The overall scheme is as follows.
[0006] A structured defect generation method based on category and shape is proposed. The method includes: S1, defective workpiece image acquisition and annotation: acquiring several images of defective workpieces, annotating the defective parts of each image, and obtaining a defective workpiece dataset Set1 containing defect category (Class) and defect mask (Mask1) information; S2, obtaining a defect generation model: improving the defect generation model structure based on a diffusion model, training it using the annotated defective workpiece dataset Set1. During training, the mask (Mask1) and defect category (Class) are input into the model, enabling the model to learn the reconstructed features of the defect region and the differential features of different defect categories; based on... Mask1 constructs the mask boundary region and assigns a higher weight to the mask boundary region than other regions to enhance the model's ability to reconstruct the boundary region; background consistency loss is introduced into the loss function to constrain the generated results of the region outside the mask, making them consistent with the original image; the defect generation model M is obtained through training; S3, obtain the normal workpiece dataset Set2: collect normal workpiece images to obtain the normal workpiece dataset Set2; S4, obtain the mask Mask2 for generation: perform graphical transformations on the defect mask Mask1 to expand its shape, to obtain the mask Mask2 for generation; graphical transformations include stretching, translation, rotation, affine transformation, etc.; S5, generate the defect dataset Set3: use the defect generation model M to redraw the images in the normal workpiece dataset Set2 to obtain the generated defect dataset Set3.
[0007] Furthermore, the mask Mask1 serves as the spatial constraint input model, and the defect category Class, after being encoded, serves as the category constraint input model.
[0008] Furthermore, during the image redrawing process in S5, candidate generation regions (ROIs) are specified or drawn in the normal workpiece image A1.
[0009] Furthermore, during defect region generation, the mask Mask2, defect category Class, normal workpiece image A1, and its candidate region ROI are input into the defect generation model M to obtain the reconstructed image A2; based on Mask2, the defect portion Q1 is cropped from the reconstructed image A2, and the defect portion Q1 is copied to the candidate generation region ROI of the normal workpiece image A1 to obtain the generated image A3.
[0010] Furthermore, the model structure of the defect generation model is an improvement on the diffusion model of the generative deep learning model; the background consistency loss is calculated on the region outside the mask during the model training process to make the background region generated by the model consistent with the original image, thereby reducing background drift.
[0011] Furthermore, the structured defect generation method also includes: S6, sample update, which involves classifying and storing or merging the generated defect dataset Set3 with the defect workpiece dataset Set1 to form an expanded dataset.
[0012] The present invention also provides a defect generation device, including an image acquisition unit, a memory, and a processor; the image acquisition unit is used to acquire workpiece images, the memory is used to store computer programs and image data, and the processor is used to execute the computer program, wherein the computer program is used to execute the aforementioned structured defect generation method.
[0013] Compared with existing technologies, the beneficial effects of this invention are as follows: Based on a diffusion model-based data generation framework, this invention introduces mask constraints, category conditions, and an improved loss function. During training, a defect mask is used to spatially constrain the generated region, and the mask boundary region is weighted, making the model pay more attention to the defect edge structure during training. Background consistency constraints are introduced into the loss function to ensure that the generated result outside the mask region is consistent with the original image, thereby reducing the risk of background structure drift. This improves the transition effect between the generated defect and the background, generating defect samples with reasonable structure, natural boundaries, and clear categories. This effectively improves the data quality of industrial vision model training, reduces the dependence on real defect data collection, and solves the problems of small number of defect samples, difficulty in collection, and uneven data distribution in industrial quality inspection projects. It can be widely applied in the field of quality inspection involving sample acquisition and image processing. Attached Figure Description
[0014] Figure 1 This is a flowchart of the structured defect generation method of the present invention; Figure 2 Flowchart for generating a single defect image. Detailed Implementation
[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.
[0017] A structured defect generation method based on category and shape, see [link to relevant documentation]. Figure 1 The method for generating structured defects includes the following steps.
[0018] S1. Image Acquisition and Annotation of Defective Workpieces. Several images of defective workpieces are acquired, and the defective regions in each workpiece image are annotated to obtain a defect mask Mask1. The corresponding defect category Class is then labeled for each defect, resulting in a defective workpiece dataset Set1 containing defect category information and defect mask information.
[0019] S2. Obtain the defect generation model. Improve the defect generation model structure based on the diffusion model, and train the model using the defective workpiece dataset Set1 to obtain the defect generation model M.
[0020] S3. Obtain the normal workpiece dataset Set2. Collect images of normal workpieces to obtain the normal workpiece dataset Set2.
[0021] S4. Obtain the mask Mask2 used for generation. Based on the defect mask Mask1, perform graphical transformations such as stretching, translation, rotation, and affine transformations to expand its shape and obtain the mask Mask2 used for generation.
[0022] S5. Generate defect dataset Set3. Use the defect generation model M to redraw the images in the normal workpiece dataset Set2 to obtain the generated defect dataset Set3.
[0023] S6. Sample Update: Store the generated defect dataset Set3 and defect workpiece dataset Set1 separately or merge them.
[0024] During the training of S2, to enable the model to focus more on defect regions and maintain background structure stability, the mask Mask1 of the labeled image is input as a spatial constraint, and the defect category Class is encoded and input as a category constraint, allowing the model to learn the structural features of defect regions and the differences between different defect categories. Simultaneously, to improve the transition between generated defects and the background, a mask boundary region is constructed based on Mask1 during training, and this boundary region is assigned a higher weight than other regions. Furthermore, a background consistency loss is introduced into the loss function to constrain the regions outside the mask, ensuring that the background region generated by the model remains consistent with the original image, thereby reducing background drift. After training, the defect generation model M is obtained.
[0025] In the image redrawing in S5, candidate generation regions (ROIs) are specified or drawn in the normal workpiece image A1. When generating the defect region, the mask Mask2, defect category Class, and the normal workpiece image A1 and its candidate ROIs are input into the defect generation model M to obtain the reconstructed image A2. The reconstructed image A2 is cropped according to the mask Mask2 to obtain the defect portion Q1. The defect portion Q1 is copied to the candidate generation region ROI of the normal workpiece image A1 to obtain the desired generated image A3.
[0026] Specific example: The model structure is an improvement on the front-diffusion model in generative deep learning models, used for image training and new image generation. See also... Figure 2 Taking any normal workpiece image A1 in dataset Set2 as an example: Candidate generation regions (ROIs) are manually drawn in normal workpiece image A1. Based on the defect mask Mask1, graphical transformations such as stretching, translation, rotation, and affine transformations are performed to obtain the generation mask Mask2, and the defect category Class is specified. Mask Mask2, the defect category Class, and normal workpiece image A1 and its candidate generation regions (ROIs) are input into the defect generation model M to obtain the reconstructed image A2. Specifically, during the redrawing process, non-defect target areas must remain unchanged. Therefore, after image reconstruction, the reconstructed image A2 is cropped along the contour of Mask2 to obtain the defect portion Q1, and the defect portion Q1 is copied to the candidate generation region (ROI) of normal workpiece image A1 to obtain the desired generated image A3.
[0027] This technology can simulate real defect features and create a large number of realistic and diverse virtual samples. It not only expands the scale of the dataset, but also improves the availability of samples of different defect categories, thereby reducing the dependence on expensive real test data, reducing data collection costs, and improving the effectiveness of model training and the accuracy of detection.
[0028] A defect generation device includes an image acquisition unit, a memory, and a processor; the image acquisition unit is used to acquire workpiece images, the memory is used to store computer programs and image data, and the processor is used to execute the computer program, which is used to execute the aforementioned structured defect generation method.
[0029] The present invention also provides a computer-readable storage medium storing computer instructions thereon, which, when executed, perform the steps of the aforementioned method. The method is described in detail in the foregoing section and will not be repeated here.
[0030] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, including permanent and non-permanent, removable and non-removable media. Information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.
[0031] The overall effect of this application is as follows.
[0032] (1) Improve the realism of defect boundaries.
[0033] By weighting the mask boundary region, the model pays more attention to the defect edge region during training, thereby improving the realism of the generated image.
[0034] (2) Keep the background structure stable.
[0035] By introducing background consistency loss, the model maintains consistency between the area outside the mask and the original image when generating defects, thereby avoiding background drift and improving the usability of generated samples.
[0036] (3) Enhance the controllability of defect categories.
[0037] By introducing defect category conditions, the model can generate defects with different category characteristics, thereby improving the diversity and controllability of the generated data.
[0038] (4) Improve data generation efficiency.
[0039] This invention can generate defect samples in a specified area of a normal workpiece image, reducing the cost of collecting real defects and shortening the data preparation cycle of industrial vision projects.
[0040] Based on the above advantages, the model structure can be widely applied in quality inspection fields involving image processing, sample acquisition, etc.
[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for generating structured defects based on category and shape, characterized in that, Methods for generating structured defects include: S1. Defective workpiece image acquisition and annotation: Acquire several defective workpiece images, annotate the defective parts of each workpiece image, and obtain a defective workpiece dataset Set1 containing defect category Class and defect mask Mask1 information; S2. Obtaining the Defect Generation Model: The defect generation model structure is improved based on the diffusion model. It is trained using the labeled defective workpiece dataset Set1. During training, the mask Mask1 and the defect category Class are input as conditions to the model, enabling it to learn the reconstruction features of the defect region and the difference features of different defect categories. A mask boundary region is constructed based on Mask1, and this boundary region is assigned a higher weight than other regions to enhance the model's ability to reconstruct the boundary region. Background consistency loss is introduced into the loss function to constrain the generation results outside the mask region, ensuring consistency with the original image. The defect generation model M is then obtained through training. S3. Obtain normal workpiece dataset Set2: Collect normal workpiece images to obtain normal workpiece dataset Set2; S4. Obtain the mask Mask2 for generation: Perform a graphical transformation based on the defect mask Mask1 to expand the shape and obtain the mask Mask2 for generation. S5. Generate defect dataset Set3: Use the defect generation model M to redraw the images in the normal workpiece dataset Set2 to obtain the generated defect dataset Set3.
2. The method for generating structured defects according to claim 1, characterized in that: The mask Mask1 serves as the spatial constraint input model, and the defect category Class, after being encoded, serves as the category constraint input model.
3. The method for generating structured defects according to claim 1, characterized in that: During the image redrawing process in S5, candidate generation regions (ROIs) are specified or drawn in the normal workpiece image A1.
4. The method for generating structured defects according to claim 1, characterized in that: When generating the defect region, the mask Mask2, the defect category Class, and the normal workpiece image A1 and its candidate region ROI are input into the defect generation model M to obtain the reconstructed image A2. Based on Mask2, the defective part Q1 is cropped from the reconstructed image A2, and the defective part Q1 is copied to the candidate generation region ROI of the normal workpiece image A1 to obtain the generated image A3.
5. The method for generating structured defects according to claim 1, characterized in that: The model structure of the defect generation model is an improvement on the diffusion model of the generative deep learning model; Background consistency loss is calculated on the region outside the mask during model training to ensure that the background region generated by the model is consistent with the original image, thereby reducing background drift.
6. The method for generating structured defects according to claim 1, characterized in that: The methods for generating structured defects also include: S6. Sample update: The defect dataset Set3 and the defect workpiece dataset Set1 will be classified, stored or merged to form an expanded dataset.
7. A defect generation device, comprising an image acquisition unit, a memory, and a processor; characterized in that: The image acquisition unit is used to acquire images of the workpiece, the memory is used to store computer programs and image data, and the processor is used to execute the computer program, which is used to execute the structured defect generation method according to any one of claims 1-6.