Industrial defect image generation system and method based on deep learning

Abstract

The invention provides an industrial defect image generation system and method based on deep learning, and relates to the technical field of artificial intelligence. A defect form adaptive module, a physical attribute modulation module and a multi-mechanism fusion module are integrated in a defect image generation module; dynamically selecting a feature extraction unit according to the defect type label based on a pre-constructed generative network model, and generating a defect feature map; generating an affine transformation parameter based on the physical attribute vector, and performing channel-by-channel linear modulation on a defect feature map of a middle layer of the generative network model, so that a multi-scale defect feature map finally generated by the generative network model contains specified defect type features and physical attribute features; and fusing the multi-scale defect feature image and the defect-free background image to obtain a fused defect image. The problems that in the prior art, industrial defect image generation is insufficient in sense of reality, poor in controllability and poor in fusion effect are solved, the method can be used for data enhancement of industrial visual inspection, and downstream model performance is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a system and method for generating industrial defect images based on deep learning. Background Art

[0002] In industrial visual object detection tasks, deep learning-based visual detection models rely on large-scale, diverse labeled data. However, in actual production, the collection of defect samples (especially rare defects) is costly and time-consuming, becoming a bottleneck restricting the improvement of model performance.

[0003] Current common image generation techniques (such as GANs and diffusion models) have several problems when directly used to generate industrial defects: 1) Insufficient realism, as general models struggle to capture the fine physical features of specific defects; 2) Poor controllability, as key physical attributes such as defect length and width cannot be quantitatively controlled; 3) Poor background fusion, easily producing visually unnatural "pasting" marks. These limitations make it difficult to effectively use the generated images for training downstream detection models.

[0004] Therefore, there is a need in the field for a technology that can generate realistic, controllable images of specific industrial defects that blend seamlessly with the background, in order to meet the demand for high-quality training data in industrial AI quality inspection. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies in generating industrial defect images, namely insufficient realism, controllability, and background blending effects. It provides a deep learning-based system and method for generating industrial defect images.

[0006] On one hand, embodiments of the present invention provide an industrial defect image generation system based on deep learning. The system includes: an input module, a defect image generation module, and an output module. The input module receives a defect-free background image, defect type labels, and physical attribute vectors. The defect image generation module integrates a defect morphology adaptation module, a physical attribute modulation module, and a multi-mechanism fusion module. The defect morphology adaptation module dynamically selects feature extraction units based on the defect type labels according to a pre-built generative network model to generate a defect feature map. The physical attribute modulation module generates affine transformation parameters based on the physical attribute vectors and performs channel-by-channel linear modulation on the defect feature map of the intermediate layer of the generative network model, so that the multi-scale defect feature map finally generated by the generative network model contains specified defect type features and physical attribute features. The multi-mechanism fusion module fuses the multi-scale defect feature map with the defect-free background image to obtain a fused defect image. The output module outputs the fused defect image.

[0007] Furthermore, the defect image generation module also integrates an encoder to convert the input defect type label and physical attribute vector into a condition vector that can be recognized and processed by the generative network model.

[0008] Furthermore, the feature extraction unit includes: a linear defect processing unit composed of anisotropic convolutional kernels, and a regional defect processing unit composed of multiple parallel receptive field branches; the linear defect processing unit is integrated in the U-Net architecture of the generative network model, integrating 1×N and N×1 anisotropic convolutional kernels in the shallow network of the U-Net architecture, and introducing an axial attention mechanism in the deep network to capture long-distance dependencies; the regional defect processing unit adopts three parallel branches, each using a different hole rate to expand the receptive field.

[0009] Furthermore, the defect image generation module also includes a segmented training module, which is used to train the pre-built generative network model based on a preset loss function and a segmented training strategy.

[0010] Furthermore, the preset loss function is:

[0011] L total =λ adv L adv +λ rec L rec +λ perc L perc +λ phy L phy ;

[0012] Where, L adv For adversarial loss, λ adv For the weights corresponding to the adversarial loss, L rec For pixel-level reconstruction loss, λ rec L represents the weights corresponding to the pixel-level reconstruction loss. perc For the perceptual loss based on deep features, λ perc L represents the weights corresponding to the perceptual loss based on deep features. phy For physical constraint loss, λ phy The weights are the weights corresponding to the physical constraint loss; the physical constraint loss is obtained by weighted summation of crack continuity and sharpness loss, region consistency loss and diversity enhancement loss.

[0013] Furthermore, the segmented training strategy includes: in the first stage, the generative network model is trained using a procedurally generated defect mask, with pixel-level reconstruction loss as the main optimization objective during training to optimize the macroscopic morphology of defects; in the second stage, the model is fine-tuned using real samples, and the complete composite loss function is activated to optimize microscopic texture and physical realism, with adversarial loss and physical constraint loss as the main optimization objectives during training to improve the realism and detail of generated defects.

[0014] Furthermore, the programmatically generated defect mask is generated using the following algorithms: linear defects are generated using an L-System, where the total length of the line segments is correlated with the length parameter in the physical property vector; regional defects are generated using Perlin noise, with the noise threshold set to T = k × Area. target In the formula, T is the noise threshold, k is the area calibration factor, and Area target This is the area parameter in the physical property vector.

[0015] Furthermore, the physical attribute modulation module inserts a FiLM layer after each residual block in the U-Net architecture to convert the physical attribute vector into a scaling factor γ and a bias β through a small parametric network. Based on a preset transformation formula: output = γ × defect feature map + β, a channel-level linear transformation is performed on the defect feature map so that the multi-scale defect feature map finally generated by the generative network model contains specified defect type features and physical attribute features.

[0016] Furthermore, the mechanism includes a mask-weighted fusion mechanism based on an auxiliary network, an adaptive fusion mechanism based on spatial attention, and a boundary post-processing mechanism based on image inpainting. The mask-weighted fusion mechanism based on an auxiliary network includes training an auxiliary network to generate an m-channel mask with smooth edges to control the fusion region between defects and the background. The adaptive fusion mechanism based on spatial attention includes learning a pixel-wise fusion weight map using a spatial attention mechanism, making the fusion effect adaptable to the local characteristics of the background and improving the naturalness of the fusion. The boundary post-processing mechanism based on image inpainting includes inputting the initially fused boundary region into the image inpainting model for post-processing to produce a more natural transition.

[0017] Secondly, embodiments of the present invention provide a deep learning-based industrial defect image generation method, applied to the aforementioned deep learning-based industrial defect image generation system. The method includes: receiving a defect-free background image, defect type labels, and physical attribute vectors; dynamically selecting feature extraction units based on the defect type labels according to a pre-built generative network model to generate a defect feature map; generating affine transformation parameters based on the physical attribute vectors, and performing channel-by-channel linear modulation on the defect feature map of the intermediate layer of the generative network model, so that the multi-scale defect feature map finally generated by the generative network model contains specified defect type features and physical attribute features; fusing the multi-scale defect feature map with the defect-free background image to obtain a fused defect image; and outputting the fused defect image.

[0018] In another aspect, the present invention also provides a computer-readable storage medium storing one or more instructions for causing a computer to execute the above-described deep learning-based industrial defect image generation method.

[0019] In another aspect, the present invention provides an electronic device, comprising: a memory and a processor; the memory storing at least one program instruction; the processor loading and executing the at least one program instruction to implement the above-described deep learning-based industrial defect image generation method.

[0020] The beneficial effects of this invention are as follows: A defect-specific network architecture (e.g., using anisotropic convolution to process cracks) provides the structural basis for generating defects of specific morphologies, determining "what" the model can generate. On the other hand, a loss function based on physical constraints (e.g., constraints on crack continuity) guides the optimization direction of the model, ensuring the physical realism of the generated results and determining "what" the model should generate. This coupled design of the architecture and loss function enables the model to effectively transform the user-specified macroscopic intent (e.g., parameters such as defect type and length) into a realistic image conforming to microscopic physical laws, overcoming the shortcomings of traditional methods that are either macroscopically uncontrollable or microscopically unrealistic. Furthermore, the following technical effects are achieved:

[0021] (1) The generated defects are closer to the real samples in terms of visual details and outperform the general generative model in objective evaluation indicators such as FID and KID.

[0022] (2) It can generate diverse samples based on parameters to cover rare forms in the real defect distribution.

[0023] (3) Users can control the key physical properties of generated defects such as length, width, and direction by setting parameters. The average error between the generated result and the set target can be controlled within a specific range (such as 5-10%).

[0024] (4) The application of the fusion module significantly reduced the synthesis traces and improved the availability of the synthesized data.

[0025] (5) After augmentation training using the data generated by this invention, key indicators such as the F1 score and rare defect recall rate of the downstream detection model are improved, while reducing the data collection cost. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings and examples.

[0027] Figure 1 This is a structural diagram of an industrial defect image generation system based on deep learning provided in Embodiment 1 of the present invention.

[0028] Figure 2 This is a schematic diagram of a preset generative network model architecture provided in Embodiment 1 of the present invention.

[0029] Figure 3 This is a schematic diagram of a defect-free background image provided in Embodiment 1 of the present invention.

[0030] Figure 4 This is a schematic diagram of a fused defect image provided in Embodiment 1 of the present invention.

[0031] Figure 5 This is a schematic diagram of a fusion technology process provided in Embodiment 1 of the present invention.

[0032] Figure 6 This is a flowchart of a deep learning-based industrial defect image generation method provided in Embodiment 2 of the present invention.

[0033] Figure 7 This is a partial block diagram of the electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0034] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0035] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0036] The present invention will now be described in detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0037] Example 1

[0038] To facilitate understanding, the working principle of this system is explained in general before describing the embodiments of the present invention in detail: This embodiment provides a deep learning-based industrial defect image generation system and method, including the following operation flow: 1. Defect morphology adaptive generation architecture: The generation network model includes a defect morphology adaptive module. This module dynamically selects or combines feature extraction paths with different convolution kernel morphologies according to the input defect type (e.g., linear defects or regional defects). This includes: when processing linear defects (e.g., cracks), this module activates feature extraction units composed of anisotropic convolution kernels; when processing regional defects (e.g., corrosion), this module activates multi-scale texture analysis units composed of multiple parallel receptive field branches. 2. Parameterized control of physical properties: The generation network model includes a physical property modulation module to achieve quantitative control of key physical properties (e.g., length, width, depth, etc.) of the generated defects. This module receives parameters representing physical properties and generates a set of affine transformation parameters to perform channel-wise linear modulation on the feature maps of the intermediate layers of the generation network. 3. Loss function based on physical constraints: The model training uses a composite loss function. This function introduces an additional physical constraint loss term in addition to the conventional adversarial loss and reconstruction loss. This loss term is used to impose explicit constraints on the geometry and topology of the generated defects. 4. Multi-mechanism fusion module: To naturally integrate the generated defects into the background image, the system integrates a fusion module containing at least one of the following mechanisms: weighted fusion based on m-masks generated by an auxiliary network, adaptive fusion based on pixel-wise fusion weights generated by a spatial attention network, or restorative fusion using an image inpainting model to post-process the fusion boundary. 5. Task-specific training strategy: A phased training scheme is adopted. This scheme uses manually labeled defect masks and a small number of real samples to resolve the optimization conflict between the accuracy of the macroscopic morphology of defects and the realism of the microscopic texture during model optimization.

[0039] The specific implementation method is as follows:

[0040] like Figure 1 The diagram shown is a structural diagram of an industrial defect image generation system based on deep learning provided by the present invention.

[0041] As an example, the system includes: an input module 1, a defect image generation module 2, and an output module 3; the input module 1 is used to receive a defect-free background image, a defect type label, and a physical attribute vector; the defect image generation module 2 integrates a defect morphology adaptation module 20, a physical attribute modulation module 21, and a multi-mechanism fusion module 22; the defect morphology adaptation module 20 is used to dynamically select feature extraction units based on the defect type label according to a pre-built generative network model to generate a defect feature map; the physical attribute modulation module 21 is used to generate affine transformation parameters based on the physical attribute vector, and perform channel-by-channel linear modulation on the defect feature map of the intermediate layer of the generative network model, so that the multi-scale defect feature map finally generated by the generative network model contains specified defect type features and physical attribute features; the multi-mechanism fusion module 22 is used to fuse the multi-scale defect feature map with the defect-free background image to obtain a fused defect image; the output module 3 is used to output the fused defect image.

[0042] Preferably, the defect type label includes 0 and 1, where 0 represents a linear defect and 1 represents a regional defect. Linear defects include, but are not limited to, cracks, scratches, inclusions, and delamination; regional defects include, but are not limited to, corrosion and bubbles. The physical attribute vector includes physical parameters such as length, width, depth, and orientation. Specifically, the user input includes: a defect-free semiconductor wafer surface image (smooth silicon substrate, uniform background texture); defect type: specified as "linear defect" (geometric defect); physical attribute parameters include: length: 50 μm (controlled by the total line segment length parameter of the L-System algorithm); width: 2 μm (adjusting the scale of the anisotropic convolution kernel); orientation: 30° angle with the wafer radial direction (mapped to a conditional vector via a multilayer perceptron); depth: simulating shallow surface cracks (controlled by linear modulation of the feature map of the physical attribute modulation module).

[0043] In some feasible implementations, the defect image generation module 2 also integrates an encoder 23, used to convert the input defect type label and physical attribute vector into conditional vectors that can be recognized and processed by the generative network model. Preferably, when processing geometric defects (such as cracks), a multilayer perceptron (MLP) can be used to map parameters such as length and orientation into high-dimensional conditional vectors.

[0044] In some feasible implementations, the feature extraction unit includes: a linear defect processing unit 201 composed of anisotropic convolutional kernels, and a regional defect processing unit 202 composed of multiple parallel receptive field branches; the linear defect processing unit 201 is integrated in the U-Net architecture of the generative network model, integrating 1×N and N×1 anisotropic convolutional kernels in the shallow network of the U-Net architecture, and introducing an axial attention mechanism in the deep network to capture long-distance dependencies; the regional defect processing unit 202 adopts three parallel branches, each using a different hole rate to expand the receptive field.

[0045] Preferred, combined Figure 2 As shown, the backbone network of the generative network model includes a U-Net structure based on the StableDiffusion diffusion model, whose depth and width can be adjusted according to the defect scale. To generate slender cracks, a linear defect processing unit 201 composed of 1xN and Nx1 anisotropic convolutional kernels is integrated into the shallow network of the U-Net. An axial attention mechanism is introduced into the deep network to capture long-distance dependencies and ensure the long-distance continuity of the crack. Specifically, 1×N and N×1 anisotropic convolutional kernels (such as 1×15 horizontal kernels and 15×1 vertical kernels) are used to extract the direction-sensitive features of slender defects such as cracks, and the horizontal and vertical features are combined to form a complete direction representation.

[0046] Preferably, the regional defect processing unit employs a three-way parallel branching approach, with each path using a different hole ratio to expand the receptive field. This includes: three parallel branches: small-scale (3×3 kernels), medium-scale (5×5 kernels), and large-scale (7×7 kernels); each path using a different hole ratio (1 / 2 / 4) to expand the receptive field; and merging multi-scale features to capture texture self-similarity. The defect morphology adaptive module 20 dynamically switches processing paths based on the input defect type to ensure that the network structure matches the physical morphology of the defect.

[0047] In some feasible implementations, the defect image generation module 2 further includes a segmented training module 24, which is used to train the pre-built generative network model based on a preset loss function and a segmented training strategy.

[0048] Preferably, the preset loss function is:

[0049] L total =λ adv L adv +λ rec L rec +λ perc L perc +λ phy L phy ;

[0050] Where, L advFor adversarial loss, λ adv For the weights corresponding to the adversarial loss, L rec For pixel-level reconstruction loss, λ rec L represents the weights corresponding to the pixel-level reconstruction loss. perc For the perceptual loss based on deep features, λ perc L represents the weights corresponding to the perceptual loss based on deep features. phy For physical constraint loss, λ phy The weights are the weights corresponding to the physical constraint loss; the physical constraint loss is obtained by weighted summation of crack continuity and sharpness loss, region consistency loss and diversity enhancement loss.

[0051] Specifically, adversarial loss (L adv ): Used to ensure that the generated image closely approximates the real defect in pixel distribution. For example, when the discriminator identifies an "artifact" of a synthetic crack, L adv Increasing the value forces the generative network model to adjust its feature extraction path (e.g., activating anisotropic convolutional kernels). Pixel-level reconstruction loss (L... rec ): The generated defect geometry (e.g., crack direction, corrosion region contour) must be substantially consistent with the input mask or real sample. For example, if the pixel difference between the programmatically generated crack mask and the generated result exceeds a threshold, L... rec Trigger backpropagation to correct the feature extraction parameters of U-Net. Perceptual loss (L perc This method extracts high-level features (such as texture roughness and edge gradients) between the generated defects and real samples based on a pre-trained visual model (e.g., VGG), forcing the minimization of the distance between their feature vectors. For example, when the edge gradient magnitude of the synthesized crack is lower than that of the real sample, L... perc Increased physical constraint loss (L) drives the network to learn sharper edge features. phy This loss term imposes an explicit constraint on the physical plausibility of the generated defects.

[0052] Specifically, the physical constraint loss includes crack continuity and sharpness loss, comprising continuity constraints and edge sharpness constraints. The continuity constraint involves extracting the skeleton from the generated crack mask (Skeleton algorithm) and calculating the number of skeleton breakpoints (i.e., disconnected endpoints) as a penalty to ensure crack continuity. The edge sharpness constraint calculates the image gradient magnitude of the crack edge region; a larger gradient value indicates a sharper edge, resulting in a smaller loss and encouraging the generation of clear edges. Its goal is to address crack breakage and edge blurring issues. Region consistency loss: For region-type defects such as erosion, the distribution of texture descriptors (e.g., LBP histograms) within the generated region is calculated, and the distance (e.g., Wasserstein distance) is calculated between this distribution and the distribution of similar descriptors in the real samples. A smaller distance results in a smaller loss, ensuring that the texture pattern is consistent with the real defect. Its purpose is to ensure that the texture of the generated region conforms to the physical evolution laws. Diversity enhancement loss: To avoid mode collapse, during training, a pair of samples can be generated using different noise levels for the same input condition, and the distance between this pair of samples in the intermediate feature layer of the discriminator is maximized. The goal is to force the model to cover morphological variants under the same parameters.

[0053] In some feasible implementations, the segmented training strategy includes: In the first stage, the generative network model is trained using procedurally generated defect masks. During training, pixel-level reconstruction loss is the primary optimization objective to optimize the macroscopic morphology of defects. In the second stage, the model is fine-tuned using real samples, activating the complete composite loss function to optimize microscopic texture and physical realism. During training, adversarial loss and physical constraint loss are the primary optimization objectives to improve the realism and detail of generated defects. Specifically, the goal of the first stage is to enable the model to quickly learn the basic geometric configuration and approximate range of physical properties of defects. This includes: using the L-System algorithm to generate over 100,000 procedural crack masks of different lengths and directions (coarse in detail but diverse in form); the loss function is primarily reconstruction loss, allowing the model to quickly learn the basic straight / curved form of cracks and avoid falling into a single pattern. In the second stage, based on the pre-trained model from the first stage, fine-tuning is performed using a small amount of real defect data or style-enhanced procedural data. This stage activates the complete composite loss function, focusing on optimizing the adversarial loss (L... adv ) and physical constraint loss (L phy This is done to enhance the realism and detail of generated defects.

[0054] More specifically, in the first stage, the focus is on reconstruction loss (weight λ). rec =0.6), quickly fitting the macroscopic shape of the defect, at which point the physical constraint loss weight is low (λ). phy =0.2), to avoid model convergence difficulties due to excessive detail requirements; Second stage (detail refinement): increase the weights of adversarial loss and physical constraint loss (λ).adv =0.4, λ phy =0.4), through L phy Forced microscopic features to conform to physical laws, while L adv To ensure visual realism, L at this time rec Weight reduction (λ) rec =0.3), which allows the model to optimize details while maintaining accuracy in macroscopic form.

[0055] Preferably, the programmatically generated defect mask is generated using the following algorithm: linear defects are generated using an L-System, where the total length of the line segments is associated with the length parameter in the physical property vector; regional defects are generated using Perlin noise, with the noise threshold set to T = k × Area. target In the formula, T is the noise threshold, k is the area calibration factor, and Area target This refers to the area parameter in the physical property vector. Specifically, procedural defect mask generation and enhancement includes generating diverse defect mask sketches using algorithms such as L-System, fractals, or Perlin noise. Algorithm parameters can be correlated with the physical properties of the defect to be generated (e.g., the number of iterations in the L-System correlates with the defect complexity) to achieve controllable data generation. For example, the length of a crack can be set by controlling the total length of line segments generated by the L-System or the scale parameter of the Perlin noise; the area of ​​the eroded region can be correlated by controlling the termination condition of the fractal iteration or the noise threshold.

[0056] In some feasible implementations, the physical attribute modulation module 21 inserts a FiLM layer after each residual block of the U-Net architecture to convert the physical attribute vector into a scaling factor γ and a bias β through a small parametric network; based on a preset transformation formula: output = γ × defect feature map + β, a channel-level linear transformation is performed on the defect feature map so that the multi-scale defect feature map finally generated by the generative network model contains specified defect type features and physical attribute features.

[0057] In some feasible implementations, the mechanism includes a mask-weighted fusion mechanism based on an auxiliary network, an adaptive fusion mechanism based on spatial attention, and a boundary post-processing mechanism based on image inpainting. The mask-weighted fusion mechanism based on an auxiliary network includes training an auxiliary network to generate an m-channel mask with smooth edges to control the fusion region between defects and the background. The adaptive fusion mechanism based on spatial attention includes learning a pixel-wise fusion weight map using a spatial attention mechanism to make the fusion effect adapt to the local characteristics of the background and improve the naturalness of the fusion. The boundary post-processing mechanism based on image inpainting includes inputting the initially fused boundary region into the image inpainting model for post-processing to produce a more natural transition.

[0058] Preferred, combined Figure 3-5 As shown, to avoid visual abruptness, the following fusion techniques can be used. A schematic diagram of the fusion technique process is shown below. Figure 5 Learning-based high-precision mask generation: Train an auxiliary network specifically for generating m-channel masks with smooth edges. Attention-guided adaptive fusion: Utilize spatial attention to learn a pixel-by-pixel fusion weight map, enabling the fusion effect to adapt to the local characteristics of the background. Image inpainting-based fusion: Post-process the initially fused boundary regions using an image inpainting model to produce a more natural transition. The fusion process includes: generating a high-precision mask; emptying the background region according to the mask; pasting the generated foreground defects into the empty regions; extracting the fusion boundary; and using the inpainting model to repair the boundary, resulting in a seamless fused image.

[0059] Specifically, the fusion process includes: high-precision mask generation: an auxiliary network generates a smooth-edged binary mask to accurately delineate the crack area; attention-guided fusion: a spatial attention network generates a pixel-by-pixel fusion weight map based on the texture direction of the background silicon substrate, allowing for a natural transition in grayscale and reflectivity between the crack and the background; image inpainting post-processing: the inpainting model is used to repair the junction between the crack ends and the background, eliminating "paste" marks and simulating the physical embedding effect of a real crack. Combined with... Figure 3-4 As shown, the generated defects blend naturally into the background image, improving the usability and realism of the synthesized image.

[0060] To highlight the advantages of this embodiment, experiments were conducted to compare the method disclosed in this embodiment with traditional methods, as detailed below:

[0061] Real-world defect images from an industrial scenario (e.g., metal surface inspection) were collected, including common defects (e.g., cracks, corrosion) and rare defects (e.g., composite morphological defects), and pixel-level masks were annotated. Group experiments were conducted: Control group A: trained using 50% real samples + 50% data generated by traditional methods (e.g., data generated using the StyleGAN diffusion model); Experimental group B (this application): trained using 20% ​​real samples + 80% data generated in this embodiment. The control group A and experimental group B had the same number of samples, the same test set (real samples only), the same downstream detection model architecture (e.g., YOLOv8, Faster R-CNN), and the same hyperparameters.

[0062] The evaluation metrics are set as follows: F1 score: a combination of precision and recall, measuring detection accuracy. Rare defect recall: a separate statistic for detecting rare defects. FID / KID metric: assessing the similarity of the generated data distribution to real data (assisting in verifying the quality of the generated data). The specific values ​​of the generated evaluation metrics are shown in Table 1 below:

[0063] Table 1:

[0064]

[0065] It is evident that experimental group B significantly outperformed control group A in F1 score, recall, and precision.

[0066] In the above embodiments, this invention focuses on industrial defect image generation. Through innovations in architecture, loss function, and other aspects, it effectively solves the problems of insufficient realism, controllability, and fusion effect of general generation technologies in industrial scenarios. Specifically, through a defect morphology adaptive module (such as anisotropic convolution for crack processing) and physical constraint loss (such as crack continuity constraint), it captures the fine physical features of defects. The generated defects are visually closer to real samples and outperform general generation models in objective evaluation metrics such as FID and KID. The physical attribute modulation module can quantitatively control key physical attributes of defects such as length, width, and depth. The average error between the generated result and the set target can be controlled within 5-10%. It can generate diverse samples according to parameters, covering rare morphologies in the real defect distribution. The multi-mechanism fusion module (such as weighted fusion based on mask generation by auxiliary network and post-processing for image inpainting) reduces synthetic traces, improves the usability of synthetic data, and allows the generated defects to be naturally integrated into the background image. The phased training scheme combined with procedural data generation uses manually labeled defect masks and a small number of real samples to solve the optimization conflict between the accuracy of macroscopic defect morphology and the realism of microscopic texture, and alleviates the dependence on real data. After using the generated data for augmented training, key metrics such as the F1 score and rare defect recall rate of the downstream detection model were improved, while the data collection cost was reduced.

[0067] It is worth mentioning that all modules involved in this embodiment are logical units. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0068] Example 2

[0069] Please see Figure 6 The above is a flowchart of a deep learning-based industrial defect image generation method provided in an embodiment of the present invention.

[0070] As an example, the method is applied to the deep learning-based industrial defect image generation system described in Embodiment 1, and the method includes:

[0071] S1. Receives a defect-free background image, defect type labels, and physical attribute vectors.

[0072] S2. Based on the pre-built generative network model, feature extraction units are dynamically selected according to the defect type label to generate a defect feature map.

[0073] S3. Based on the physical attribute vector, generate affine transformation parameters and perform channel-by-channel linear modulation on the defect feature map of the intermediate layer of the generator network model so that the multi-scale defect feature map finally generated by the generator network model contains the specified defect type features and physical attribute features.

[0074] S4. The multi-scale defect feature map is fused with the defect-free background map to obtain a fused defect image.

[0075] S5. Output the fused defect image.

[0076] It is not difficult to see that this embodiment is a method embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0077] Example 3

[0078] This invention also proposes a storage medium storing a deep learning-based industrial defect image generation method. When the deep learning-based industrial defect image generation program is executed by a processor, it implements the steps of the deep learning-based industrial defect image generation method described above. Since this storage medium employs all the technical solutions of the above embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated upon further here.

[0079] Example 4

[0080] Please see Figure 7 The present invention also provides an electronic device, including: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the deep learning-based industrial defect image generation method provided in Embodiment 2.

[0081] The memory 702 and processor 701 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 701 and memory 702 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 701 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 701.

[0082] Processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 702 can be used to store data used by processor 701 during operation.

[0083] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A deep learning-based industrial defect image generation system, characterized in that, The system includes: an input module, a defect image generation module, and an output module; The input module is used to receive a defect-free background image, defect type labels, and physical attribute vectors. The defect image generation module integrates a defect morphology adaptive module, a physical property modulation module, and a multi-mechanism fusion module. The defect morphology adaptive module is used to dynamically select feature extraction units based on the defect type label according to the pre-built generative network model to generate a defect feature map. The physical property modulation module is used to generate affine transformation parameters based on the physical property vector, and to perform channel-by-channel linear modulation on the defect feature map of the intermediate layer of the generative network model, so that the multi-scale defect feature map finally generated by the generative network model contains specified defect type features and physical property features. The multi-mechanism fusion module is used to fuse the multi-scale defect feature map with the defect-free background map to obtain a fused defect image; The output module is used to output the fused defect image.

2. The deep learning-based industrial defect image generation system according to claim 1, characterized in that, The defect image generation module also integrates an encoder, which is used to convert the input defect type label and physical attribute vector into a condition vector that can be recognized and processed by the generative network model.

3. The deep learning-based industrial defect image generation system according to claim 1, characterized in that, The feature extraction unit includes: a linear defect processing unit composed of anisotropic convolution kernels, and a regional defect processing unit composed of multiple parallel receptive field branches; The linear defect processing unit is integrated into the U-Net architecture of the generative network model. 1×N and N×1 anisotropic convolutional kernels are integrated into the shallow network of the U-Net architecture, and an axial attention mechanism is introduced into the deep network to capture long-distance dependencies. The regional defect processing unit adopts three parallel branches, with each branch using a different void ratio to expand the receptive field.

4. The deep learning-based industrial defect image generation system according to claim 1, characterized in that, The defect image generation module also includes a segmented training module, which is used to train the pre-built generative network model based on a preset loss function and a segmented training strategy.

5. The deep learning-based industrial defect image generation system according to claim 4, characterized in that, The preset loss function is: L total =λ adv L adv +λ rec L rec +λ perc L perc +λ phy L phy ; In the formula, L adv For adversarial loss, λ adv For the weights corresponding to the adversarial loss, L rec For pixel-level reconstruction loss, λ rec L represents the weights corresponding to the pixel-level reconstruction loss. perc For the perceptual loss based on deep features, λ perc L represents the weights corresponding to the perceptual loss based on deep features. phy For physical constraint loss, λ phy The weights corresponding to the physical constraint loss; The physical constraint loss is obtained by weighted summation of crack continuity and sharpness loss, region consistency loss, and diversity enhancement loss.

6. The deep learning-based industrial defect image generation system according to claim 5, characterized in that, The segmented training strategy includes: In the first stage, the generated network model is trained using a programmatically generated defect mask. During the training process, pixel-level reconstruction loss is the main optimization objective to optimize the macroscopic morphology of the defects. The second stage combines real samples to fine-tune the model, activates the complete composite loss function to optimize micro-texture and physical realism. During training, adversarial loss and physical constraint loss are the main optimization targets to improve the realism and detail of generated defects.

7. The deep learning-based industrial defect image generation system according to claim 6, characterized in that, The programmatically generated defect mask is generated using the following algorithm: Linear defects are generated using an L-System, and the total length of their line segments is associated with the length parameter in the physical property vector. Regional defects are generated using Perlin noise, with the noise threshold set to T = k × Area. target ; In the formula, T is the noise threshold, k is the area calibration factor, and Area target This is the area parameter in the physical property vector.

8. The deep learning-based industrial defect image generation system according to claim 1, characterized in that, The physical property modulation module inserts a FiLM layer after each residual block in the U-Net architecture to convert the physical property vector into a scaling factor γ and a bias β through a small parametric network. Based on the preset transformation formula: Output = γ × Defect Feature Map + β, a channel-level linear transformation is performed on the defect feature map so that the multi-scale defect feature map finally generated by the generative network model contains the specified defect type features and physical attribute features.

9. The deep learning-based industrial defect image generation system according to claim 1, characterized in that, The mechanisms include a mask-weighted fusion mechanism based on auxiliary networks, an adaptive fusion mechanism based on spatial attention, and a boundary post-processing mechanism based on image inpainting. The mask weighted fusion mechanism based on the auxiliary network includes: training the auxiliary network to generate an m-channel mask with smooth edges to control the fusion region between defects and the background; The adaptive fusion mechanism based on spatial attention includes: learning a pixel-by-pixel fusion weight map using a spatial attention mechanism, so that the fusion effect adapts to the local characteristics of the background and improves the naturalness of the fusion. The image restoration-based boundary post-processing mechanism includes: inputting the initially fused boundary region into the image restoration model for post-processing to produce a more natural transition.

10. A deep learning-based method for generating industrial defect images, applied to the deep learning-based industrial defect image generation system according to any one of claims 1-9, characterized in that, The method includes: Receives a defect-free background image, defect type labels, and physical attribute vectors; Based on a pre-built generative network model, feature extraction units are dynamically selected according to the defect type labels to generate defect feature maps; Based on the physical property vector, affine transformation parameters are generated, and the defect feature map of the intermediate layer of the generative network model is linearly modulated channel by channel, so that the multi-scale defect feature map finally generated by the generative network model contains the specified defect type features and physical property features. The multi-scale defect feature map is fused with the defect-free background map to obtain a fused defect image; Output the fused defect image.

Images