Method for generating magnetic sheet defect image based on adaptive sampling and LD3PM model

Abstract

The present application relates to the technical field of image processing and magnetic sheet detection, and discloses a magnetic sheet defect image generation method based on adaptive sampling and LD3PM model, which comprises collecting magnetic sheet images and presetting defect category labels, and obtaining latent space representation through a latent encoding module; then at each time step, the latent space representation is subjected to a forward diffusion process to generate a noisy latent feature, and then a noise component is obtained by predicting through a Denoising ID-Net model, and then in a dynamic step module, the noise component is dynamically adjusted based on a dynamic condition guide weight and a dynamic noise intensity to obtain a de-noised latent feature of the previous time step; after iteration reaches a preset step number, the de-noised latent feature is input into a latent decoding module to reconstruct and generate a defect image of a preset target defect category. The present application can be used to supplement a defect sample library or assist in training a defect detection model, thereby effectively improving the recognition ability of a detection system in a small sample or rare defect scene.

Description

Technical Field

[0001] This invention belongs to the field of image processing and magnetic sheet detection technology, specifically relating to a method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model. Background Technology

[0002] With the rapid development of intelligent manufacturing and industrial automation, product quality inspection has become an indispensable and critical link in the production process. Taking magnetic sheet production as an example, common defects (such as chipped corners, incomplete grinding, and edge trimming) not only reduce magnetic performance but may also lead to device failure, thereby affecting the reliability of the entire system. In recent years, deep learning-driven visual inspection methods have shown great potential in industrial defect identification, but their performance is highly dependent on large-scale, high-quality training data. However, in actual production, real defect images are often extremely scarce, especially for some rare or subtle defect categories, making it difficult to obtain enough samples for model training. This data scarcity problem significantly restricts the generalization ability and robustness of detection models.

[0003] In deep learning-driven visual inspection, generative models demonstrate significant value in scenarios with insufficient defect data. Traditional generative adversarial networks (GANs) and variational autoencoders (VAEs) are widely used for image generation and enhancement, but in real-world industrial scenarios, they often face challenges such as unstable training and difficulty in generating high-fidelity details. In recent years, diffusion models, especially the denoising diffusion probability model (DDPM) and latent spatial diffusion model (LDM), have made breakthrough progress in image generation tasks. Their iterative denoising mechanism not only ensures the stability of the generated results but also generates higher-quality images with richer details. However, most existing diffusion models focus on natural images and still have two major shortcomings when processing magnetic sheet defect images: first, the lack of a dynamic control mechanism for latent space sampling makes it difficult to achieve a balance between early random exploration and later conditional constraints, resulting in a difficulty in achieving both diversity and high fidelity in the generated magnetic sheet defect images; second, the general UNet architecture has limitations in latent space feature modeling. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model, which can generate high-quality magnetic sheet defect images through a dynamic control mechanism of latent space sampling and an industrial defect denoising network.

[0005] To address the aforementioned technical problems, this invention provides a method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model, comprising the following process:

[0006] The surface image of the magnetic sheet is acquired and the target defect category label is preset. After preprocessing, the magnetic sheet surface image and the target defect category label are input into the offline trained LD3PM model.

[0007] In the LD3PM model, the preprocessed magnetic sheet surface image and target defect category labels obtain a latent spatial representation through the latent coding module E; then, at each time step t, the latent spatial representation undergoes a forward diffusion process to generate noisy latent features. Then, the noise components are predicted using the Denoising ID-NET model. Furthermore, in the dynamic step module, weights are guided based on dynamic conditions. and dynamic noise intensity For noise components Dynamic adjustments are made to calculate the denoising latent features from the previous time step. When the iteration reaches the preset number of steps, the final denoised latent features are input into the latent decoding module D to reconstruct and generate a magnetic sheet defect image of the preset target defect category.

[0008] As an improvement to the magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention:

[0009] The Denoising ID-NET model is built on the UNet model and includes an encoder ENC and a decoder DEC. The encoder ENC includes a frequency-edge module and multiple coding blocks that are downsampled stepwise. The decoder DEC includes multiple decoding blocks that are upsampled stepwise. Coding blocks and decoding blocks at the same level perform cross-layer feature fusion through a gated cross-jump module.

[0010] The encoding block includes a cascaded downsampling operation, an adaptive self-attention module, and an LGT module, while the decoding block includes a cascaded upsampling operation and an adaptive self-attention module.

[0011] As a further improvement to the magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention:

[0012] The dynamic step module obtains potential noise reduction features. The method is as follows:

[0013]

[0014] in, Let represent the noise variance scheduling parameter at step t; ,in, Indicates the first Noise retention factor for each time step; Indicates the first Random noise injected at each time step; To guide weights based on dynamic conditions and dynamic noise intensity The dynamically adjusted noise component; Labels for defect categories.

[0015] As a further improvement to the magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention:

[0016] The dynamic condition-guided weight for:

[0017]

[0018] The dynamic noise intensity for:

[0019]

[0020] in, To achieve the maximum conditional guidance strength, For the maximum noise amplitude, The condition-guided adjustment coefficient. is the noise adjustment factor, and T is the total number of time steps.

[0021] The dynamic adjustment method is as follows:

[0022] .

[0023] As a further improvement to the magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention:

[0024] The operation process of the frequency-edge module includes:

[0025] First, the noisy latent features of the input are processed using Sobel convolution kernels. Extracting local gradient response features Then, the noisy latent features Local gradient response characteristics After splicing, the data undergoes a Fast Fourier Transform (FFT), and the resulting frequency domain information is analyzed. High frequency amplitude Enhancement:

[0026]

[0027] in, Represents the frequency domain amplitude spectrum. It is the enhancement coefficient;

[0028] Next, the output feature R is reconstructed through inverse Fourier transform:

[0029] in, Indicates the inverse Fourier transform. This indicates a feature fusion operation. This indicates a convolution operation.

[0030] As a further improvement to the magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention:

[0031] The LGT module includes a global self-attention branch and a local convolution branch. The global self-attention branch includes a first linear function, a multi-head attention (MHA) module, and a second linear function in series. The local convolution branch includes convolution.

[0032] The input features are split along the channel dimension and then fed into the global self-attention branch and the local convolution branch respectively. The outputs of the two branches are fused using a weighted fusion method.

[0033]

[0034] in, For learnable weights, This represents the output of the global self-attention branch. This represents the output of a local convolution branch;

[0035] Then, the fused feature F is normalized through layers and fed-forward network, and then output through residual connection.

[0036] The global self-attention branch includes a first linear function, a multi-head attention (MHA) module, and a second linear function in series, while the local convolution branch includes a convolution operation.

[0037] As a further improvement to the magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention:

[0038] The operation of the gated cross-jump module is as follows:

[0039]

[0040] in, This indicates a cross-attention operation. For gated functions, This represents element-wise multiplication. This represents the feature representation obtained by upsampling the output features from the previous decoding stage. This represents the feature representation extracted by the encoder ENC at the corresponding scale and passed to the decoder DEC.

[0041] As a further improvement to the magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention:

[0042] Both the potential encoding module E and the potential decoding module D are variational autoencoders.

[0043] The beneficial effects of this invention are mainly reflected in:

[0044] (1) This invention aims to solve the problem of poor generalization ability of deep learning in small sample scenarios in industrial defect image generation. By modeling and sampling in the compressed latent space, the computational complexity is significantly reduced while ensuring the representation ability and the generation efficiency is improved.

[0045] (2) This invention achieves synergistic optimization of dynamic sampling control and fine feature modeling. First, this invention proposes a Latent Dynamic Sampling (LDS) mechanism, replacing the fixed-intensity denoising step in the traditional latent spatial diffusion model, and introducing dynamic condition-guided weights and dynamic noise intensity for dynamic adjustment. This mechanism enhances the random exploration capability in the early stage of sampling to improve image diversity, and strengthens the conditional constraints in the later stage of sampling to ensure generation stability and high fidelity, thereby effectively balancing the contradiction between diversity and realism.

[0046] (3) The industrial defect denoising network of the present invention (referred to as Denoising ID-NET) includes three key modules: the frequency-edge (FE) module combines edge information in the spatial domain and detail information in the frequency domain, which significantly enhances the expressive power of small defects and the sensitivity to defect contours; the lightweight global-local Transformer (LGT) module combines global self-attention branch and local convolution branch, which effectively preserves local small defect features while improving global modeling ability, and has lower computational overhead; the gated cross-jump (GCS) module uses a gating mechanism to selectively fuse cross-layer features, which can preserve key information while suppressing invalid or redundant features, thereby improving the accuracy of generated images and defect identifiability.

[0047] (4) Experimental results show that, in the self-built magnetic sheet surface defect dataset, after the DenoisingID-NET and LDS mechanisms of this invention work together, the generated image FID reaches 86.89 and LPIPS drops to 0.273. All performance indicators are superior to other comparative networks such as UNet and DiT. The industrial defect images generated by this invention can be used to supplement the defect sample library or assist in training the defect detection model, thereby effectively improving the detection system's recognition ability in small sample or rare defect scenarios. Attached Figure Description

[0048] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0049] Figure 1 This is a schematic diagram of the LD3PM model of the present invention;

[0050] Figure 2 Example diagrams showing the dynamic adjustment of condition guidance and dynamic noise intensity curves;

[0051] Figure 3 This is a schematic diagram of the frequency-edge module.

[0052] Figure 4 This is a schematic diagram of the lightweight global-local Transformer module;

[0053] Figure 5 This is a structural diagram of the gated cross-jump module;

[0054] Figure 6 This invention provides an industrial defect image of a preset target defect category, reconstructed based on the magnetic sheet defect image generation method of adaptive sampling and LD3PM model. Detailed Implementation

[0055] The present invention will be further described below with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto:

[0056] Example 1: A method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model, such as... Figure 1 As shown,

[0057] 1. Construct an implicit dynamic denoising diffusion probability model (LD3PM)

[0058] The overall framework of the Latent Dynamic Denoising Diffusion Probabilistic Model (LD3PM) of this invention is as follows: Figure 1 As shown, LD3PM, based on the LDM concept, performs modeling and sampling within the compressed latent space, thereby significantly reducing computational complexity and improving generation efficiency while maintaining representational power. The overall generation process can be summarized into three stages:

[0059] (1) Latent encoding: The input magnetic sheet defect image is first mapped to the latent space through the latent encoding module E (in this example, a pre-trained variational autoencoder (VAE) is preferred) to obtain a low-dimensional and semantically rich representation, providing compact and operable features for subsequent generation.

[0060] (2) Latent Space Sampling: This includes the Forward Diffusion Process, the Denoising ID-NET, and the Latent Dynamic Sampling (LDS) process, which iteratively generates denoised latent features in the latent space. To improve generation quality and diversity, LD3PM proposes a Dynamic Step module to implement the latent dynamic sampling process. The Dynamic Step introduces dynamic condition guidance and dynamic noise scheduling in each iteration: it enhances random exploration capabilities in the early stages of sampling to improve image diversity, and strengthens conditional constraints in the later stages of sampling to ensure generation stability and high fidelity, thereby effectively balancing the contradiction between diversity and realism.

[0061] In terms of latent space feature modeling, LD3PM introduces an industrial defect denoising network (Denoising ID-NET) based on the U-Net network architecture as the backbone network. Denoising ID-NET is specifically designed for industrial defect image generation tasks, including an encoder (ENC) and a decoder (DEC). The encoder (ENC) includes a frequency-edge module (FE module) and multiple coded blocks (ENC1~ENC3) that are downsampled stepwise. The decoder (DEC) includes multiple decoded blocks (DEC3~DEC1) that are upsampled stepwise. The coded blocks and decoded blocks at the same level are fused across layers through a gated cross-skip (GCS) module, which can characterize fine-grained features in the latent space and highlight defect-related regions.

[0062] By organically combining Dynamic Step with Denoising ID-NET, LD3PM achieves synergistic optimization of dynamic sampling control and fine feature modeling during the latent diffusion process, significantly improving the diversity and realism of generated images and providing solid support for industrial defect data enhancement and subsequent detection tasks.

[0063] (3) Latent decoding: The denoised latent features generated through iteration are reconstructed into a high-fidelity defect image in the image space by the latent decoding module D (in this example, a pre-trained variational autoencoder (VAE) is preferred) for industrial defect data enhancement and subsequent detection tasks.

[0064] 1.1 Dynamic Step Module

[0065] The Dynamic Step module employs a Latent Dynamic Sampling (LDS) mechanism to enhance the flexibility and stability of the latent spatial diffusion model in industrial defect image generation. This invention's Latent Dynamic Sampling (LDS) replaces the fixed-intensity denoising step in the traditional latent spatial diffusion model by dynamically adjusting condition guidance and dynamic noise intensity during the sampling process. This achieves a balance between early random exploration and later conditional constraints, thereby improving the diversity and high fidelity of the generated images. In this invention, the method for dynamically adjusting condition guidance is as follows:

[0066] Let the total number of diffusion sampling steps be In the In step sampling, dynamic condition-guided weights and dynamic noise intensity The definition is as follows:

[0067]

[0068] in, and These are the maximum conditional guidance strength and the maximum noise amplitude, respectively. Their parameter values ​​were determined through experimental evaluation of the generated image quality on a validation set, and are used to balance the diversity and realism of the generated images. In this embodiment, Set to 3.0, Set to 1.0. For conditional adjustment coefficients, This is the noise adjustment factor, used to control the rate at which it changes with the sampling progress. We set... and It can fully enhance random exploration capabilities in the early stages, while smoothly transitioning to high-fidelity generation in the later stages, thus balancing diversity and detail preservation.

[0069] In this invention, the method for implementing dynamic noise scheduling is as follows: in each sampling step, the predicted noise components are... Dynamic adjustment:

[0070]

[0071] in, This is the dynamically adjusted noise component. To generate the first Gaussian noise by gradually injecting it during the forward diffusion process Noisy latent features at each time step For conditional information (such as defect category labels). This indicates a dynamic condition-guided weight that changes as the sampling progresses. Let represent the noise component predicted by the Denoising ID-NET model. Through this strategy, the Latent Dynamic Sampling (LDS) mechanism enhances the random exploration capability in the early stages of sampling and gradually strengthens the constraint on conditional information in the later stages, thereby generating defect images that are both diverse and high-fidelity.

[0072] In each sampling step, the dynamic step module outputs denoising latent features. The updated formula is:

[0073]

[0074] in, Denotes the noise variance scheduling parameter at step t. , Indicates the first The noise retention factor corresponding to each time step. This represents the cumulative noise retention factor. In this embodiment, The settings can be configured using either linear or cosine scheduling. For dynamic noise, where, This represents the noise adjustment coefficient, used to control the rate of change of noise intensity as the sampling progresses; Indicates the first Random noise injected at each time step; Indicates a Gaussian distribution; This represents the identity matrix, whose dimensions are consistent with the feature dimensions of the latent space. This indicates the total number of sampling steps.

[0075] In this way, LDS introduces controllable random perturbations during the denoising process and gradually enhances conditional guidance, thereby generating diverse and high-fidelity industrial defect images in the latent space.

[0076] Figure 2 shows an example of the dynamic noise and dynamic condition guidance curves of the LDS, where red represents the dynamic noise intensity. Blue indicates dynamic condition-guided weights. The color intensity indicates the sampling progress. As can be seen from the curve, the early stage of sampling has high noise and weak conditions, which helps to explore more potential samples; the later stage of sampling has reduced noise and enhanced conditions, ensuring that the generated image meets the constraints of the target defect category.

[0077] 1.2 Frequency-Edge Module (FE Module)

[0078] In recent years, frequency domain-based augmentation methods have made significant progress in image generation tasks. However, in industrial defect image generation, traditional CNNs struggle to capture subtle features. To address this, the FE module is proposed in Denoising ID-NET, such as... Figure 3As shown, combining spatial domain edge information and frequency domain amplitude features effectively enhances detail representation, especially in the generation of minute defects, outperforming traditional methods.

[0079] The design of the FE module is based on two key technologies: edge feature extraction and frequency domain amplitude enhancement.

[0080] First, use Sobel convolution kernels to process noisy latent features. Local gradient response features are obtained by extracting local gradient responses. This is to enhance the local structural information and high-frequency detail representation in the latent features. Local gradient response features The extraction process is represented as follows:

[0081]

[0082] in, Use Sobel convolution kernels.

[0083] Next, the input noisy latent features Local gradient response characteristics The concatenation is performed along the channel dimension, and then the concatenated features are analyzed using Fast Fourier Transform (FFT). Transforming from the spatial domain to the frequency domain is used to separate the low-frequency and high-frequency components of the image.

[0084]

[0085] in, This indicates the Fast Fourier Transform operation.

[0086] Frequency domain information obtained by Fast Fourier Transform The high-frequency components in the image contain detailed information such as edges, textures, and minute defects. To enhance the ability to represent defect details, this invention utilizes frequency domain information... The high-frequency amplitude components are enhanced to improve the model's ability to represent minute defect features. This operation can be expressed as:

[0087]

[0088] in, Represents the frequency domain amplitude spectrum. Frequency domain High frequency amplitude, It is an enhancement coefficient used to control the enhancement degree of high-frequency amplitude components. In this embodiment, the value range is 1.2 to 2.0, preferably 1.5.

[0089] Enhanced frequency domain information Frequency domain information before enhancement Feature fusion is performed along the channel dimension. Then, in the frequency domain, a 1x1 convolution is used to perform cross-channel information interaction and linear transformation on the fused frequency domain features, resulting in processed complex frequency domain features. These are then transformed back to the spatial domain using an inverse Fourier transform (IFFT). At this point, the enhanced amplitude spectrum is combined with the phase information from the original frequency domain features, thereby preserving the spatial details of the image and enhancing the expression of subtle defects. The reconstruction process can be represented by the following formula:

[0090]

[0091] in, Indicates the inverse Fourier transform. This indicates a feature fusion operation. R represents the convolution operation, and R represents the reconstructed features output by the FE module.

[0092] By combining edge information from the spatial domain and detail information from the frequency domain, the FE module significantly enhances the representation of minute defects. Edge information extraction ensures the model's sensitivity to defect contours, while frequency domain enhancement improves the preservation of detail components. Through the fusion of these two types of information, the FE module can effectively improve the quality and accuracy of generated results during image generation, providing more powerful data augmentation capabilities for industrial defect detection tasks.

[0093] 1.3 Lightweight Global-Local Transformer Module (LGT Module)

[0094] In the encoder ENC of Denoising ID-NET, modeling global context and local details is crucial for generating high-quality industrial defect images. While CNNs can extract local features, they struggle to capture long-range dependencies; ViT relies on self-attention to model global relationships, but its computational complexity is limited for industrial applications due to a quadratic increase in resolution. Therefore, this invention proposes a lightweight global-local Transformer module (LGT module), such as... Figure 4 As shown, this method enhances global modeling capabilities while preserving the local details required for minor defects, and also takes computational efficiency into account.

[0095] The LGT module includes a global self-attention branch and a local convolutional branch. The global self-attention branch consists of a concatenated first linear function, a multi-head attention (MHA) module, and a second linear function. The local convolutional branch consists of a 3x3 convolution. Given an input feature map... After a 1:1 split along the channel dimension, the code proceeds to the global self-attention branch and the local convolution branch, respectively. In the global self-attention branch, the query is first obtained through linear projection. Vector, key Vector and value vector:

[0096]

[0097] in, For learnable parameters, This represents the local features input to the local convolutional branch after splitting. Global attention models long-range dependencies by calculating formula (8), thereby capturing the overall contextual relationships of the image.

[0098]

[0099] in, Represents the attention function. express The function, where d is the feature dimension.

[0100] In this embodiment, Multi-Head Attention (MHA) performs the attention calculation in formula (10) in parallel using multiple attention heads, and then concatenates and fuses the outputs of the multiple attention heads, thereby enhancing the model's ability to model different feature subspaces. Simultaneously, local convolutional units extract detailed features such as edges and textures within the spatial neighborhood to compensate for the insufficient characterization of local patterns by self-attention. The two are fused using a weighted fusion method:

[0101]

[0102] in, These are learnable weights used to adaptively balance global and local information.

[0103] To reduce computational complexity, LGT employs a low-dimensional embedding space. To reduce the size of the attention matrix, local convolutional branches are used to extract detailed neighborhood features, thereby reducing the number of model parameters and computational overhead while maintaining feature expressiveness. Finally, the fused feature F is processed by layer normalization (LN) and a feedforward network (FFN) to enhance its expressiveness. To improve feature stability and alleviate gradient degradation during deep network training, the LGT module employs a residual connection structure, adding the fused feature to the feature processed by layer normalization and the feedforward network using residuals. This can be formally represented as:

[0104]

[0105] Through the above design, the LGT module significantly reduces computational overhead while maintaining global dependency modeling capabilities and effectively preserving local defect features. Experimental results show that this module significantly improves inference speed and parameter efficiency while ensuring image generation quality, verifying its practical value in industrial defect image generation tasks.

[0106] 1.4 Gated Cross Jump (GCS) Module

[0107] In the Denoising ID-NET decoder (DEC), fully utilizing the multi-scale features of each encoder layer is crucial for generating high-quality industrial defect images. Traditional skip connections typically pass encoder features to the decoder through simple feature concatenation or addition, but the presence of redundant information or noise can lead to blurred details or insufficient defect representation in the generated image. Therefore, this invention designs a gated cross-skip module (GCS module) (e.g., ...). Figure 5 As shown, the decoder (DEC) selectively fuses cross-layer features through a gating mechanism, thereby enhancing the DEC's utilization of key information while suppressing irrelevant and redundant features.

[0108] The core idea of ​​the GCS module is: at each stage of the decoder DEC, the features from the upsampled features... Corresponding scale features of encoder ENC Cross-Attention alignment is used, followed by gating to control information flow, enabling selective fusion of cross-layer features. This represents the feature representation obtained by upsampling the output features from the previous decoding stage. This represents the feature representation extracted by the encoder (ENC) at the corresponding scale and passed to the decoder (DEC). The formula is as follows:

[0109]

[0110] in, This indicates a cross-attention operation used to align upsampled features with encoder features; It is a gated function, usually defined by Convolutional and sigmoid activations are used to control the feature weights of each channel; This represents element-wise multiplication. Through this design, the GCS module can retain key information while suppressing invalid or redundant features, thereby improving the precision and defect identifiability of the decoder-generated image.

[0111] In this way, the GCS module can dynamically select and enhance useful encoder features during the decoder stage, effectively improving the generation quality of minor defects while avoiding unnecessary feature interference.

[0112] 1.5 Overall Structure of the LD3PM Model

[0113] The LD3PM model proposed in this invention consists of a latent encoding module, a latent spatial sampling module, and a latent decoding module. It introduces a latent dynamic sampling (LDS) mechanism during the latent spatial sampling process and uses Denoising ID-NET as the core feature modeling network of the latent spatial sampling module.

[0114] Specifically, the overall structure and data flow of the LD3PM model are as follows:

[0115] (1) Latent feature encoding stage

[0116] The input image of the magnetic sheet defect is fed into a latent encoding module E located in the pixel space (e.g., using a pre-trained VAE encoder). The latent encoding module E compresses and maps the high-dimensional pixel-level image into the latent space, resulting in low-dimensional and semantically rich initial latent space features. The initial potential spatial features While preserving key structural information, it significantly reduces data dimensionality, providing an efficient representation for subsequent diffusion modeling.

[0117] (2) Forward diffusion and conditional feature extraction stage

[0118] First, the initial latent space features output by the latent coding module E. The forward diffusion process is performed to generate the first Gaussian noise by progressively injecting it. Noisy latent features at each time step The process can be represented as follows:

[0119]

[0120] Where q represents the probability distribution of "state transition" during forward diffusion.

[0121] Furthermore, the first The noisy latent features at each time step can be represented as:

[0122]

[0123] in, It is standard Gaussian noise and follows a standard normal distribution.

[0124] Simultaneously, the preset category label information (Class) is obtained and input into the label embedding network in the Conditioning module. (The tag embedding network) This is used to map preset defect category labels to conditional semantic features, so as to implement conditional constraints on the diffusion generation process, and generate corresponding conditional label information.

[0125] Subsequently, the noisy latent features The conditional label information is input into the Denoising ID-NET for noise prediction and inverse denoising sampling.

[0126] (3) Latent space inverse denoising iterative stage

[0127] Noisy latent features In the latent space, conditional label information is incorporated to perform a cyclically iterative inverse denoising process. In any iteration at any time step, the following sub-steps are specifically executed:

[0128] (3.1) The conditional label information output by the feature extractor is distributed via the switch module and injected into each specified network layer of the industrial defect denoising network (Denoising ID-NET);

[0129] (3.2) Noisy latent features of the current time step t The input to the Denoising ID-NET network first extracts edge and frequency domain enhancement features through the Frequency-Edge (FE) module. Then, it passes through multiple encoding blocks (ENC1 to ENC3) with progressive downsampling to extract global and local features and gradually reduce the feature map resolution. After passing through the bottleneck layer, it enters multiple decoding blocks (DEC3 to DEC1) with progressive upsampling for feature recovery. Each encoding block (ENC1 to ENC3) includes concatenated downsampling operations, a self-attention adaptive module, and an LGT module, while each decoding block (DEC3 to DEC1) includes concatenated upsampling operations and a self-attention adaptive module. Encoding blocks (ENC) and decoding blocks (DEC) at the same level perform cross-layer feature fusion through a gated cross-skip (GCS) module, thereby enhancing the utilization of key defect features in the decoding stage and suppressing redundant feature interference.

[0130] (3.3) The noise component of the current time step is predicted and output by the industrial defect denoising network. The input is fed into the Dynamic Step module, which introduces a latent dynamic sampling (LDS) mechanism to dynamically adjust the conditional guidance strength and noise injection strength, and calculates the denoising latent features of the previous time step. In this process, the random exploration capability is enhanced in the early stage of sampling, and the conditional constraint capability is enhanced in the later stage of sampling, in order to balance the diversity and realism of the generated images.

[0131] (3.4) Repeat steps (3.1) to (3.3) above according to the preset number of iterations until the final denoising time step T is reached, and output the final denoising potential feature z*.

[0132] (4) Input the denoising latent feature z* into the latent decoding module D, reconstruct it from the latent space back to the pixel space, and generate a high-fidelity industrial defect image.

[0133] (5) Overall Collaborative Mechanism: In the above structure, Latent Dynamic Sampling (LDS) and the Industrial Defect Denoising Network (Denoising ID-NET) work together: LDS is responsible for dynamically balancing randomness and conditional constraints during the generation process, thereby improving the diversity and realism of the generated images; the FE, LGT, and GCS modules built into Denoising ID-NET are responsible for fine modeling of latent spatial features, enhancing the model's ability to express minute defects. The combination of the two enables LD3PM to generate high-quality and diverse defect images even under small sample conditions.

[0134] In summary, LD3PM achieves a comprehensive improvement in generation quality, generation efficiency, and model stability by introducing a dynamic sampling mechanism in the latent space and combining it with a dedicated feature modeling network.

[0135] 2. Training dataset construction and offline training

[0136] 2.1 Dataset for Detecting Surface Defects on Magnetic Sheets

[0137] The magnetic sheet defect dataset used in this experiment was derived from a real industrial site. An image acquisition device was set up above the magnetic sheet production line. As the magnetic sheets were conveyed to the area below the image acquisition device via a conveyor belt, images were captured by a high-speed industrial camera. Given the high yield rate of the products, defects were manually selected and confirmed by quality inspectors. After cropping and processing, 4497 ​​images were finally obtained, as shown in Table 1.

[0138] The dataset contains several typical edge defect types for magnetic sheets. Among them, DJ (corner chipping) indicates local missing or broken edges of the magnetic sheet; LM (unpolished) indicates that there are insufficiently polished areas on the edges of the magnetic sheet, which appear as rough or irregular surfaces; and XB (edge ​​trimming) indicates that the edges of the magnetic sheet have been over-cut, resulting in abnormal shrinkage or irregular boundaries.

[0139] Table 1. Dataset partitioning for training, validation, and test sets.

[0140] DJ 1606 200 1806 LM 1366 174 1540 XB 1016 135 1151

[0141] 2.2 Hardware and software environment configuration

[0142] Hardware configuration: The experimental platform's hardware configuration includes an Intel(R) Xeon(R) Gold 6430 processor, 120GB of memory, and an RTX 4090 graphics card with 24GB of video memory.

[0143] Software configuration: The software environment includes the following versions: Ubuntu 22.04 operating system, PyTorch 2.1.2 deep learning framework, Python 3.10 programming language and CUDA 11.8 parallel computing platform.

[0144] 2.3 Offline Training Process

[0145] During the offline training phase, the training set from the magnetic sheet surface defect dataset is input into the LD3PM model for training. First, the magnetic sheet defect images with defect category labels are input into the latent coding module E and mapped to the latent space to obtain the latent representation. Subsequently, noise is progressively added to the latent representation in the latent space to construct diffusion process samples.

[0146] During training, the noisy diffusion process samples, corresponding time steps, and defect category labels are input to the Denoising ID-NET model to predict the noise component at the current time step. Model parameters are optimized by minimizing the difference between predicted and actual noise. The mean squared error loss function is used to calculate the loss and perform backpropagation. The AdamW optimizer is used to optimize the network, with the first-order momentum parameter set to 0.9, the second-order momentum parameter set to 0.999, the batch size set to 8, the initial learning rate set to 3e-4, and a cosine learning rate decay function used to control the learning rate decay. The weight decay is set to 0.0001, and the total training epochs are 400. Each training epoch is validated on a validation set, and the optimal weights based on the implicit dynamic denoising diffusion probability model are obtained at the end of training.

[0147] Then, the performance metrics of the implicit dynamic denoising diffusion probability model with optimal weights were verified on the test set. The FID reached 86.89 and the LPIPS was 0.273, which is suitable for use in online inference.

[0148] 3. Online Reasoning

[0149] The magnetic sheet defect image generation method based on adaptive sampling and the LD3PM model of the present invention is applied to the defect image generation and data augmentation scenario in an actual industrial production line, specifically including the following steps:

[0150] (1) Data Acquisition and Preprocessing: On the industrial production line, images of the magnetic sheet surface are acquired in real time using an industrial camera. The acquired images are then preprocessed, including size normalization, noise filtering, and format conversion, to meet the model input requirements. At the same time, target defect category labels are preset according to the inspection needs. The target defect category labels are used to guide the generation of images of different defects. The defect categories include DJ (corner chipping), LM (grinding gaps), and XB (edge ​​chipping).

[0151] (2) Latent space mapping

[0152] The preprocessed magnetic sheet image and the target defect category label are input into the pre-trained latent encoding module E to obtain the corresponding latent space representation.

[0153] (3) Iterative generation of latent space

[0154] At each time step, the latent space representation first undergoes a forward diffusion process to generate noisy latent features. Then, the noise component is predicted using the Denoising ID-NET model. Then, the dynamic condition-guided weights and dynamic noise intensity are dynamically adjusted through the LDS mechanism of the Dynamic Step module, and an iterative denoising process based on latent dynamic sampling (LDS) is performed in the latent space to gradually update the latent variables.

[0155] (4) Image reconstruction

[0156] After the preset number of iterations is reached, the final denoised latent features are input into the latent decoding module D to reconstruct and generate an industrial defect image of the preset target defect category. An example of the generated image is shown below. Figure 6 As shown, where, Figure 6 The first row of images shows the generated image of the magnetic sheet with DJ (corner chipping) defects, and the second row of images shows the generated image of the magnetic sheet with LM (mold bleed) defects.

[0157] (5) Output and application of results The generated industrial defect images are output to the subsequent quality inspection system to expand the defect sample library or assist in training the defect detection model, thereby improving the detection system’s ability to identify defects in small sample or rare defect scenarios.

[0158] In practical applications, the online inference process can be deployed on industrial field servers or edge computing devices to achieve real-time generation and data enhancement of magnetic sheet defect images.

[0159] experiment

[0160] 1. Experimental Data

[0161] The experiment uniformly used the test set samples from the magnetic sheet surface defect detection dataset constructed in Section 2.1 of Example 1.

[0162] 2 Evaluation Indicators

[0163] To objectively evaluate the performance of the LD3PM model in the task of generating images of magnetic sheet defects, the following three core metrics were selected for quantitative analysis:

[0164] Fréchet Inception Distance (FID): This metric assesses generation quality by calculating the distribution distance between the generated image and the real image in feature space. A lower FID value indicates that the statistical distribution of the generated image is closer to the real data distribution. Let the real data feature distribution be... The generated data feature distribution is The calculation formula is as follows:

[0165]

[0166] in, This represents the mean vector of the real image in the feature space. This represents the mean vector of the generated image in the feature space. The covariance matrix represents the features of the generated image. The trace operation represents the sum of the elements on the main diagonal of a matrix.

[0167] Learned Perceptual Image Patch Similarity (LPIPS): This measure assesses the perceptual difference between generated and real images. Unlike pixel-level MSE, LPIPS utilizes pre-trained networks (such as AlexNet) to extract features and calculates the average distance between channels, thus better reflecting the human eye's perception of image details and textures. Its calculation logic can be expressed as follows:

[0168]

[0169] in, Represents a real image; Indicates the generated image; Indicates the number of channels in the feature map; Indicates the width of the feature map; Indicates the height of the feature map; These represent index variables in the channel, width, and height directions, respectively; Indicates the image Enter to the number After one pre-trained network layer Indicates that an image will be generated. Enter to the number After the nth pre-trained network layer, at the nth Features extracted from each channel location; A distance metric function representing the distance between features; This represents the squared distance, used to measure the difference between two images in the feature space.

[0170] 3 Ablation Experiment

[0171] Ablation experiments were conducted to analyze the impact of different modules on the quality of the generated images. The experimental results are shown in Table 1. The first row of Table 1 shows the basic LDM model. The second to fourth rows progressively introduce various modules (LDS, FE, LGT, GCS) based on the basic model. The fifth and sixth rows, based on the model in the third row, replace the LGT module with the Vision Transformer (ViT) module and the lightweight feature enhancement module (Life module), respectively, to compare the impact of different global modeling methods on the generation performance. The seventh row shows the model of this invention. As can be seen from Table 2, the comprehensive introduction of LDS, FE, LGT, and GCS modules in this invention significantly improves image quality. The introduction of LDS reduces FID from 126.20 to 114.31 and LPIPS from 0.343 to 0.301, demonstrating its effect on improving image diversity and sharpness. The FE module further reduces FID to 105.69 and LPIPS to 0.287, enhancing the image's detail recovery capability. The LGT module significantly improved global modeling and local detail restoration, reducing FID to 91.67 while maintaining LPIPS at 0.287. In comparisons with the ViT and Life modules, the LGT module performed best. While ViT and Life improved global modeling, they failed to surpass LGT in detail restoration and image quality.

[0172] Table 2 Ablation Experiment Results

[0173]

[0174] In the experimental comparison, the final combination of LDS + FE + LGT + GCS performed best, achieving an FID of 86.89 and LPIPS of 0.273, demonstrating the strongest generated quality and detail fidelity. This indicates that the synergistic effect of LD3PM is most significant in improving image quality. Although the Params were slightly increased to 24.65M, the significant performance improvement proves the effectiveness of this combination in optimizing generated quality.

[0175] 4. Comparative Experiment

[0176] In the comparative experiment, the six networks in Table 3 were first trained offline on the dataset constructed in step 2 to obtain their respective converged model weight files. Then, based on the LDM model, the six offline-trained networks were used as the backbone networks in the latent space sampling process. DDPM (Denoising Diffusion Probability Model), DDIM (Denoising Diffusion Implicit Model), and the LDS method used in the dynamic step module of this invention were then used as iterative denoising methods in the latent space sampling process to obtain different sets of generated images. The experimental results of the performance indicators of each method are shown in Table 3.

[0177] Table 3 Comparison of experimental results

[0178]

[0179] The results show that using the Denoising ID-NET of this invention as the backbone network in the latent space sampling process effectively reduces the FID and LPIPS values ​​of the generated image, regardless of the denoising method used, resulting in the best image quality. This demonstrates that Denoising ID-NET has advantages in improving generation quality, better balancing global structure and local details.

[0180] Compared to the other five contrasting networks used as the backbone network in the latent space sampling process, the combination of Denoising ID-NET and LDS employed in this invention produces a synergistic effect, achieving the best generation quality among all control groups. Specifically, Denoising ID-NET achieves the best performance in LDS sampling with an FID of 86.89 and LPIPS of 0.273, significantly outperforming methods such as UNet and U-ViT. Although the number of parameters increases slightly to 24.65M, this is reasonable compared to the improvement in generation quality, demonstrating the effectiveness and superiority of the LD3PM framework in industrial defect image generation. Overall, the proposed LD3PM framework demonstrates excellent performance in both the realism and diversity of generated images, proving its effectiveness and superiority in industrial defect image generation tasks. Figure 6 The image shows a magnetic sheet defect generated by LD3PM.

[0181] Finally, it should be noted that the above examples are merely some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model, characterized in that... The process includes: The surface image of the magnetic sheet is acquired and the target defect category label is preset. After preprocessing, the magnetic sheet surface image and the target defect category label are input into the offline trained LD3PM model. In the LD3PM model, the preprocessed magnetic sheet surface image and target defect category labels obtain a latent spatial representation through the latent coding module E; then, at each time step t, the latent spatial representation undergoes a forward diffusion process to generate noisy latent features. Then, the noise components are predicted using the Denoising ID-NET model. Furthermore, in the dynamic step module, weights are guided based on dynamic conditions. and dynamic noise intensity For noise components Dynamic adjustments are made to calculate the denoising latent features from the previous time step. When the iteration reaches the preset number of steps, the final denoised latent features are input into the latent decoding module D to reconstruct and generate a magnetic sheet defect image of the preset target defect category.

2. The method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model according to claim 1, characterized in that: The Denoising ID-NET model is built on the UNet model and includes an encoder ENC and a decoder DEC. The encoder ENC includes a frequency-edge module and multiple coding blocks that are downsampled step by step, and the decoder DEC includes multiple decoding blocks that are upsampled step by step. Cross-layer feature fusion is performed between encoded and decoded blocks at the same level through a gated cross-jump module; The encoding block includes a cascaded downsampling operation, an adaptive self-attention module, and an LGT module, while the decoding block includes a cascaded upsampling operation and an adaptive self-attention module.

3. The method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model according to claim 2, characterized in that: The dynamic step module obtains potential noise reduction features. The method is as follows: in, Let represent the noise variance scheduling parameter at step t; ,in, Indicates the first Noise retention factor for each time step; Indicates the first Random noise injected at each time step; To guide weights based on dynamic conditions and dynamic noise intensity The dynamically adjusted noise component; Labels for defect categories.

4. The method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model according to claim 3, characterized in that: The dynamic condition-guided weight for: The dynamic noise intensity for: in, To achieve the maximum conditional guidance strength, For the maximum noise amplitude, The condition-guided adjustment coefficient. The noise adjustment factor is T, and the total number of time steps is T. The dynamic adjustment method is as follows: 。 5. The method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model according to claim 4, characterized in that: The operation process of the frequency-edge module includes: First, the noisy latent features of the input are processed using Sobel convolution kernels. Extracting local gradient response features Then, the noisy latent features Local gradient response characteristics After splicing, the data undergoes a Fast Fourier Transform (FFT), and the resulting frequency domain information is analyzed. High frequency amplitude Enhancement: in, Represents the frequency domain amplitude spectrum. It is the enhancement coefficient; Next, the output feature R is reconstructed through inverse Fourier transform: in, Indicates the inverse Fourier transform. This indicates a feature fusion operation. This indicates a convolution operation.

6. The method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model according to claim 5, characterized in that: The LGT module includes a global self-attention branch and a local convolution branch. The global self-attention branch includes a first linear function, a multi-head attention (MHA) module, and a second linear function in series. The local convolution branch includes convolution. The input features are split along the channel dimension and then fed into the global self-attention branch and the local convolution branch respectively. The outputs of the two branches are fused using a weighted fusion method. in, For learnable weights, This represents the output of the global self-attention branch. This represents the output of a local convolution branch; Then, the fused feature F is normalized through layers and fed-forward network, and then output through residual connection. The global self-attention branch includes a first linear function, a multi-head attention (MHA) module, and a second linear function in series, while the local convolution branch includes a convolution operation.

7. The method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model according to claim 6, characterized in that: The operation of the gated cross-jump module is as follows: in, This indicates a cross-attention operation. For gated functions, This represents element-wise multiplication. This represents the feature representation obtained by upsampling the output features from the previous decoding stage. This represents the feature representation extracted by the encoder ENC at the corresponding scale and passed to the decoder DEC.

8. The method for generating magnetic sheet defect images based on adaptive sampling and the LD3PM model according to claim 7, characterized in that: Both the potential encoding module E and the potential decoding module D are variational autoencoders.

Images