Anomaly image generation method and system based on anisotropic information diffusion

Abstract

The application discloses an abnormal image generation method and system based on anisotropic information diffusion, and the method comprises the following steps: S1, reading an abnormal image, a defect mask and a defect category label, and performing pretreatment to form a standardized input; S2, extracting anisotropic structure information in a defect area of the defect mask to obtain a structure description quantity representing a defect morphology; S3, constructing the structure description quantity into a pixel-aligned multi-channel structure prior graph, encoding the pixel-aligned structure prior graph to obtain F aniso , obtaining F pos and F cla , and obtaining an anisotropic conditional field; S4, injecting and diffusing the conditional field composed of F aniso , F pos and F cla into a diffusion denoising network in multiple levels to constrain local details and global structure consistency; S5, training and optimizing the diffusion denoising network based on a joint loss function to obtain an abnormal image generation model; and S6, loading the trained abnormal image generation model into an inference mode, giving a conditional input to be generated, and generating an abnormal image through a reverse diffusion denoising sampling process.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and industrial vision inspection technology. More specifically, it relates to an abnormal image generation method and system based on anisotropic information diffusion, which realizes the controllable generation of industrial abnormal images by constructing an anisotropic structural condition field. Background Technology

[0002] Industrial visual anomaly detection is a crucial link in intelligent manufacturing, directly impacting product quality assessment, defect classification, and online screening effectiveness. A long-standing core challenge in this field is the difficulty in acquiring, covering, and accurately labeling real anomaly samples. On actual production lines, defects occur with low probability and vary widely in type, resulting in naturally scarce and unevenly distributed anomaly data. Simultaneously, pixel-level annotation requires professional personnel to meticulously draw each image, leading to high costs and long processing times. Consequently, many detection systems rely on unsupervised / semi-supervised or few-sample learning paradigms. While this makes the models acceptable in determining "whether it is an anomaly," it often significantly limits pixel-level localization accuracy and the ability to characterize defect morphology (especially structural details). To overcome this data bottleneck, anomaly generation has gradually become an important technological direction for constructing training data and improving downstream detection and localization performance.

[0003] Early cropping and pasting methods were simple to implement, but often lacked physical consistency between the generated defects and the background, leading to abrupt boundaries and inconsistent textures. Subsequently, generative models such as GANs improved the realism of appearance, but training was sensitive to data scale and stability, and lacked explicit constraints on the structural morphology of defects, making it difficult to consistently generate structurally reasonable defects. In recent years, diffusion models have been introduced into anomaly generation due to their high-fidelity generation capabilities. They can generate relatively natural anomaly appearances under constraints such as masks, becoming an important development direction for few-shot anomaly generation. However, the control signals for existing diffusion-based anomaly generation are still mainly semantic / mask-based, typically only describing where the defect is and what type it belongs to. They lack direct constraints on the fine-grained structures such as "continuous extension along the main direction and consistent texture change with direction" that strongly directional defects like cracks, scratches, and wear stripes rely on. Therefore, phenomena such as directional disorder, structural breakage, or local blurring still easily occur, thus affecting the gain effect of the generated data on pixel-level localization tasks.

[0004] To address the aforementioned issues, the proposed solution introduces anisotropic structural priors that explicitly express direction, consistency, and scale into the image's physical space. These priors are then used as conditional fields throughout the injection-diffusion denoising process, thereby constraining the local morphology and long-range structural continuity of defects during the generation stage. Simultaneously, by combining masking and category conditions, controllable generation of location and defect type is achieved. This results in anomaly images with stronger structural consistency and morphology closer to real defect features. When used for data augmentation, this effectively improves the robustness of industrial anomaly detection and significantly enhances downstream pixel-level positioning accuracy. Summary of the Invention

[0005] To address the issues of structural breakage, orientation disorder, and unnatural blending of anomalies with the background that easily occur when synthesizing highly directional defects (such as cracks and scratches) under limited sample conditions, this invention provides an anomaly image generation method and system based on anisotropic information diffusion. This invention extracts structural information such as principal orientation, anisotropic consistency, and diffusion scale from a small number of anomaly samples and their masks in the physical image space, constructing a pixel-aligned anisotropic structural feature map. This map, together with defect location features and category-time conditions, forms a structure-location-category anisotropic conditional field. This conditional field is injected into a diffusion denoising network in a multi-layered manner and optimized by combining noise regression, perceptual consistency, mask supervision, and color-structure collaborative constraints, thereby generating anomaly images with stronger structural consistency and morphology closer to real defect features. The generated anomaly samples can be used for data augmentation in industrial anomaly detection scenarios, significantly improving downstream anomaly detection performance and pixel-level positioning accuracy.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] An anomalous image generation method based on anisotropic information diffusion includes the following steps:

[0008] S1: Read a set amount of abnormal images, corresponding defect masks and category labels, and perform preprocessing such as size and numerical normalization to form standardized input;

[0009] S2: Extract anisotropic structural information such as principal direction, uniformity and diffusion scale within the defect region of the defect mask to obtain structural descriptive quantities characterizing the defect morphology;

[0010] S3: Construct the aforementioned structural description into a pixel-aligned multi-channel structural prior map, and encode the pixel-aligned structural prior map to obtain an anisotropic structural feature map. The defect mask and defect category label-diffusion time condition are encoded to obtain... The anisotropic conditional field c is obtained and used to guide the diffusion denoising network;

[0011] S4: Will The constructed conditional field uses a multi-level injection diffusion denoising network to conditionally guide the inverse denoising process, thereby simultaneously constraining the consistency of local details and global structure.

[0012] S5: The diffusion denoising network is trained and optimized based on the joint loss function to stabilize the training process and enhance the structural fidelity, thereby obtaining an anomalous image generation model (diffusion denoising model) that can generate structurally consistent anomalous images.

[0013] S6: Load the trained abnormal image generation model into inference mode, and generate abnormal images by providing the input conditions to be generated (defect mask, defect category, diffusion steps and anisotropic structure information) through the back diffusion denoising sampling process.

[0014] Preferably, in step S1, a small number of abnormal images, corresponding defect masks, and defect category labels are first acquired, wherein the abnormal images and defect masks correspond one-to-one; the abnormal images and defect masks are preprocessed, the preprocessing including: adjusting the images and masks to a preset spatial resolution and cropping them to ensure alignment of the images and masks in spatial position; normalizing the pixel values ​​of the abnormal images to a preset range; and converting the processed images and masks into a data format that can be input to the network to obtain standardized input samples for subsequent structural information extraction and diffusion model training.

[0015] Preferably, in step S2, the abnormal image is used. Taking the corresponding binary mask m as input, local structural information is extracted only within the defect region defined by the mask to obtain the principal direction required for subsequent construction of the anisotropic structure prior. Structural consistency (COH) and diffusion scale .

[0016] Furthermore, step S2 specifically includes the following sub-steps:

[0017] S21: Obtain the abnormal image according to step S1. And its corresponding binary mask m. To highlight the structural texture of the defect itself, the anomalous image is first converted into a grayscale image and only the pixels within the mask area are retained, thereby suppressing the interference of background texture on the structural orientation estimation. Then, the mask grayscale image is Gaussian smoothed to reduce noise, and the Sobel operator is used to calculate the gradient information in the horizontal and vertical directions. The calculation formula is:

[0018] Where I represents the grayscale image of the mask after Gaussian smoothing.

[0019] S22: To obtain more stable local structural features, the gradient information is first smoothed and a structure tensor is constructed to comprehensively characterize the directional distribution characteristics within the local region. Based on this, the local principal direction (axial direction) and structural consistency of each pixel are calculated according to the components of the structure tensor. The principal direction is used to depict the main direction of the local texture, and the structural consistency is used to measure the salience of directionality. Their calculation forms are as follows:

[0020]

[0021] in, To prevent tiny constants with a denominator of zero, The structural components are calculated from gradient information. The structural consistency value reflects the directional reliability of local regions; stronger directionality indicates higher consistency, thus providing a basis for subsequent prior structural modeling. The formula for coh is to calculate the coh value of each pixel in the abnormal region, divide it into multiple patch blocks, and then calculate the coh value within each patch block. The median of the coh values ​​of all pixels within each patch block is obtained by taking the median of the coh values ​​of each pixel within that patch block.

[0022] S23: Divide the image into non-overlapping segments. (A patch refers to a local image block obtained by dividing an abnormal image into fixed-size regions. It is used to statistically analyze the structural orientation information of defects within a local area and is a conventional image processing partitioning method.) Insufficient mask coverage is ignored. For a valid patch, define Aggregation is performed using the mean of two-angled circles:

[0023]

[0024] consistency Median aggregation is used within the patch to enhance robustness to a small number of anomalous noise points or local extrema. Here, E represents the mean statistical calculation of the direction value θ for each pixel within the effective patch, used to obtain the principal direction estimate at the patch level, and θ(p) represents the principal direction θ of the local structure at pixel p.

[0025] S24: Perform Euclidean distance transformation on the mask area to characterize the local scale of defects, obtain the distance from the pixel within the mask to the mask boundary, and thus obtain the horizontal diffusion scale. for:

[0026]

[0027] in, The geometric scale representing the defects within a patch is obtained by taking the median Euclidean distance of all pixels within the patch, and is then applied based on a consistency threshold. Define the vertical diffusion scale:

[0028]

[0029] in, The scaling factor. As a regulating factor, This indicates the geometric scale of defects within a patch.

[0030] when When the scale is degenerated to isotropic to avoid directional constraints; when Anisotropic scales are used to match the defect directionality and geometric width, thereby providing a spatial prior consistent with the defect morphology for subsequent generation.

[0031] Preferably, in step S3, a pixel-aligned five-channel structure prior map is constructed based on the patch-level structural element information obtained in step S2. It is then mapped to pixel-aligned anisotropic structural feature maps through structural encoding. Furthermore, to explicitly introduce spatial location constraints and defect category control information for defects, the binary mask m and the defect category label y (combined with the diffusion time condition t) are respectively feature-encoded to obtain pixel-aligned positional feature maps. and category-time condition features This serves as the conditional input for the subsequent diffusion denoising network.

[0032] Furthermore, step S3 may include the following sub-steps:

[0033] S31: Write the prior structure of each valid patch back to the pixel grid to construct a five-channel structure map. ,Right now:

[0034]

[0035] Among them, diffusion scale To compress the numerical range, the logarithm is taken. In order to ensure that the structural prior only applies to the defect region and suppress background texture interference, each channel represents the local principal direction of the defect, the direction consistency, and the anisotropic diffusion scale information related to the geometric width. To avoid background texture interference, the structural prior is only effective within the mask region, and the structural prior of pixels outside the mask is set to zero, so that the subsequent structural condition injection focuses on the morphological constraints of the defect region.

[0036] S32: The constructed prior structure diagram Input structure encoder Perform feature mapping to obtain anisotropic structural feature maps aligned with pixel positions:

[0037]

[0038] Where H and W represent the height and width of the feature map, respectively, corresponding to the spatial resolution of the input image; Ce represents the number of feature channels output by the structural encoder; the structural encoder is used to convert the multi-channel structural prior into a high-dimensional structural embedding representation while maintaining the same spatial resolution as the input, and consists of three layers. The system is constructed using convolutions, with GroupNorm and SiLU activation applied between the convolutional layers. It can be invoked directly as needed during the training or inference phase.

[0039] S33: To further provide conditional information on defect location and defect category to the diffusion denoising network, the binary mask m and the defect category label y are encoded separately. Specifically, the binary mask is mapped to a pixel-aligned positional feature map to characterize the spatial distribution of the defect region; simultaneously, the diffusion time step and the defect category label are jointly encoded to obtain conditional features used to characterize the defect category and time information.

[0040] The resulting location features With category-time features Together with structural features Together they constitute the anisotropic conditional field c used for subsequent diffusion denoising network conditional injection.

[0041] Preferably, step S4 includes the following steps:

[0042] S41: In the residual blocks of each scale of U-Net (diffusion denoising network), the input features are first processed... Intermediate features are obtained through normalization, nonlinear transformation, and convolution. Where the superscript (l) represents the index of the l-th scale in U-Net, These represent the number of channels and spatial resolution corresponding to this scale, respectively; subsequently, the category-time condition is... Injected additively after linear projection and broadcasting. This introduces semantic constraints on diffusion time steps and defect categories, while also incorporating location feature maps. Aligned to the current scale and injected additively after convolutional mapping This is done to constrain the spatial alignment of the anomaly generation region with the target mask and improve boundary stability. Based on this, the anisotropic structure feature map... After aligning to the current scale, gating and bias parameters are generated, and the intermediate features are structurally modulated using a feature linear modulation method.

[0043]

[0044] in, For element-wise multiplication, This is used to limit the gating amplitude to improve training stability. Finally, the modulated features are subjected to subsequent convolutional transformation and fused with the residual branch to obtain the residual block output. This allows the structural conditions to continuously constrain the main direction, reliability, and scale prior of defects during multi-scale denoising, thereby improving the consistency of local detail structure in the generated anomalies.

[0045] S42: The technical problem of struggling to explicitly model long-distance structural associations (e.g., long scratches, continuous crack textures) solely relying on local multi-scale linear feature modulation. This invention introduces a cross-attention module into the U-Net bottleneck layer of the diffusion denoising network to achieve global aggregation and consistency constraints of structural conditions. Specifically, the bottleneck feature z is used as the query vector (Q), and the anisotropic structural feature map... This is a key / value vector (where Query represents the query representation of the current bottleneck feature, Key represents the importance identifier used for matching in the structural feature, and Value represents the corresponding structural feature content; by calculating the similarity between Query and Key, the structural information most relevant to the current feature in Value is automatically weighted and aggregated), and normalized and linearly projected to align the channel dimensions; then, scaled dot product attention is used to calculate the relevance weights between structural conditions and bottleneck features, and the value vectors are weighted and aggregated, which can be expressed as:

[0046]

[0047] Where Q is the query matrix obtained by linearly projecting the bottleneck layer feature map through a 1×1 convolution; K and V are respectively obtained by linearly projecting the bottleneck layer feature map through F... aniso The key and value matrices obtained by 1×1 convolution linear projection; d h The feature dimension of a single attention head; The similarity matrix is ​​normalized to K dimensions to obtain the attention weights. The channel dimension of the attention head is represented. Finally, the structural enhancement features obtained by the attention aggregation are mapped back to the bottleneck feature space through output projection and injected into the bottleneck features using residual fusion. A lightweight feedforward network can be further connected after the cross-attention block and stacked using residual fusion to enhance nonlinear expressive power and stabilize training. Through the above methods, the bottleneck features can be explicitly perceived and utilize the long-range directional correlation of defects, thereby complementing the multi-scale linear feature modulation and further improving the overall structural consistency.

[0048] Preferably, in step S5: the network parameters of the diffusion denoising network in the diffusion model are optimized during the model training process using a joint loss function, so that the network operates under conditional conditions. Guided by this, the training process is stabilized by simultaneously satisfying constraints on noise regression, perceptual consistency, defect region localization, and color-structure appearance consistency, thereby obtaining an anomaly image generation model for generating structurally consistent anomalies.

[0049] Furthermore, step 56 includes the following sub-steps:

[0050] S51: Set the abnormal image as the diffusion target , (x a This is the actual abnormal image read in step S1, where x is... a Let x0 represent the "clean image" of the diffusion target, i.e., a noise-free, real anomalous image. During training, diffusion time steps t are randomly sampled and Gaussian noise is sampled. ( Let represent Gaussian noise with a mean of 0 and a covariance of identity matrix I; where I represents the identity matrix and indicates that each dimension is independently and identically distributed; that is, this distribution yields Gaussian noise ε). Noisy samples are constructed according to the forward diffusion process:

[0051]

[0052] in, This is a noise diffusion scheduling parameter used to control the noise intensity at time step t.

[0053] S52: The The time step t and the conditional field c are input into the diffusion denoising network to obtain the noise prediction. and defect mask prediction Noise prediction drives the inverse denoising learning, while mask prediction explicitly constrains the spatial localization and shape consistency of defect regions. A joint loss function is constructed and used for training optimization; the joint loss is:

[0054]

[0055] in, These are loss weights used to balance the impact of various constraints on training.

[0056] S53: Calculate adaptive noise loss This spatially weights the noise regression, making the model focus more on the detailed restoration of defective areas and structurally distorted regions.

[0057]

[0058] in, Indicates pixel position, Pixel weights; The structural consistency coh construction obtained from steps S2 / S3 is used to emphasize defective regions or structurally unstable regions, thereby improving the local structural morphological consistency of generated defects. This represents the actual noise at pixel position p. This represents the noise predicted by the denoising network at pixel location p.

[0059] S54: Calculate the perceptual consistency loss, mask supervision loss, and color-structure collaborative loss, and use them together with the noise loss to update the network parameters. Perceptual consistency loss The texture and appearance of the generated results are constrained in the perceptual feature space, making the generated anomalies more similar to real anomalies in terms of overall texture style and local structural appearance. To this end, the texture and appearance are first derived by inversely from noise prediction. Estimate:

[0060]

[0061] And construct LPIPS is a learned perceptual image patch similarity metric used to measure the perceptual difference between two images in a pre-trained feature space; mask-supervised loss. The defect mask used for the output of the explicit constraint network is consistent with the real mask, thereby enhancing the spatial localization stability and boundary consistency of the defect region. This ensures that anomalies are mainly generated within the mask region and avoids overflow, defined as: Color-structure cooperative loss This technique, used to simultaneously constrain color shift and structural similarity within a masked area, makes the generated anomalies more closely resemble real defects in terms of color variations and structural details, thereby improving the naturalness of the anomaly-background blend. Its form is as follows:

[0062]

[0063] in, For element-wise multiplication, For the color difference of the mask area, This serves as a structural similarity index. Finally, an optimization algorithm (such as an optimizer) is used to iteratively update the network parameters to minimize... The optimizer preferably uses AdamW, and a learning rate scheduling strategy is used to adaptively adjust the training process to obtain a trained abnormal image generation model.

[0064] Preferably, in step S6: the trained diffusion denoising network is loaded into the inference mode, given the target defect mask m, the category label y, and the anisotropic structure conditions extracted from a small number of abnormal samples and encoded in step S3. The sampling step t is used to generate anomaly images by performing backdiffusion iterations from random Gaussian noise initialization and employing the DDIM (Denoising Diffusion Implicit Models) sampling strategy. Specifically, at time step t, the current sample... Input diffusion denoising network with time step t and conditional field c, and network output noise prediction. The noise-free image is estimated by first predicting the noise and then working backwards from that prediction. (Consistent with the calculation formula in step S5 of the training phase) Subsequently, update from DDIM. Iteratively obtained :

[0065]

[0066] in, The randomness control term for DDIM can be determined by parameters. Given:

[0067]

[0068] Repeat the above reverse iteration until... Obtain the generated abnormal image The conditional field *c* serves as the conditional guidance input during the inference phase, comprising structural features, positional features, and a category-temporal condition. Structural features constrain the orientation and scale consistency of the defect morphology; positional features constrain the alignment of the anomaly generation region with the mask space; and the category-temporal condition controls the defect type and, together with the temporal step information, stabilizes the backdiffusion denoising process, thereby generating anomaly images with structural morphology, directional continuity, and texture appearance highly consistent with real defects. Through the aforementioned DDIM conditional sampling and anisotropic conditional field guidance mechanism, the final generated anomaly image exhibits structural morphology, directional continuity, and texture appearance highly consistent with real defects within the defect region, significantly improving the realism and structural consistency of the anomaly image.

[0069] This invention also discloses an anomaly image generation system based on anisotropic information diffusion, used to execute the above method, comprising the following modules: a preprocessing module: reading a set amount of anomaly images, corresponding defect masks, and defect category labels, and performing preprocessing to form standardized input; a structural descriptor representation module: extracting anisotropic structural information within the defect region of the defect mask to obtain a structural descriptor representing the defect morphology; and an anisotropic conditional field acquisition module: constructing the structural descriptor into a pixel-aligned multi-channel structural prior map, and encoding the pixel-aligned structural prior map to obtain the anisotropic structure. The F is obtained by encoding the defect mask and the defect category label-diffusion time condition.pos With F cla The anisotropic conditional field c is obtained and used to guide the diffusion denoising network; the structural consistency constraint module: converts the anisotropic structural feature map F aniso Location feature map F pos Category-Time Conditional Feature F cla The conditional field is injected into the diffusion denoising network at multiple levels to simultaneously constrain the consistency of local details and global structure; the training and optimization module trains and optimizes the diffusion denoising network based on the joint loss function to obtain the abnormal image generation model; the abnormal image generation module loads the trained abnormal image generation model into the inference mode, and given the conditional input to be generated, including the defect mask, defect category, diffusion steps and anisotropic structural information, it generates abnormal images through the reverse diffusion denoising sampling process.

[0070] In summary, by introducing orientation, consistency, and scale structure priors into the physical image space, this invention can generate abnormal images with stronger structural consistency and morphology closer to real defect features under conditions of few samples, thereby effectively improving the data augmentation effect and model training performance of industrial anomaly detection and pixel-level localization tasks. Attached Figure Description

[0071] Figure 1 This is a flowchart illustrating a preferred embodiment of the present invention for an abnormal image generation method based on anisotropic information diffusion.

[0072] Figure 2 This is a general framework diagram of an anomaly image generation method based on anisotropic information diffusion, which is a preferred embodiment of the present invention. Figure 3 This is an example image of an abnormal image generated according to a preferred embodiment of the present invention.

[0073] Figure 4 for Figure 3 The grayscale image.

[0074] Figure 5 This is a block diagram of an abnormal image generation system based on anisotropic information diffusion, which is a preferred embodiment of the present invention. Detailed Implementation

[0075] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. The drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate the embodiments, some parts of the drawings may be omitted, enlarged, or reduced, and do not represent actual dimensions. It is understandable to those skilled in the art that some well-known content may be omitted in the drawings.

[0076] like Figure 1 As shown, this embodiment relates to an anomaly image generation method based on anisotropic information diffusion, which is carried out according to the following steps:

[0077] S1. Data reading and preprocessing;

[0078] S2. Extraction of anisotropic structural information;

[0079] S3. Construction and encoding of anisotropic conditional fields;

[0080] S4, anisotropic conditional field multi-level injection diffusion denoising network;

[0081] S5. Training of the anisotropic joint loss constraint model;

[0082] S6, Abnormal image generation and output.

[0083] The following is a detailed explanation of each step.

[0084] Step S1 specifically includes:

[0085] Read a small number of anomaly images from publicly available industrial anomaly detection datasets or real anomaly datasets collected from industrial sites. The defect mask m and defect category label y are used to correspond to each other. The abnormal image and the defect mask are then uniformly adjusted to the preset spatial resolution. (This embodiment takes) And, by cropping, ensure that the image and the mask are strictly aligned in space; normalize the pixel values ​​of abnormal images to Range; convert the processed image and mask into a data format that the network can input, where the mask is represented by a single-channel binary tensor (1 inside the mask and 0 outside the mask), thereby obtaining standardized input samples for subsequent structural information extraction and diffusion model training.

[0086] Step S2 specifically includes:

[0087] Obtain the abnormal image according to step S1. And its corresponding binary mask m. To highlight the structural texture of the defect itself, the anomalous image is first converted into a grayscale image and only pixels within the mask area are retained, thereby suppressing the interference of background texture on structural orientation estimation. To reduce the influence of noise, Gaussian smoothing is performed on the grayscale image after mask constraint, with a smoothing scale of [value missing]. It is implemented using separable Gaussian convolution, with the convolution radius set according to... The truncation was determined to reduce computation while maintaining a smooth effect. Then, the classical... The operator convolves the grayscale image, calculates the gradient components in the horizontal and vertical directions respectively, and normalizes the resulting gradient magnitudes to ensure that the gradient dimensions are consistent between different samples, thereby improving the stability of subsequent structure estimation.

[0088] Next, components of the structure tensor are constructed based on the gradient, and each component is further Gaussian-weighted smoothed to stabilize the structure estimation. The smoothing scale is set to... ,Right now:

[0089]

[0090] in, It is a second-order statistic of the gradient. These are structural components. Based on this, the local principal direction (axial direction) of each pixel is calculated. and structural consistency coh:

[0091]

[0092] in, To prevent tiny constants with a denominator of zero.

[0093] To obtain more robust structural priors under few-sample conditions, the image is divided into non-overlapping segments. The patch (in this embodiment) The system calculates the mask coverage ratio for each patch; when the mask coverage ratio within a patch is lower than a preset threshold of 0.05, that patch is not included in the structural information output. For valid patches, the corresponding pixels within the patch's mask are extracted. , among which direction Aggregation is performed using the mean of two-angled circles, i.e.:

[0094]

[0095] consistency Median aggregation is used within the patch to enhance robustness to a small number of anomalous noise points or local extrema. Furthermore, to ensure that the structural prior reflects the geometric width and diffusion scale characteristics of defects, an Euclidean distance transformation is performed on the binary mask to obtain the distance from pixels within the mask to the mask boundary, thereby obtaining the horizontal diffusion scale. for:

[0096]

[0097] in, The geometric scale of defects within a patch is represented by the median Euclidean distance of all pixels within the patch, and the scaling factor is used. In addition, vertical diffusion scale for:

[0098]

[0099] in, Gating threshold .

[0100] Step S3 specifically includes:

[0101] Using the patch-level structuring information obtained in step S2, a pixel-aligned five-channel structure map is constructed. and through the structural encoder Map it to a pixel-aligned anisotropic structure feature map This serves as the structural condition input for the subsequent diffusion denoising network. Specifically, the structural priors of each valid patch are first written back to the pixel grid to construct a five-channel structure map. ,Right now:

[0102]

[0103] Among them, diffusion scale To compress the numerical range, the logarithm is taken. To ensure that the structural prior only acts on the defect region and suppress background texture interference, the structural prior is set to a zero vector outside the mask region, i.e., when... season Subsequently, Lightweight input convolutional encoder The encoder consists of three layers The system is constructed using convolutions, with GroupNorm and SiLU activation applied between convolutional layers to maximize the number of channels. The structure is embedded while maintaining the same spatial resolution as the input image, thus obtaining a pixel-aligned anisotropic structural feature map:

[0104]

[0105] Furthermore, to introduce explicit conditional information about defect location and defect category, the binary mask m and the defect category label y are encoded separately to obtain pixel-aligned positional feature maps. and category-time condition features ,Right now:

[0106] in, It is a shallow convolutional encoder, consisting of two layers. Convolutional layers are constructed using SiLU activation, which is used to map binary masks to continuous position embeddings. At time step t, the vector is first encoded by a sinusoidal positional encoder PE(t) and then input into an MLP to obtain a time vector of dimension d. The category label y is obtained through a learnable embedding matrix. Mapped to category vectors of the same dimension And then add it to the time vector and fuse it. Therefore, Together they constitute an anisotropic conditional field This is used in the subsequent step S4 to inject the diffusion denoising network.

[0107] Step S4 specifically includes:

[0108] In this embodiment, a diffusion denoising network is first constructed (see...). Figure 2 The basic structure of the overall framework (middle part) is as follows: the denoising network adopts the U-Net backbone structure, the baseline channel number is set to 128, each scale contains 2 residual blocks, and dropout with a weight of 0.1 is set to enhance the generalization ability.

[0109] Within the residual blocks of each scale in U-Net, for the input features First, normalization and nonlinear transformation are performed, followed by convolution to obtain intermediate features. Wherein, the superscript (l) represents the index of the l-th scale in U-Net. These represent the number of channels and spatial resolution corresponding to this scale, respectively. Subsequently, the category-time condition will be... Projected onto the current channel dimension via a linear mapping, and then injected additively into the spatial dimension after broadcasting. This allows for the explicit introduction of semantic constraints on diffusion time steps and defect categories at various scales; simultaneously, it integrates location feature maps... Align to the current scale via interpolation. and through Convolutional mapping to After the channel is opened, it is injected additively. This constrains the spatial alignment between the anomaly generation region and the target mask and improves boundary stability.

[0110] Based on this, the anisotropic structure feature map After alignment to the current scale, gating and bias parameters are generated, and the intermediate features are structurally modulated using a feature linear modulation method, in the form of:

[0111]

[0112] Where l represents the l-th scale of U-Net; This represents an intermediate feature of the residual block at this scale; These are respectively aligned to the current scale. The gating parameters and bias parameters obtained by convolutional projection and channel-dimensional splitting are, and... Consistent dimensions; This is the scaling factor; For element-wise multiplication, This is used to limit the gating amplitude to improve training stability. Finally, the modulated features are subjected to subsequent convolutional transformations and fused with the residual branch to obtain a combined feature. The residual block output is linearly modulated for each feature, thereby enabling the local, layer-by-layer injection of structural priors at each scale.

[0113] Furthermore, a bottleneck layer of U-Net is introduced with For key / value cross-attention to aggregate long-range structural associations, in this embodiment, bottleneck layer features are used as the query, and the structural feature map is used as the query. As the key and value, the cross-attention uses four attention heads, and through... Convolution completes the projection and calculates the attention weights, which can be expressed in the following form:

[0114]

[0115] Where Q is the feature map generated from the bottleneck layer. The query matrix obtained by convolutional linear projection; K and V are respectively derived from... The key matrix and value matrix obtained by convolutional linear projection; The feature dimension of a single attention head is used to normalize and scale the dot product similarity. This indicates that the similarity matrix is ​​normalized by the Key dimension to obtain attention weights, which are used to aggregate anisotropic structural context information in a global scope. At the same time, a dropout with a weight of 0.1 is set to enhance training stability.

[0116] The attention output, after being projected, is added to the feature residual of the bottleneck layer. If necessary, an FFN feedforward network with an expansion coefficient of 2 is cascaded afterward and the residuals are superimposed to enhance expressive power. Through this bottleneck cross-attention mechanism, the network can aggregate structural context information globally, explicitly capturing the long-range directional correlations of defects such as long scratches and continuous cracks, complementing the linear modulation of features, thereby simultaneously constraining the consistency of local details and the overall structure.

[0117] Step S5 specifically includes:

[0118] Step S5 specifically includes: setting up the diffusion process and noise scheduling, constructing forward diffusion training samples, and performing denoising training and joint loss optimization based on the anisotropic conditional field. This embodiment uses linear noise scheduling. At that time step, the general linear change to During training, the network receives noisy input. Time step t and condition field Output noise prediction and Gaussian noise Construct a noisy input:

[0119]

[0120] in, Time step t and condition field Input the noise to the diffusion denoising network to obtain noise prediction. and mask prediction .

[0121] Next, a joint loss function is constructed to optimize the network parameters during training. The total loss is based on noise regression loss, supplemented by perceptual consistency, mask supervision, and color-structure co-constraint. The weights of each loss term are configured as follows: perceptual consistency loss weight is 0.1, color consistency loss weight is 0.1, structural similarity loss weight is 0.1, and mask supervision loss adopts a combination of BCE and Dice. The joint loss is defined as follows:

[0122]

[0123] in, Adaptive noise regression loss The calculation form is:

[0124]

[0125] Where p represents the pixel position and w(p) is the pixel weight, constructed from the structural consistency coh obtained in step S2; To constrain the mask monitoring loss, a combination of BCE and Dice is used. Consistency with the actual mask m; To achieve perceptual consistency constraints, the noise-free image estimate is first derived by inversely calculating the noise prediction.

[0126]

[0127] Recalculate The goal is to generate textures that closely resemble real defects, where LPIPS is a learned perceptual image patch similarity index used to measure the perceptual difference between two images in a pre-trained feature space. For color-structure co-constraint, color offset and structural similarity are simultaneously constrained within the mask region. Color difference is obtained by converting RGB to Lab color space and calculating the error in the chroma channel, while structural similarity is calculated within the mask region. get.

[0128] Finally, under the aforementioned loss constraints, an optimizer iteratively updates the network parameters until convergence. In this embodiment, a learning rate of is used. optimizer, weight decay The batch size is set to 4, and the ReduceLROnPlateau learning rate scheduling strategy is adopted. Adaptive learning rate reduction is performed; in single-class training scenarios, each class is trained for 2000 epochs, and the number of data loading threads is set to num_workers=4. After training in the diffusion denoising network as described above, an abnormal image generation model (diffusion denoising model) is obtained, which is used for abnormal image generation in the subsequent inference stage.

[0129] Step S6 specifically includes:

[0130] In this embodiment, the abnormal image generation model trained in step S5 is switched to inference mode. Given a target defect mask m, a category label y, and anisotropic structural conditions extracted from a small number of abnormal samples and encoded in step S3, the model is used to generate the abnormal image. The sampling strategy employs DDIM (Denoising Diffusion Implicit Models) conditional sampling to perform inverse denoising. DDIM is a well-known sampling strategy in the field of diffusion models, which maintains generation quality while reducing the number of sampling steps through non-Markovian deterministic / semi-random updates. The parameters... To obtain deterministic sampling, which is more conducive to maintaining structural continuity and morphological consistency. The input resolution is fixed during inference. Noise scheduling adopts linear , range And calculate accordingly At the start of sampling, starting from the standard Gaussian noise image, for any reverse time step (sampling step number t = 600), the current sample... The time step t and the conditional field c are input into the diffusion denoising network. Based on this noise prediction, an estimate of the noise-free image is first derived from the noise prediction. (Consistent with the calculation formula in step S5 of the training phase), and further updated according to the DDIM update rules from Iteratively obtained :

[0131]

[0132] in, The randomness control term for DDIM can be determined by parameters. Given:

[0133]

[0134] Repeat the above reverse iteration until t = 0 to obtain the generated abnormal image. This refers to anomaly RGB images generated under the constraints of defect masks and category conditions, and guided by anisotropic structural conditions. Through the aforementioned DDIM conditional sampling and anisotropic conditional field guidance mechanism, the generated anomaly images possess structural morphology, directional continuity, and texture appearance highly consistent with real defects within the defect region, thereby significantly improving the realism and structural consistency of the anomaly images.

[0135] This invention addresses the task of synthesizing few-shot anomaly images in industrial vision scenarios. It implements and trains an anomaly image generation model based on anisotropic information diffusion on a general-purpose computer platform. Experiments are presented below. The computer platform was configured with an NVIDIA GeForce RTX 4090 (24GB VRAM) graphics processor and Ubuntu 24.04 operating system. The method of this invention is implemented based on the deep learning framework PyTorch, and model training and image generation are completed on a self-implemented U-Net structure diffusion training framework. Experimental results are as follows... Figure 3 , 4 As shown, this invention constructs a pixel-aligned anisotropic conditional field and injects it into a diffusion denoising network in a multi-layered manner to achieve controllable guidance of abnormal morphology and spatial location. Combined with joint loss constraints, it improves training stability and structural fidelity. Thus, this invention can still generate abnormal images with continuous structure, consistent texture, and high alignment with the mask even under conditions of few samples. Furthermore, the generated samples can significantly improve the performance of downstream anomaly classification, detection, and localization when used for data augmentation.

[0136] like Figure 5 As shown, this embodiment discloses an anomaly image generation system based on anisotropic information diffusion, used to execute the above method. It includes the following modules: a preprocessing module: reading a set amount of anomaly images, corresponding defect masks, and defect category labels, and performing preprocessing to form standardized input; a structural descriptor representation module: extracting anisotropic structural information within the defect region of the defect mask to obtain a structural descriptor representing the defect morphology; and an anisotropic conditional field acquisition module: constructing the structural descriptor into a pixel-aligned multi-channel structural prior map, and encoding the pixel-aligned structural prior map to obtain the anisotropic structure. The F is obtained by encoding the defect mask and the defect category label-diffusion time condition. pos With F cla The anisotropic conditional field c is obtained and used to guide the diffusion denoising network; the structural consistency constraint module: converts the anisotropic structural feature map F aniso Location feature map F pos Category-Time Conditional Feature F claThe conditional field is injected into the diffusion denoising network at multiple levels to simultaneously constrain the consistency of local details and global structure. The training and optimization module trains and optimizes the diffusion denoising network based on the joint loss function to obtain the abnormal image generation model. The abnormal image generation module loads the trained abnormal image generation model into the inference mode. Given the conditional input to be generated, including the defect mask, defect category, diffusion steps, and anisotropic structural information, the abnormal image is generated through the reverse diffusion denoising sampling process.

[0137] Other aspects of this embodiment can be found in the above method embodiments.

[0138] In summary, this invention belongs to the field of deep generative models and industrial visual inspection technology. It aims to achieve highly consistent and controllable synthesis of industrial anomaly images under limited sample conditions by explicitly modeling the anisotropic structure prior of the defect region and integrating it throughout the diffusion generation process. Specifically, it proposes an anomaly image generation method and system based on anisotropic information diffusion. The method includes: S1, data acquisition and preprocessing; S2, extracting anisotropic structural information of the main direction, structural consistency, and diffusion scale within the defect mask-defined region; S3, constructing and encoding a pixel-aligned anisotropic conditional field, combining structural, positional, and category information to guide the generation of the diffusion denoising network; S4, injecting the anisotropic conditional field, composed of structural features, positional features, and category-time conditions, into the diffusion denoising network at multiple levels to simultaneously constrain local details and global structural consistency; S5, training and optimizing the model based on a joint loss function of noise regression, perceptual consistency, mask supervision, and color-structure co-constraints; and S6, generating anomaly images according to the anomaly image generation model.

[0139] The preferred embodiments and principles of the present invention have been described in detail above. For those skilled in the art, there may be changes in the specific implementation based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating abnormal images based on anisotropic information diffusion, characterized in that, The process includes the following steps: S1: Read the acquired abnormal image, the corresponding defect mask, and the defect category label, and preprocess them to form a standardized input; S2: Extract anisotropic structural information from the defect region of the defect mask to obtain a structural descriptor representing the defect morphology; S3: Construct the structural descriptor into a pixel-aligned multi-channel structural prior map, and encode the pixel-aligned structural prior map to obtain an anisotropic structural feature map. The F is obtained by encoding the defect mask and the defect category label-diffusion time condition. pos With F cla S4: Obtain the anisotropic conditional field c to guide the diffusion denoising network; S5: Apply the anisotropic structural feature map F aniso Location feature map F pos Category-Time Conditional Feature F cla The conditional field is injected into the diffusion denoising network in a multi-layered manner to simultaneously constrain the consistency of local details and global structure; S5: The diffusion denoising network is trained and optimized based on the joint loss function to obtain the abnormal image generation model; S6: Load the trained abnormal image generation model into inference mode. Given the conditions to be generated, including defect mask, defect category, diffusion steps and anisotropic structural information, generate abnormal images through the back diffusion denoising sampling process.

2. The method for generating abnormal images based on anisotropic information diffusion according to claim 1, characterized in that: In step S1, the preprocessing is as follows: the abnormal image and the defect mask are normalized in size and cropped; the abnormal image is normalized in pixel value; and the processed image and the defect mask are converted into a data format that can be input to the diffusion denoising network to obtain standardized input samples.

3. The method for generating abnormal images based on anisotropic information diffusion according to claim 1 or 2, characterized in that: Step S2 is as follows: S21: Using the abnormal image x a The structural information is extracted within the defect region defined by the mask, and the defect region is smoothed and gradients are calculated. S22: Calculate the pixel-level local principal direction θ and structural consistency coh based on the structure tensor, with the following formulas: Among them, J 11 J 12 J 22 It is a structural component calculated from gradient information, where ε is a constant; S23: Divide the abnormal image read in step S1 into non-overlapping patches, and aggregate the direction θ and consistency coh of the effective patches to obtain patch-level structural metadata. Specifically, for effective mask pixel set The main directions at the patch level are obtained by using bi-angular circular mean aggregation. : Where E represents the average value of the direction value θ calculated for each pixel within the effective patch, and θ(p) represents the main local structural direction θ at pixel p; S24: Perform Euclidean distance transformation on the masked region of step S1 to estimate the local scale of the defect, and adaptively determine the diffusion scale in the parallel direction. Diffusion scale in the vertical direction Specifically: A Euclidean distance transformation is performed on the mask region to characterize the local scale of defects, obtaining the distance from pixels within the mask to the mask boundary, thereby obtaining the horizontal diffusion scale. for: in, The geometric scale representing the defects within a patch is obtained by taking the median Euclidean distance of all pixels within the patch; and is then determined based on a consistency threshold. Define the vertical diffusion scale: in, The scaling factor. As a regulating factor, This indicates the geometric scale of defects within a patch.

4. The method for generating abnormal images based on anisotropic information diffusion according to claim 3, characterized in that: Step S3 is as follows: S31: Write the structural prior of each valid patch back to the pixel grid to construct a five-channel structural prior map. : S32: Transfer the prior diagram of the structure F str The input structure encoder Estr(•) produces pixel-aligned anisotropic structure feature maps: Where H and W represent the height and width of the feature map, respectively, and C e This represents the number of feature channels output by the structure encoder; S33: Encode the defect mask to obtain a location feature map, and fuse the defect category label and diffusion time step to obtain category-time conditional features, in the following forms: in, It is a location feature. It is a category-time condition feature. It is a shallow convolutional encoder used for mapping binary masks to continuous position embeddings; It is a time-step encoder used to map diffusion time step t into continuous features; It is a category encoder used to map category labels y to continuous features; The resulting location features With category-time features Together with structural features Together they constitute the anisotropic conditional field c used for subsequent diffusion denoising network conditional injection.

5. The method for generating abnormal images based on anisotropic information diffusion according to claim 4, characterized in that: Step S4 includes the following sub-steps: S41: In the residual blocks of each scale of U-Net, the input features are first processed... Intermediate features are obtained through normalization, nonlinear transformation, and convolution. Category-Time Conditions Injected additively after linear projection and broadcasting. At the same time, location features Aligned to the current scale and injected additively after convolutional mapping Then the structural feature map After alignment to the current scale, gating and bias parameters are generated, and the intermediate features are structurally modulated using a feature linear modulation method, in the form of: Where l represents the l-th scale of U-Net; Intermediate features of the residual block at this scale; g (l) With s (l) These are the gating parameters and bias parameters, respectively, and are related to h. (l) Consistent dimensions; γ (l) ∈[0,2] represents the scaling factor; For element-wise multiplication, Used to limit the gating amplitude; S42: Introduce cross-attention in the bottleneck layer of U-Net, using the bottleneck features as the query and the structural feature map as the input. Global structure aggregation for Key / Value pairs, using multi-head attention: Where Q is the query matrix Query obtained by linearly projecting the bottleneck layer feature map through a 1×1 convolution; K and V are respectively obtained by linearly projecting the bottleneck layer feature map through F... aniso The key matrix Key and value matrix Value are obtained by 1×1 convolution linear projection; d h The feature dimension of a single attention head; Softmax(•) represents normalizing the similarity matrix in K dimensions to obtain the attention weights; The attention output is projected and then residually fused with the bottleneck layer features.

6. The method for generating abnormal images based on anisotropic information diffusion according to claim 5, characterized in that: Step S5 includes the following sub-steps: S51: Obtain the actual abnormal image read in step S1 Set as the diffusion target, that is: During training, random sampling diffusion time step t and Gaussian noise Construct noisy samples: in, The noise dissipation scheduling parameter is x0, which is the actual anomaly image read. S52: The The time step t and the conditional field c are input into the diffusion denoising network to obtain the noise prediction. and defect mask prediction ; S53: Construct a joint loss function and use it for training optimization. The joint loss is: in, To lose weight, To mask the monitoring loss, For color-structure cooperative loss, To perceive consistency loss; Adaptive noise loss Using spatially weighted mean square error: Where p represents the pixel position; These are pixel weights, where H and W represent the height and width of the input image, respectively. This represents the actual noise at pixel position p; This represents the noise predicted by the denoising network at pixel location p; S54: Calculate the perceived consistency loss Mask monitoring loss and color-structure co-loss Together with the adaptive noise loss, it forms the joint loss function during the training phase, which is used to update the parameters of the diffusion denoising network. Among them, the perceptual consistency loss is calculated. First, the noise predicted is used to infer the noise level from the input image. The estimate is: And construct LPIPS is a learned perceptual image patch similarity metric used to measure the perceptual difference between two images in a pre-trained feature space. The mask supervision loss is used to constrain the consistency between the generated result and the real defect mask in spatial location, so as to enhance the localization stability of the defect region and suppress the spread of anomalies to the region outside the mask. The color-structure collaborative loss is used to simultaneously constrain color variation and structural similarity within the defect mask region, thereby improving the naturalness of the fusion between the generated anomaly and the background region. Based on the joint loss function, an optimization algorithm is used to iteratively update the parameters of the diffusion denoising network to obtain a trained abnormal image generation model.

7. The method for generating abnormal images based on anisotropic information diffusion according to claim 6, characterized in that: Step S6 is as follows: At time step t, the current sample Input diffusion denoising network with time step t and conditional field c, and network output noise prediction. The noise-free image is estimated by first predicting the noise and then working backwards from that prediction. Subsequently, following the DDIM sampling strategy, from x t Updated to x t-1 : in, For randomness control terms, determined by parameters Given: Repeat the above reverse iteration until t = 0 to obtain the generated abnormal image. The conditional field c is the conditional guidance input for the inference stage, which consists of structural features, positional features, and category-temporal conditions. Structural features are used to constrain the orientation and scale consistency of the defect morphology, positional features are used to constrain the alignment of the anomaly generation region with the mask space, and category-temporal conditions are used to control the defect type and stabilize the back-diffusion denoising process together with the time step information, thereby generating an anomaly image with a structural morphology, directional continuity, and texture appearance that are highly consistent with the real defect.

8. An anomaly image generation system based on anisotropic information diffusion, used to perform the method as described in any one of claims 1-7, characterized in that, It includes the following modules: Preprocessing module: reads a set amount of abnormal images, corresponding defect masks and defect category labels, and performs preprocessing to form standardized input; The structural descriptor representation module extracts anisotropic structural information within the defect region of the defect mask to obtain a structural descriptor representing the defect morphology. The anisotropic conditional field acquisition module constructs the structural descriptor into a pixel-aligned multi-channel structural prior map, and encodes the pixel-aligned structural prior map to obtain the anisotropic structural feature map F. aniso The F is obtained by encoding the defect mask and the defect category label-diffusion time condition. pos With F cla The anisotropic conditional field c is obtained and used to guide the diffusion denoising network; Structural consistency constraint module: converts the anisotropic structural feature map F aniso Location feature map F pos Category-Time Conditional Feature F cla The conditional field is constructed by injecting a multi-level diffusion denoising network to simultaneously constrain local details and global structural consistency. Training and optimization module: The diffusion denoising network is trained and optimized based on the joint loss function to obtain the abnormal image generation model; Abnormal image generation module: The trained abnormal image generation model is loaded into the inference mode. Given the conditions to be generated, including the defect mask, defect category, diffusion steps and anisotropic structural information, the abnormal image is generated through the reverse diffusion denoising sampling process.

Images