An industrial image anomaly intelligent detection method and device

Abstract

The application discloses an industrial image anomaly intelligent detection method and device, an abnormal image generation framework based on prompt learning, through the introduction of prompt learning and a mixed training strategy, the learning process is more focused on abnormal features rather than global expression, more correct abnormal information is learned while the diversity of abnormal images is increased, and the generated pictures remain reasonable authenticity; secondly, a pixel-level mask matching framework is designed, the image generation process is constrained through cross-attention matching loss, and a mask prediction module is introduced, accurate abnormal image mask pairs are provided for each abnormal type; finally, the generated image-mask pairs are used to train and evaluate the final unified supervised segmentation model, and the detection ability of subtle abnormalities such as texture and edge is improved. The three complement each other and progress layer by layer, and jointly constitute the overall technical system of the application method for improving the generalization, robustness and explainability of industrial anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to an intelligent method and apparatus for detecting anomalies in industrial images. Background Technology

[0002] Industrial anomaly detection typically involves identifying and locating defects through image inspection after the manufacturing process is complete. Traditional manual inspection suffers from problems such as fatigue-related false positives, high costs, and low efficiency. While deep learning methods have improved detection capabilities, industrial defects are characterized by "small sample sizes, weak saliency, diverse shapes, numerous categories, and the need for pixel-level localization," posing challenges to model training and generalization. At the technical level, common industrial anomaly detection methods can be categorized as follows:

[0003] Classification / Self-Supervised Approaches: Self-supervised methods are representative of classification-based approaches, using models trained through surrogate tasks to detect anomalies. Surrogate tasks include image transformation prediction, contrastive learning, and surrogate binary classification. These methods rely on pre-trained models and sufficient training data because classification-based methods need to identify feature classification boundaries in normal data and use data augmentation to generate pseudo-anomaly data, which is then used in a binary classification surrogate task with normal training samples to train the feature extraction model. Because of its reliance on surrogate tasks and data augmentation to construct pseudo-anomalies, it is sensitive to data domain transfer.

[0004] Reconstruction-based methods: These methods follow a relatively consistent paradigm, training the reconstruction network only with normal samples. However, since they cannot effectively reconstruct anomalous regions in the input image, they can locate anomalies by comparing the original and reconstructed images to obtain the reconstruction error. But in cases of complex image structures or textures, the network may produce simple duplication instead of selective reconstruction. Furthermore, inherent style differences between the original and reconstructed images may also lead to false positives or undetected anomalies.

[0005] Generative approaches: Few-shot learning, as a mainstream anomaly detection method, has solved the challenges of data collection and annotation to a certain extent, which greatly benefits its application in real-world industrial scenarios. On the one hand, researching few-shot learning can reduce the cost of data collection and annotation for industrial products. On the other hand, it can address the problem from a data perspective and investigate what kind of data is most valuable for anomaly detection in industrial images. One particularly noteworthy area of ​​research is supplementing few-shot learning by artificially generating anomaly images to improve the performance of models with limited data. While few-shot learning focuses on improving models with fixed data, generative approaches focus on artificially adding reliable data to improve model performance when training data is insufficient. Currently, there are already some anomaly detection works based on generative methods, such as the DRAEM model, the RealNet model, and the SimpleNet model.

[0006] However, existing technologies for industrial anomaly detection (IAD) face challenges in practical application in real-world production environments, as follows:

[0007] 1. Insufficient semantic authenticity: There is a significant semantic gap between existing “synthetic anomalies” and “real anomalies”. The generated anomalies are not realistic enough or lack diversity. Copy / fusion pseudo-anomalies are difficult to match the physical form and semantic distribution of real defects and are difficult to use as high-quality supervised samples for downstream segmentation training, resulting in training-test distribution shift.

[0008] 2. High cost of covering multiple anomaly types: The scarcity of anomaly samples and the variability of anomaly types make it difficult for models to cover "multiple anomaly forms / categories of a class of anomaly samples of the same product", thus affecting generalization ability and usability. Methods such as GAN often require training multiple models separately according to anomaly type / product, which results in high training costs and complex maintenance.

[0009] 3. Inaccurate mask alignment: Generative anomaly data often makes it difficult to obtain pixel-level accurate masks that are strictly aligned with the anomaly regions; mask alignment errors will significantly affect the training effect of subsequent supervised segmentation models.

[0010] 4. Insufficient robustness of supervised segmentation models: Downstream anomaly segmentation models require anomaly mask supervision. When faced with complex backgrounds such as edges, shadows, and textures in industrial scenarios, they lack robustness. The lack of accurate mask information makes it difficult to reliably identify subtle texture anomalies, which can easily lead to misjudgment of spatial features. More accurate anomaly mask information needs to be introduced to improve the ability to identify fine-grained anomalies. Summary of the Invention

[0011] This invention addresses two key technical bottlenecks in industrial anomaly detection: "how to generate samples of all different anomaly types for a class of anomalies within a unified model" and "how to ensure that the generation of locally diverse anomalies maintains the semantic distribution of real samples in other regions, i.e., maintaining global consistency and local diversity during the generation process." It proposes an intelligent anomaly detection method for industrial images. Based on a prompt-learning-based anomaly image generation framework, by introducing prompt learning and a hybrid training strategy, the learning process focuses more on anomaly features rather than global representations. This increases the diversity of anomaly images while learning more accurate anomaly information, and most importantly, the generated images still maintain reasonable realism. Secondly, to address the application dilemma of current mainstream models where "inaccurate pixel-level mask alignment severely affects subsequent training of anomaly-supervised segmentation models," a pixel-level mask matching framework is designed. Cross-attention matching loss constrains the image generation process, and a mask prediction module is introduced to provide accurate anomaly image mask pairs for each anomaly type. Finally, using the generated image-mask pairs, the final unified supervised segmentation model is trained and evaluated to improve the detection capability for subtle anomalies such as texture and edges. These three elements complement each other and progress step by step, together forming the overall technical system of the present invention that improves the generalizability, robustness, and interpretability of industrial anomaly detection.

[0012] To achieve the above objectives, the present invention provides the following technical solution:

[0013] An intelligent method for detecting anomalies in industrial images includes the following steps:

[0014] S1. Data Acquisition and Preparation: Collect a dataset containing normal images and a small number of abnormal images with pixel-level masks;

[0015] S2. Construction of the prompt learning module: Learn a specific set of exception prompt tokens for each exception type, and define normal prompt tokens;

[0016] S3. Fine-tuning training based on the pre-trained latent space diffusion model: Train a unified latent space diffusion model using the dataset from step S1; The training process includes: adopting a hybrid training strategy, including both normal and abnormal images in the training batch; By introducing cross-attention matching loss, constraining the model to align abnormal regions with abnormal cue tokens and normal regions with normal cue tokens during the generation process.

[0017] S4. Inference and Generation Stage: After training is completed, a normal image is input and an anomaly type is specified. The corresponding abnormal image is generated through the unified latent space diffusion model, and a pixel-level anomaly mask is output simultaneously to obtain an abnormal image-mask pair.

[0018] S5. Anomaly segmentation model training: Using the anomaly image-mask pairs generated in step S4 and real normal images, a unified supervised anomaly segmentation network is trained; the anomaly segmentation network adopts a spatial-frequency domain fusion structure, decomposes spatial features into low-frequency and high-frequency components through wavelet transform, and fuses spatial and frequency domain features through a cross-domain semantic alignment mechanism.

[0019] S6. Anomaly Segmentation Inference: Input the industrial product image to be detected into the trained anomaly segmentation network and output pixel-level anomaly segmentation results.

[0020] Furthermore, in step S2, learnable anomaly hints for each anomaly type. <un>The number of tokens is 4 to 6, which can be used to learn normal prompts. <na>The number of tokens is 2 to 3.

[0021] Furthermore, the training of the unified latent space diffusion model in step S3 specifically includes:

[0022] Freeze most of the parameters of the pre-trained latent space diffusion model;

[0023] Design a set of learnable exception hints for each exception type. <un>Tokens, and design a set of learnable normal prompts. <na>token, the <un>Token sum <na>The token's embedding vector is used as a conditional parameter for training;

[0024] In each training batch, normal images and abnormal images are sampled in a fixed ratio;

[0025] For abnormal image samples, a joint loss consisting of cross-attention matching loss and noise prediction loss is calculated to update the model parameters; wherein, the cross-attention matching loss constrains the response of the cross-attention map in the U-Net (U-shaped network) according to the true anomaly mask, such that abnormal regions respond to the anomaly prompt token, normal regions respond to the normal prompt token, and the response of abnormal regions to the normal prompt token is suppressed.

[0026] Furthermore, in step S3, the cross-attention matching loss is calculated based on the cross-attention graph of the U-Net in the latent space diffusion model;

[0027] For abnormal images It also includes the actual mask M, which is fused with multiple layers l of U-Net and the anomaly message token is extracted. <un>Corresponding cross-attention map And normal token prompt <na>Corresponding cross-attention map ;

[0028] Calculate the anomalous alignment loss, prompting It has a high response value in the abnormal region indicated by mask M;

[0029] Calculate the normal alignment loss, prompting It exhibits a high response value in the normal region and suppresses its response value in the abnormal region;

[0030] The cross-attention matching loss This is the sum of the abnormal alignment loss and the normal alignment loss.

[0031] Furthermore, in step S4, the synchronous output of pixel-level anomaly masks specifically includes:

[0032] During the denoising process of model inference, the features output by the U-Net decoder are fused with the high-resolution features of the VAE (Variational Autoencoder) decoder.

[0033] In the last few denoising time steps, the intermediate mask is predicted based on the fused features;

[0034] The average value of all intermediate masks is taken, and the final binarized anomaly mask is obtained by thresholding.

[0035] Furthermore, in step S5, the spatial-frequency domain fusion structure of the anomaly segmentation network specifically includes:

[0036] Frequency domain decomposition module: Decomposes the spatial feature map extracted by the segmentation network encoder into low-frequency and high-frequency components using wavelet transform;

[0037] Cross-domain alignment and fusion module: Multi-scale convolution is performed on spatial features, low-frequency components and high-frequency components to achieve scale alignment, and cross-domain semantic alignment and feature selection are achieved through cross-attention mechanism. The fused features are then input into the segmentation decoder.

[0038] Furthermore, in step S5, the wavelet transform is a Haar wavelet transform, the high-frequency component is used to enhance the detection of texture and edge anomalies, and the low-frequency component is used to preserve the overall structural information of the image.

[0039] In a second aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the preceding claims.

[0040] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. The intelligent detection method for industrial image anomalies proposed in this invention constructs a learnable prompt structure for industrial anomalies, by introducing a normal token. <na>"Exception token groups categorized by exception type" <un>"Using the U-Net cross-attention map, anomaly / normal alignment constraints are constructed to align abnormal regions to..." <un>Normal area aligned to <na>This enables fine-grained semantic expression and controllable generation of anomalies, resulting in more realistic anomalies and reducing the semantic and texture gap between synthetic and real anomalies.

[0043] 2. The intelligent detection method for industrial image anomalies proposed in this invention establishes a unified generation model and a hybrid training strategy. Only one diffusion generation backbone (unified U-Net) is trained for the same product. Through normal / abnormal mixed sampling, the same anomaly type can generate anomaly samples with different locations / shapes / sizes, thereby improving downstream coverage and diversity and reducing training costs.

[0044] 3. The intelligent detection method for industrial image anomalies proposed in this invention integrates the features of U-Net decoder and VAE decoder to output a high-resolution and boundary-fitting anomaly mask. Wavelet frequency domain features are introduced into the downstream unified segmentation network, and cross-domain alignment and feature selection are achieved through multi-scale convolution and cross-attention, thereby improving robustness and accuracy. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0046] Figure 1 The overall flowchart of the intelligent detection method for industrial image anomalies provided in the embodiments of the present invention is shown.

[0047] Figure 2 For each anomaly type of each product, a different learnable text prompt is designed for the embodiments of the present invention.

[0048] Figure 3 A block diagram for mask matching and prediction provided in an embodiment of the present invention.

[0049] Figure 4 This addresses the semantic and textural gap between synthesized images and real images using existing technologies.

[0050] Figure 5 This invention provides a method to reduce the semantic and texture gap between synthesized normal images and training images.

[0051] Figure 6 The same anomaly type provided in the embodiments of the present invention can generate anomaly samples with different locations / shapes / sizes, thereby improving downstream coverage.

[0052] Figure 7 The alignment constraint + fine mask prediction provided in the embodiments of the present invention significantly improves the boundary fit and enhances the supervised segmentation training effect. Detailed Implementation

[0053] To better understand this technical solution, the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0054] The overall process of the intelligent detection method for industrial image anomalies proposed in this invention is as follows: Figure 1 As shown, this invention achieves high-fidelity generation of multiple anomaly types within a unified diffusion generation model framework. It outputs pixel-level accurate masks through cross-attention alignment constraints and fine-grained mask prediction, and trains a unified anomaly segmentation model using the generated high-quality "anomaly image-mask pairs," thereby improving the accuracy and robustness of industrial anomaly detection in multi-type, fine-grained, and complex backgrounds. The method of this invention specifically includes the following steps:

[0055] S1. Data Acquisition and Preparation: Collect a dataset of a certain industrial product category, which includes normal images and a small number of abnormal images with pixel-level masks (divided by abnormality type).

[0056] S2. Construction of the prompt learning module: Learn a specific set of exception prompt tokens for each exception type, and define normal prompt tokens; Specifically, in step S2, the learnable exception prompts for each exception type... <un>The number of tokens is 4 to 6, which can be used to learn normal prompts. <na>The number of tokens is 2 to 3.

[0057] S3. Fine-tuning training based on the pre-trained latent space diffusion model (LDM): Using the dataset from step S1, freeze most of the pre-trained parameters and train only the conditional embedding (prompt token) and the specified sub-module to obtain a unified latent space diffusion model.

[0058] The training process includes: employing a hybrid training strategy, which includes both normal and abnormal images in the training batch; and introducing cross-attention matching loss to constrain the model to align abnormal regions with abnormal cue tokens and normal regions with normal cue tokens during the generation process.

[0059] Specifically, training the unified latent space diffusion model in step S3 includes:

[0060] Freeze most of the parameters of the pre-trained latent space diffusion model;

[0061] Design a set of learnable exception hints for each exception type. <un>Tokens, and design a set of learnable normal prompts. <na>token, the <un>Token sum <na>The token's embedding vector is used as a conditional parameter for training;

[0062] In each training batch, normal images and abnormal images are sampled in a fixed ratio;

[0063] For abnormal image samples, a joint loss consisting of cross-attention matching loss and noise prediction loss is calculated to update the model parameters; wherein, the cross-attention matching loss constrains the response of the cross-attention map in U-Net according to the true anomaly mask, such that abnormal regions respond to the anomaly prompt token, normal regions respond to the normal prompt token, and the response of abnormal regions to the normal prompt token is suppressed.

[0064] Further, in step S3, the cross-attention matching loss is calculated based on the cross-attention map of the U-Net in the latent space diffusion model; for anomalous images And its real mask M, extracting the anomaly message token on multiple layers l of U-Net. <un>Corresponding cross-attention map And normal token prompt <na>Corresponding cross-attention map ; Calculate the anomalous alignment loss, prompting High response values ​​are observed in anomalous regions indicated by mask M; the normal alignment loss is calculated to promote... The mask M indicates a high response value in the normal region and suppresses its response value in the abnormal region; the cross-attention matching loss This is the sum of the abnormal alignment loss and the normal alignment loss.

[0065] S4. Inference and Generation Stage: After training, a normal image is input and noise is added to obtain latent variables. A specified anomaly type prompt is used to guide denoising. The corresponding anomalous image is generated through the unified latent space diffusion model, and a pixel-level anomaly mask is output simultaneously to obtain anomaly image-mask pairs.

[0066] The research process of abnormal image generation based on cue learning in this invention is as follows.

[0067] Research Motivation: The WinCLIP model, using a pre-trained CLIP (Contrastive Language-Image Pre-trained) model, effectively extracts and utilizes text-guided and visual features for zero-shot and few-shot anomaly segmentation. It is the first work to explore and demonstrate the effectiveness of text-guided zero-shot anomaly classification and segmentation. However, experiments with WinCLIP revealed that typical text prompts, such as "photo of a defective bottle" or "damaged bottle," are not optimal for industrial anomaly generation. Descriptive semantic vocabulary for normal and abnormal products may not be able to capture the degree of variation between them with fine granularity. Since defects are rare and vary widely, representative anomaly samples are lacking in the training data. Therefore, it is difficult to obtain accurate and diverse anomaly descriptions. Furthermore, due to the limited number of manually labeled anomaly samples, it is difficult to directly and explicitly control the boundary between normal and anomaly prompts using contrastive loss. Therefore, this invention requires a new design for the prompt to accurately represent anomaly samples.

[0068] (1) Learnable prompt structure

[0069] The similar technical solution PrmoptAD cleverly solves this problem for the first time by utilizing prompt learning and semantic connectivity. Prompt learning aims to automatically learn a prompt through contrastive learning to guide image classification. Semantic connectivity, for example, transforms the prompt "a photo of cable" into "a photo of cable with flaw" with an anomalous meaning when concatenated with a suffix. Clearly, concatenating a text with the opposite meaning of the prompt with the original prompt can change its semantics. Based on this idea, semantic connectivity first designs a learnable normal prompt for normal samples, for example... Next, the normal message will be concatenated with many abnormal texts to generate an error message, for example... It can also be used as a negative cue for normal samples during the cue learning period.

[0070] Since manually annotated exception text is limited, in order to enrich exception information, SC also designed a learnable exception message by connecting normal prompts with a learnable token suffix. ,in It is a learnable token. The distribution of learnable anomaly prompts and manually designed anomaly prompts is aligned, thus ensuring that learnable anomaly prompts learn more correct anomaly information. However, this design is still limited by normal sample prompts. In real-world industrial scenarios, it is difficult to obtain detailed sample descriptions, and experiments have shown that generating pseudo-normal samples can deviate from the semantic distribution of actual samples.

[0071] Therefore, as Figure 2 The experiment shown designs different learnable text prompts for each anomaly type of each product, where P = <na1> <na2> <na3> <na4>with <un1> <un2> <un3> <un4>,here <na>and <un>These are tokens for normal industrial products (referred to as normal tokens) and tokens for abnormal products (referred to as abnormal tokens). A set of N abnormal tokens is used for each abnormality type, with different sets corresponding to different abnormality types, trained using normal images and... <na>The corresponding embeddings are trained using anomalous images and <un>The corresponding embedding. <un>The number of tokens should be 4-6. If the number of tokens is too small, it will be difficult to align them with multiple different anomalies belonging to the same category. During inference, if the denoised anomaly features are... <un>When the distance is large, U-Net will assign it a smaller response, resulting in an "abnormal missing" phenomenon. <na>The number of tokens is 2-3, if <na>If there are too many tokens, each token will focus on local features of the normal product, which will affect the global consistency of the normal product.

[0072] (2) Hybrid training strategy

[0073] During training, normal and abnormal images are sampled simultaneously in each batch and mixed in a fixed ratio. This allows the model to maintain global consistency with normal products while generating diverse abnormal patterns in local areas, thereby alleviating overfitting on small samples and improving the diversity of abnormalities.

[0074] (3) Generation process

[0075] The input image is encoded as a latent variable z, and then diffused and noise is added to obtain... U-Net predicts noise and performs progressive noise reduction under the guidance of text conditional vectors, ultimately decoding the latent variables into the generated image. .

[0076] In step S4, the synchronous output of pixel-level anomaly masks specifically includes:

[0077] During the denoising process of model inference, the features output by the U-Net decoder are fused with the high-resolution features of the VAE decoder;

[0078] In the last few denoising time steps, the intermediate mask is predicted based on the fused features;

[0079] The average value of all intermediate masks is taken, and the final binarized anomaly mask is obtained by thresholding.

[0080] The research process of pixel-level mask matching and fine prediction is as follows.

[0081] Research Motivation: Current mainstream anomaly detection models using generative methods suffer from poor performance. While CutPaste and DRAEM models synthesize anomalies from external datasets or noise and then merge and crop the anomaly regions into normal regions, they fail to accurately reflect the semantic features of real anomalies due to the inability to directly generate complete anomaly images, resulting in suboptimal performance in practical applications. The Generative Adversarial Network (GAN)-based DFMGAN method can generate complete anomaly images, but requires training multiple independent models to generate normal and anomaly images for each anomaly type, incurring significant training costs. Furthermore, the generated masks often fail to accurately align with the anomalies, making it difficult to obtain unified anomaly image-mask pairs. Anomaly Diffusion models, based on diffusion models, can edit anomalies on normal images under the guidance of predefined masks, but they cannot generate pseudo-normal images, and the generated anomaly samples may be affected by mask misalignment, limiting their practicality in training supervised segmentation models. Pixel-level inaccurate mask alignment severely impacts subsequent supervised training of anomaly segmentation models. Therefore, the experimental goal of this invention is to provide accurate anomaly image mask pairs for each anomaly type.

[0082] (1) Cross-attention matching loss

[0083] The cross-attention mechanism used in the U-Net architecture is a common technique in conditional generation tasks such as text-to-image generation. Its goal is to integrate external conditional information (such as text descriptions) into the image generation process, achieving precise control over the generated content, as shown in Equation (1).

[0084]

[0085] Where the condition vector Typically derived from a text encoder, image features contain semantic information. The feature representations of the intermediate layers of U-Net contain spatial information about the current generation state. To control the generation process, a conditional mechanism computes conditional vectors. Image features of U-Net inner layers Cross-attention is achieved through cross-attention; cross-attention graph ,in Indicates through linear layer from The query obtained from projection, It is the resolution of the feature maps in U-Net. It is an index of the inner layer of U-Net. Indicates through another linear layer from The key obtained by projection, This represents the number of text embedded after padding.

[0086] The mask matching task requires providing accurate anomaly image mask pairs for each anomaly type, so as... Figure 3 As shown, the experiment utilizes the cross-attention map in the U-Net architecture to design normal alignment loss and abnormal alignment loss, and incorporates a mask prediction module.

[0087] The anomaly alignment loss strictly distinguishes between normal and abnormal regions. The anomaly alignment loss is based on the mask. Align the abnormal region to <un>During training, element-wise multiplication will also be considered to reduce the cross-attention graph corresponding to normal tokens. The response value in the abnormal region prevents normal... <na>token aligned to Abnormal areas.

[0088] Therefore, as shown in formula (2), the abnormal image The total loss is:

[0089]

[0090] In the second term of the equation, random noise is used in the experiment. and time step Abnormal images Perform forward diffusion to obtain the noisy latent representation. Condition vector Used to guide U-Net in predicting noise, and then calculating with noise The loss.

[0091] Attention maps are extracted using U-Net's multi-layer cross-attention mechanism. The attention maps are then weighted by a real anomaly mask *f* to construct anomaly alignment losses and normal alignment losses, ensuring that anomaly regions are aligned to the correct values. <un>Align the normal area to " <na>"and suppress abnormal regions on " <na>The response improves the consistency between generated anomalies and masks from the source.

[0092] (2) Fine mask prediction module

[0093] Considering the low resolution of cross-attention maps, direct interpolation can lead to uncertain boundaries. This invention integrates U-Net decoder features with VAE decoder high-resolution features to output a high-resolution fine mask, achieving pixel-level boundary fitting.

[0094] (3) Time step integration

[0095] During inference, the mask is output and averaged in the last few denoising steps. Finally, the final anomaly mask is obtained by binarization through a threshold τ, and the final anomaly image is output.

[0096] S5. Anomaly Segmentation Model Training: Using the anomaly image-mask pairs generated in step S4, and real normal images, a unified supervised anomaly segmentation network is trained. The anomaly segmentation network adopts a spatial-frequency domain fusion structure, decomposes spatial features into low-frequency and high-frequency components through wavelet transform, and fuses spatial and frequency domain features through a cross-domain semantic alignment mechanism.

[0097] Specifically, in step S5, the spatial-frequency domain fusion structure of the anomaly segmentation network includes:

[0098] Frequency domain decomposition module: The spatial feature map extracted by the segmentation network encoder is decomposed into low-frequency components and high-frequency components through wavelet transform; the wavelet transform is Haar wavelet transform, the high-frequency components are used to enhance the detection of texture and edge anomalies, and the low-frequency components are used to preserve the overall structural information of the image.

[0099] Cross-domain alignment and fusion module: Multi-scale convolution is performed on spatial features, low-frequency components and high-frequency components to achieve scale alignment, and cross-domain semantic alignment and feature selection are achieved through cross-attention mechanism. The fused features are input into the segmentation decoder and the output is pixel-level anomaly segmentation results.

[0100] S6. Anomaly Segmentation Inference: Input the industrial product image to be detected into the trained anomaly segmentation network and output pixel-level anomaly segmentation results.

[0101] In a second aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the preceding claims.

[0102] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0103] Example

[0104] 1. Dataset Description:

[0105] The experiments were conducted on the MVTEC AD dataset, VisA dataset, and MVTEC 3D AD dataset, which contains only RGB images and are commonly used for industrial anomaly detection.

[0106] The MVTec AD dataset contains 5354 high-resolution images of 5 textures and 10 objects from different domains, with a total of 73 types of defects. Pixel-level accurate masks and labels are provided for each defect image (1888 images in total).

[0107] The VisA dataset covers 12 objects across 3 domains. The MVTec 3D AD dataset includes 10 product categories, with up to 4 distinct anomalies per category. It incorporates more natural influences, such as variations in lighting conditions and product pose changes.

[0108] 2. Explanation of evaluation indicators:

[0109] For image generation, referencing the evaluation metrics used by AnomalyDiffusion, our evaluation comprises three levels: anomalous images, normal images, and anomalous regions, using the following metrics:

[0110] Inception Score (IS) and Intra-cluster pairwise LPIPS Distance (IC-LPIPS) are used to evaluate the realism and diversity of anomalous images. IS is an objective performance metric that measures the mutual information (expectation of image quality and k-dispersion (KL)) between input samples and their predicted categories using a pre-trained Inception model to assess the fidelity and diversity of generated images. IC-LPIPS quantifies the perceptual similarity between image patches within the same cluster.

[0111] KID is an unsupervised image generation quality assessment metric. It evaluates the quality of generated images by calculating the distance between the generated image and the real image in the representation space. It is used to evaluate the realism of pseudo-normal images.

[0112] Two metrics were used for pixel-level anomaly segmentation and image-level anomaly detection: AU-ROC and AU-PR (these two metrics measure the model's performance on anomaly classification and segmentation tasks, with AU-PR providing a better measure of segmentation performance). All these metrics were calculated using the scikit-learn library.

[0113] 3. Experimental verification data:

[0114] The experimental results of this technical solution are shown in the table below, demonstrating its superiority:

[0115] Model IS (↑) IC-LPIPS (↑) KID (↓) Image AUROC (↑) Pixel AUROC (↑) Pixel AP (↑) DFMGAN 1.72 0.20 0.12 91.08 94.91 67.06 AnomalyDiffusion 1.80 0.32 - 95.15 96.30 69.77 This invention 1.83 0.33 0.08 95.30 97.75 75.10

[0116] Compared with the prior art, the method of the present invention has the following advantages:

[0117] (1) More realistic generated images: reducing the semantic and texture gap between synthetic and real images. The semantic and texture gap between synthetic and real images using existing technologies is as follows: Figure 4 As shown in the figure. The embodiments of the present invention provide a method for reducing the semantic and texture gap between synthesized normal images and training images, as shown in the figure. Figure 5 As shown.

[0118] (2) Higher anomaly diversity: The same anomaly type can generate anomaly samples with different locations / shapes / sizes, improving downstream coverage. For example... Figure 6 As shown.

[0119] (3) More accurate mask alignment: Alignment constraints + fine mask prediction significantly improve boundary fit and enhance supervised segmentation training results. For example Figure 7 As shown.

[0120] (4) Controllable training costs: The same product generates a unified backbone, avoiding the need to train multiple models for each anomaly type, thus reducing computation and maintenance costs.

[0121] The experiments were trained using a pre-trained Stable Diffusion v1-4 dataset with fine-tuning. In the anomalous image generation experiments, the MVTec AD dataset was primarily used. The model could use 60 normal images and one-third of the anomalous images for each anomaly type with pixel-level precise masks as the training set. A generative model was trained for each product (including textures and objects), covering all anomaly types in the industrial product training dataset. During inference, the model generated rich and accurate anomalous image-mask pairs for one anomaly type (to compare with anomaly diffusion, the experiment generated 1000 images for each anomaly type). Regarding resource requirements and time consumption, the experiments used Alibaba Cloud ModelScope: 1.31.0-pytorch2.8.0-gpu-py311-cu124-ubuntu22.04 a10 model single GPU to sequentially train each product category, which likely consumed approximately 22GB of memory. For the MVTec AD dataset, our training time was 82 hours (approximately 5.3 hours per field class), shorter than the 249 hours required by AnomalyDiffusion and the 414 hours required by DFMGAN. Regarding inference time, for all experiments, we used t=1500 to perform a forward diffusion process on normal images to obtain initial noise. We used T=25 steps for sampling, resulting in an inference time of 800 ms per image, shorter than the 3830 ms per image required by the diffusion-based AnomalyDiffusion method. The inference time for the GAN-based DFMGAN method was 48 ms per image.

[0122] (5) Better segmentation robustness: Spatial-frequency domain fusion is more sensitive to complex backgrounds such as textures, edges, and shadows, improving the stability of subtle anomaly detection and localization.

[0123] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.< / na> < / na> < / un> < / na> < / un> < / na> < / na> < / un> < / un> < / un> < / na> < / un> < / na> < / un3> < / un2> < / un1> < / na3> < / na2> < / na1> < / na> < / un> < / na> < / un> < / na> < / un> < / na> < / un> < / na> < / un> < / un> < / na> < / na> < / un> < / na> < / un> < / na> < / un> < / na> < / un>

Claims

1. An intelligent method for detecting anomalies in industrial images, characterized in that, Includes the following steps: S1. Data Acquisition and Preparation: Collect a dataset containing normal images and a small number of abnormal images with pixel-level masks; S2. Construction of the prompt learning module: Learn a specific set of exception prompt tokens for each exception type, and define normal prompt tokens; S3. Fine-tuning training based on the pre-trained latent space diffusion model: Train a unified latent space diffusion model using the dataset from step S1. The training process includes: employing a hybrid training strategy, which simultaneously includes normal and abnormal images in the training batch; and by introducing cross-attention matching loss, constraining the model to align abnormal regions with abnormal cue tokens and normal regions with normal cue tokens during the generation process. S4. Inference and Generation Stage: After training is completed, a normal image is input and an anomaly type is specified. The corresponding abnormal image is generated through the unified latent space diffusion model, and a pixel-level anomaly mask is output simultaneously to obtain an abnormal image-mask pair. S5. Anomaly segmentation model training: Using the anomaly image-mask pairs generated in step S4 and real normal images, a unified supervised anomaly segmentation network is trained; the anomaly segmentation network adopts a spatial-frequency domain fusion structure, decomposes spatial features into low-frequency and high-frequency components through wavelet transform, and fuses spatial and frequency domain features through a cross-domain semantic alignment mechanism. S6. Anomaly Segmentation Inference: Input the industrial product image to be detected into the trained anomaly segmentation network and output pixel-level anomaly segmentation results.

2. The intelligent detection method for industrial image anomalies according to claim 1, characterized in that, In step S2, learnable exception hints for each exception type <un>The number of tokens is 4 to 6, which can be used to learn normal prompts. <na> The number of tokens is 2 to 3.< / na> < / un> 3. The intelligent detection method for industrial image anomalies according to claim 1, characterized in that, The training of the unified latent space diffusion model in step S3 specifically includes: Freeze most of the parameters of the pre-trained latent space diffusion model; Design a set of learnable exception hints for each exception type. <un>Tokens, and design a set of learnable normal hints. <na>token, the <un>Token sum <na> The token's embedding vector is used as a conditional parameter for training;< / na> < / un> < / na> < / un> In each training batch, normal images and abnormal images are sampled in a fixed ratio; For abnormal image samples, a joint loss consisting of cross-attention matching loss and noise prediction loss is calculated to update the model parameters; wherein, the cross-attention matching loss constrains the response of the cross-attention map in U-Net according to the true anomaly mask, such that abnormal regions respond to the anomaly prompt token, normal regions respond to the normal prompt token, and the response of abnormal regions to the normal prompt token is suppressed.

4. The intelligent detection method for industrial image anomalies according to claim 1, characterized in that, In step S3, the cross-attention matching loss is calculated based on the cross-attention graph of the U-Net in the latent space diffusion model; For abnormal images It also includes the actual mask M, which is fused with multiple layers l of U-Net and the anomaly message token is extracted. <un>Corresponding cross-attention map And normal token prompt <na>Corresponding cross-attention map ;< / na> < / un> Calculate the anomalous alignment loss, prompting It has a high response value in the abnormal region indicated by mask M; Calculate the normal alignment loss, prompting It exhibits a high response value in the normal region and suppresses its response value in the abnormal region; The cross-attention matching loss This is the sum of the abnormal alignment loss and the normal alignment loss.

5. The intelligent detection method for industrial image anomalies according to claim 1, characterized in that, In step S4, the synchronous output of pixel-level anomaly masks specifically includes: During the denoising process of model inference, the features output by the U-Net decoder are fused with the high-resolution features of the VAE decoder; In the last few denoising time steps, the intermediate mask is predicted based on the fused features; The average value of all intermediate masks is taken, and the final binarized anomaly mask is obtained by thresholding.

6. The intelligent detection method for industrial image anomalies according to claim 1, characterized in that, In step S5, the spatial-frequency domain fusion structure of the anomaly segmentation network specifically includes: Frequency domain decomposition module: Decomposes the spatial feature map extracted by the segmentation network encoder into low-frequency and high-frequency components using wavelet transform; Cross-domain alignment and fusion module: Multi-scale convolution is performed on spatial features, low-frequency components and high-frequency components to achieve scale alignment, and cross-domain semantic alignment and feature selection are achieved through cross-attention mechanism. The fused features are then input into the segmentation decoder.

7. The intelligent detection method for industrial image anomalies according to claim 1, characterized in that, In step S5, the wavelet transform is a Haar wavelet transform, the high-frequency components are used to enhance the detection of texture and edge anomalies, and the low-frequency components are used to preserve the overall structural information of the image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images