A memory bank-based multi-class industrial anomaly detection and localization system

Abstract

The application discloses a kind of based on memory pool multi-class industrial anomaly detection and positioning system, including dual-module network architecture, respectively unified memory pool feature processing network and prototype enhanced multi-scale positioning network;Unified memory pool feature processing network contains feature extraction unit, conditional position embedding unit, flip test unit, double memory pool storage unit and near distribution anomaly enhancement unit;Prototype enhanced multi-scale positioning network integrates feature adaptation layer, cross-scale attention interaction unit and prototype contrast learning unit, realize feature denoising and soft alignment.The application discards the deployment mode of traditional one-class one model, adopts single model to complete multi-class industrial image anomaly detection, can accurately identify logical anomaly and near distribution micro distortion, defect positioning edge is clear, detection precision and inference efficiency are excellent, suitable for the quality inspection scene of multi-product flexible industrial production line.

Description

Technical Field

[0001] This invention relates to the fields of industrial machine vision, deep learning and unsupervised anomaly detection technology, specifically a multi-category industrial anomaly detection and localization system based on a memory bank. Background Technology

[0002] With the rapid development of intelligent manufacturing and digital production lines, surface defect detection of industrial products has become a core aspect of quality control in flexible production lines. Traditional manual inspection is inefficient and susceptible to subjective factors, and traditional image processing-based detection algorithms are ill-suited for complex textures and diverse industrial scenarios. Currently, deep learning-based memory-based anomaly detection methods are widely used. These methods rely on features from normal samples for modeling, eliminating the need for a large number of defect samples and demonstrating strong practicality.

[0003] Existing memory-based industrial anomaly detection technologies have several shortcomings: First, they generally adopt a one-model-per-class deployment paradigm. As the number of product categories increases, the number of models, storage overhead, and maintenance costs rise significantly, and cross-category knowledge transfer cannot be achieved. Second, most models can only identify common texture anomalies such as scratches and color differences, lacking spatial awareness and unable to identify logical anomalies such as missing components or component offsets. Third, industrial scenarios contain a large number of closely distributed micro-distortions whose features are highly similar to normal samples, and existing models have blurred discrimination boundaries and insufficient identification capabilities. Fourth, cross-scale feature fusion often uses linear interpolation and hard alignment, which easily causes the defect localization edges to become diffuse, resulting in poor pixel-level localization accuracy. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing a multi-category industrial anomaly detection and location system based on a memory bank.

[0005] The present invention adopts the following technical solution:

[0006] A multi-category industrial anomaly detection and localization system based on a memory bank includes a unified memory bank feature processing network and a prototype-enhanced multi-scale localization network; the unified memory bank feature processing network includes a feature extraction unit, a flip test unit, a conditional location embedding unit, a dual memory bank storage unit, and a near-distribution anomaly enhancement unit.

[0007] The feature extraction unit uses the WideResNet backbone network to extract the second and third layer features of the image as block-level features and the fourth layer features as global features. After unifying the size of features at different scales, they are aggregated into a standardized feature vector.

[0008] The flip test unit is used to calculate the position scores of the original image and the transposed image to distinguish between textured images and non-textured images; the conditional position embedding unit generates a dual-channel coordinate tensor based on the flip test results and fuses it into the block-level features;

[0009] The dual-memory storage unit includes a global feature memory and a block-level feature memory, and uses greedy kernel set sampling to complete feature compression and storage.

[0010] The near-distribution anomaly enhancement unit combines sinusoidal curve distortion and Perlin noise to generate near-distribution anomaly samples and constructs a discriminative focusing loss function;

[0011] Multi-scale feature extraction is performed by inputting multi-category industrial images into the feature extraction unit. Spatial coordinate information is adaptively added by the conditional location embedding unit combined with the flip test. The feature storage and cross-category retrieval are then completed by the dual memory storage unit. Finally, the near distribution anomaly enhancement unit synthesizes samples and combines them with the loss function to optimize the model, thereby improving the near distribution anomaly identification capability.

[0012] The aforementioned multi-category industrial anomaly detection and localization system based on a memory bank uses WideResNet as its base network for feature extraction. It achieves multi-scale feature size unification through bilinear interpolation and then uses adaptive average pooling to achieve feature aggregation.

[0013] The aforementioned multi-category industrial anomaly detection and localization system based on a memory bank, in its flip test unit and conditional position embedding unit: firstly, the position scores of the original image and the transposed image are calculated through a flip test, and a threshold T is set to filter image categories; when the image position score is lower than T, it is determined to be a non-texture product image, and conditional position embedding is enabled, converting the image's horizontal and vertical coordinates into a dual-channel coordinate tensor, expanding it to the feature channel dimension, and then setting a position weight α to fuse it with block-level features; when the image position score is higher than T, it is determined to be a texture image, and no position embedding is added.

[0014] The aforementioned multi-category industrial anomaly detection and localization system based on a memory bank employs a greedy core set sampling algorithm in its dual memory bank storage unit, with a sampling rate set to 10%, storing features in the form of category-feature key-value pairs. During retrieval, the system prioritizes matching categories through global features and then retrieves the corresponding block-level features to complete local comparison.

[0015] The aforementioned multi-category industrial anomaly detection and localization system based on a memory bank, wherein the sample synthesis method of the near-distribution anomaly enhancement unit is as follows: the original image is geometrically distorted using a sine curve, the pixel difference is calculated to generate a basic mask, a random mask is generated by combining Perlin noise, and the near-distribution anomaly samples are obtained by pixel fusion after the two masks are superimposed.

[0016] The aforementioned multi-category industrial anomaly detection and localization system based on a memory bank includes a prototype-enhanced multi-scale localization network comprising a pre-trained network, a feature adaptation unit, a cross-scale attention interaction unit, a prototype construction unit, and a prototype contrastive learning unit.

[0017] The feature adaptation unit consists of a lightweight structure composed of convolutional layers, batch normalization layers, and ReLU activation layers; the cross-scale attention interaction unit uses high-level features as queries and low-level features as keys, and completes the fusion using scaled dot product attention, while adding residual constraints.

[0018] The prototype building unit uses a clustering algorithm to condense discrete features into cluster centroids and build a prototype memory library.

[0019] The prototype contrastive learning unit combines prototype contrastive loss and discriminative focusing loss to form a joint loss function;

[0020] Industrial images are input into a pre-trained network to extract texture features at layer 2, local features at layer 3, and global features at layer 4. The multi-scale features are then input into the corresponding feature adaptation units to complete reprojection and redundancy filtering. Feature soft alignment is achieved through a cross-scale attention interaction unit. Then, a category feature prototype is generated through a prototype construction unit. Finally, the feature distribution is constrained by the prototype contrast learning unit to complete anomaly detection and pixel localization.

[0021] The aforementioned multi-category industrial anomaly detection and localization system based on a memory bank includes a feature adaptation unit that performs reprojection mapping on the original pre-trained features, filters out task-irrelevant redundancies such as color and background, and maps the features to an industrial detection-specific subspace.

[0022] The aforementioned multi-class industrial anomaly detection and localization system based on a memory bank, in its cross-scale attention interaction unit: high-level feature Q is used as the query, mid-level features are upsampled and then concatenated with low-level features to obtain K and V, and attention weights are... d is the feature dimension, and the final aligned feature is obtained by splicing the residual features after fusion.

[0023] The aforementioned multi-category industrial anomaly detection and localization system based on a memory bank includes a prototype construction unit that performs K-means clustering on each category of normal features and selects the cluster centroids as category prototypes. The prototype comparison learning unit uses exponential moving average to update the dynamic proxy centers, with a momentum coefficient set to 0.8. The joint loss function expression is as follows: ,in For prototype comparison loss, To discriminate and focus on loss, , This is the loss weighting coefficient.

[0024] After the model is trained, image-level anomaly detection and pixel-level defect localization are achieved through feature retrieval and distance calculation.

[0025] Beneficial effects

[0026] 1. This invention adopts a unified architecture of single model and multiple categories, abandoning the traditional one-class-one-model mode, which greatly reduces the cost of model training, storage and later operation and maintenance, and is perfectly adapted to flexible industrial production lines with multiple categories.

[0027] 2. Introducing a combination mechanism of flip test and conditional position embedding, which adaptively distinguishes between textured and non-textured images and injects spatial coordinates, giving the model spatial topology verification capability, and can accurately identify logical anomalies such as missing components and positional offsets.

[0028] 3. A near-distribution anomaly synthesis scheme of sinusoidal curve distortion + Perlin noise is adopted, combined with discriminative focusing loss, to enhance the model's ability to identify small distortions and construct a robust feature discrimination boundary.

[0029] 4. A lightweight feature adaptation layer is added to filter out irrelevant feature redundancy in the pre-trained network; cross-scale attention interaction is adopted to achieve soft feature alignment, which completely solves the problem of localization edge diffusion caused by traditional interpolation and improves the accuracy of defect edge characterization.

[0030] 5. Cluster discrete features into prototype storage, and combine prototype comparative learning to generate a hypersphere contraction effect, compressing the normal feature distribution. While improving detection sensitivity, the size of the memory bank is reduced, ensuring the real-time performance of model inference. Attached Figure Description

[0031] Figure 1 is a schematic diagram of the overall network architecture and data flow of the present invention;

[0032] Figure 2 is a flowchart of the synthesis process for near-distribution anomaly samples;

[0033] Figure 3 is a schematic diagram of the overall structure of the prototype augmentation network;

[0034] Figure 4 is a flowchart of the cross-scale attention interaction unit operation;

[0035] Figure 5 shows a performance comparison between the present invention and existing algorithms on the MVTec AD dataset;

[0036] Figure 6 shows the ROC curves of the present invention and existing algorithms on the VisA dataset. Detailed Implementation

[0037] The present invention will be described in detail below with reference to specific embodiments.

[0038] Data Collection and Preprocessing

[0039] A1 collects surface images of various types of industrial products. The dataset uses the internationally available industrial anomaly datasets MVTecAD and VisA, and is supplemented with images of parts, sheets, and fabrics collected on-site from industrial production lines to enrich the sample diversity.

[0040] A2 performs uniform cropping on all original images, resizing them to 288×288 pixels, and then performs routine image preprocessing operations such as grayscale normalization and noise reduction.

[0041] A3 follows the conventional dataset partitioning rules, dividing all samples into training, validation, and test sets. The training set retains only defect-free normal images, while the test set includes normal samples, various texture defects, logical anomalies, and samples with near-distributed minor distortions, thus meeting the experimental requirements for unsupervised anomaly detection.

[0042] B. Training a unified memory feature processing network

[0043] This invention first trains a unified memory feature processing network, which includes a feature extraction unit, a flip test unit, a conditional location embedding unit, a dual memory storage unit, and a near-distribution anomaly enhancement unit. The overall process corresponds to... Figure 1 .

[0044] B1 Feature Extraction Unit

[0045] The preprocessed industrial image input to B11 enters the feature extraction unit, which uses a pre-trained WideResNet101 as the backbone network.

[0046] B12 extracts the output features of the second and third layers of the network as block-level features and extracts the output features of the fourth layer as global features. It uses bilinear interpolation to unify the spatial size of multi-scale features and then uses adaptive average pooling to aggregate features and generate standardized feature vectors.

[0047] B2 Conditional Position Embedding and Flipping Test

[0048] B21 calculates the position scores of the original image and the transposed image, and sets the judgment threshold T=30. When the image position score is below 30, it is judged as a non-texture product image, and conditional position embedding is enabled; when the score is above 30, it is judged as a texture image, and position embedding is not added.

[0049] B22 converts the horizontal and vertical coordinates of the image into a two-channel coordinate tensor, and expands the coordinate tensor to the same channel dimension as the block-level feature to obtain the position feature; sets the position weight to 0.17, and adds the position feature to the block-level feature element by element to supplement the block-level feature with spatial topological information.

[0050] B3 Dual-Memory Storage Unit Construction

[0051] B31 employs a greedy core set sampling algorithm to reduce the dimensionality of the aggregated features, with a sampling rate of 10%. It constructs a global feature memory and a block-level feature memory, both of which store feature data in the form of "category-feature" key-value pairs.

[0052] During B32 feature retrieval, global features are first used to match the category to which the image belongs, and then block-level features of the corresponding category are retrieved to achieve fast cross-category retrieval and local feature comparison.

[0053] B4 Training of Network for Synthesis of Nearly Distributed Outlier Samples and Feature Processing Based on Unified Memory

[0054] As shown in Figure 2, using the normal image I as input, a geometrically deformed image N is obtained by twisting the image using a sine curve. The difference between the normal image I and the geometrically deformed image N is then filtered to obtain the basic mask M representing the deformed region. o Then, the connected regions in the basic mask are correlated to obtain the mask M. I On the other hand, a Perlin noise mask M is generated based on the normal image I. u Mask M u With mask M I Multiplying each element together yields the final anomaly mask M.

[0055] B42 performs region fusion on the normal image I and the geometrically deformed image N based on the final anomaly mask M, according to I... t =(1-M)⊙I+(1-β)(M⊙I)+β(M⊙N) Generates near-distributed outlier samples I t Where 1-M represents the complement of mask M, ⊙ represents element-wise multiplication, β represents the fusion coefficient of the anomalous region, and the final anomalous mask M is fused with the near-distributed anomalous sample I. t correspond.

[0056] Step B43 trains the Unified Memory Feature Processing Network. Normal samples and the near-distributed anomalous samples generated in Step B42, along with their corresponding anomalous masks, are used for network training. A discriminative focusing loss function is employed for constraint. The training epochs are set to 100, the optimizer is Adam, and the initial learning rate is set to 1e-4. The training of the Unified Memory Feature Processing Network is completed through parameter iteration.

[0057] C prototype augmented multi-scale localization network training

[0058] After the unified memory feature processing network is trained, the prototype-enhanced multi-scale localization network is trained. As shown in Figure 3, the processing flow of the prototype-enhanced multi-scale localization network includes a pre-trained network, a feature adaptation unit, a cross-scale attention interaction unit, a prototype construction unit, and a prototype contrastive learning unit. Specifically, the pre-trained network outputs layer 2 texture features, layer 3 local features, and layer 4 global features; the feature adaptation unit performs reprojection and denoising; the cross-scale attention interaction unit completes multi-scale feature fusion; the prototype construction unit forms the prototype memory; and the prototype contrastive learning unit jointly optimizes the network. The overall architecture and computational flow correspond to Figure 3.

[0059] C1 Feature Adaptation

[0060] C11 inputs the multi-scale features output from layers 2, 3, and 4 of the pre-trained network into the corresponding feature adaptation units; the feature adaptation units consist of a lightweight structure composed of convolutional layers, batch normalization layers, and ReLU activation layers.

[0061] C12 uses self-supervised reprojection learning to filter out task-irrelevant redundant information such as color and background in ImageNet pre-trained features, and maps the features to a feature subspace dedicated to industrial anomaly detection.

[0062] C2 Cross-Scale Attention Interaction

[0063] As shown in Figure 3, C21 performs an upsampling operation on the mid-level features and then concatenates them with the bottom-level features. The high-level global features are used as the query vector, and the concatenated bottom-level features are used as the key-value vector. Feature fusion is completed through a scaling dot product attention mechanism.

[0064] C22 also adds residual constraints to preserve the original topological structure of the image, solving the problems of pixel misalignment and edge blurring caused by traditional linear interpolation.

[0065] C3 Feature Prototype Construction

[0066] C31 summarizes the processed features of all normal samples in the training set, uses the K-means clustering algorithm to cluster each type of feature separately, selects the cluster centroid as the prototype of the category feature, and uses the prototype to replace the massive discrete features to build a lightweight prototype memory.

[0067] C4 Prototype Comparison Joint Training

[0068] C41 initializes the dynamic agent center and uses an exponential moving average algorithm to update the agent center in real time, with the momentum coefficient set to 0.8.

[0069] C42 Joint Prototype Contrast Loss Discriminative Focused Loss Construct the total loss function:

[0070]

[0071] Among them, the weighting coefficient , By using a joint loss constraint model, normal features are induced to shrink toward the prototype center, forming a hypersphere shrinkage effect, which tightens the distribution range of normal features and widens the discrimination distance between normal features and near-distributed abnormal features, thus completing the prototype enhancement network training.

[0072] D Industrial Image Anomaly Detection and Localization Inference

[0073] After completing the training of both networks, fixed network parameters are loaded, and anomaly detection and pixel localization are performed on the industrial image to be tested.

[0074] D1 performs the same cropping and normalization preprocessing operations as in the training phase on the image to be tested, and then proceeds through feature extraction, location embedding, feature adaptation, and cross-scale attention fusion to generate the final features of the image to be tested.

[0075] D2 inputs the features to be tested into the prototype memory, traverses all prototypes in the memory and calculates the Euclidean distance, and takes the maximum value of all distances as the image-level anomaly score, thereby determining whether there are defects in the overall image.

[0076] D3 generates a distance matrix based on the distance between pixel-by-pixel features and the prototype, obtaining the original anomaly heatmap; a Gaussian function is used to smooth the heatmap, correcting noise interference, and finally outputting pixel-level defect localization results, completing the anomaly detection and localization of the entire image.

[0077] E. Experimental Testing and Result Analysis

[0078] To verify the detection performance of this invention, two mainstream multi-class industrial anomaly detection algorithms, PatchCore and UniAD, were selected as control methods, and the MVTec AD dataset was used for comparative experiments.

[0079] Figure 5 shows a performance comparison between the present invention and two other methods, with Method 1 and Method 2 being the comparison algorithms mentioned above. It can be seen that the present invention performs best in Image-AUROC, Pixel-AUROC, and AUPRO metrics, achieving higher overall detection accuracy.

[0080] Figure 6 shows the ROC curves of the present invention and two other methods on the VisA dataset. Method 1 and Method 2 are described as in Figure 5. It can be seen that the AUC value of the present invention on the VisA dataset reaches 0.989. Compared with the other two methods, the overall accuracy, generalization ability and robustness of the system are stronger.

[0081] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A multi-category industrial anomaly detection and location system based on a memory bank, characterized in that, It includes a unified memory feature processing network and a prototype-enhanced multi-scale localization network; the unified memory feature processing network includes a feature extraction unit, a flip test unit, a conditional location embedding unit, a dual memory storage unit, and a near-distribution anomaly enhancement unit. The feature extraction unit uses the WideResNet backbone network to extract the second and third layer features of the image as block-level features and the fourth layer features as global features. After unifying the size of features at different scales, they are aggregated into a standardized feature vector. The flip test unit is used to calculate the position scores of the original image and the transposed image to distinguish between textured images and non-textured images; the conditional position embedding unit generates a dual-channel coordinate tensor based on the flip test results and fuses it into the block-level features; The dual-memory storage unit includes a global feature memory and a block-level feature memory, and uses greedy kernel set sampling to complete feature compression and storage. The near-distribution anomaly enhancement unit combines sinusoidal curve distortion and Perlin noise to generate near-distribution anomaly samples and constructs a discriminative focusing loss function; Multi-scale feature extraction is performed by inputting multi-category industrial images into the feature extraction unit. Spatial coordinate information is adaptively added by the conditional location embedding unit combined with the flip test. The feature storage and cross-category retrieval are then completed by the dual memory storage unit. Finally, the near distribution anomaly enhancement unit synthesizes samples and combines them with the loss function to optimize the model, thereby improving the near distribution anomaly identification capability.

2. The multi-category industrial anomaly detection and location system based on a memory bank according to claim 1, characterized in that, The feature extraction unit uses WideResNet as its base network, achieves multi-scale feature size unification through bilinear interpolation, and then uses adaptive average pooling to achieve feature aggregation.

3. The multi-category industrial anomaly detection and location system based on a memory bank according to claim 1, characterized in that, In the flip test unit and the conditional position embedding unit: firstly, the position scores of the original image and the transposed image are calculated through a flip test, and a threshold T is set to filter the image category; when the image position score is lower than T, it is determined to be a non-texture product image, and conditional position embedding is enabled. The horizontal and vertical coordinates of the image are converted into dual-channel coordinate tensors, expanded to the feature channel dimension, and then the position weight α is set to fuse with the block-level features; when the image position score is higher than T, it is determined to be a texture image, and no position embedding is added.

4. The multi-category industrial anomaly detection and location system based on a memory bank according to claim 1, characterized in that, The dual-memory storage unit adopts a greedy core set sampling algorithm with a sampling rate of 10%, and stores features in the form of category-feature key-value pairs. During retrieval, the category is matched first through global features, and then the corresponding block-level features are retrieved to complete the local comparison.

5. The multi-category industrial anomaly detection and location system based on a memory bank according to claim 1, characterized in that, The sample synthesis method of the near-distribution anomaly enhancement unit is as follows: the original image is geometrically distorted using a sine curve, the pixel difference is calculated to generate a basic mask, and a random mask is generated by combining Perlin noise. After the two masks are superimposed, the near-distribution anomaly samples are obtained by pixel fusion.

6. The multi-category industrial anomaly detection and location system based on a memory bank according to claim 1, characterized in that, The prototype-enhanced multi-scale localization network includes a pre-trained network, a feature adaptation unit, a cross-scale attention interaction unit, a prototype construction unit, and a prototype contrastive learning unit. The feature adaptation unit consists of a lightweight structure composed of convolutional layers, batch normalization layers, and ReLU activation layers; the cross-scale attention interaction unit uses high-level features as queries and low-level features as keys, and completes the fusion using scaled dot product attention, while adding residual constraints. The prototype building unit uses a clustering algorithm to condense discrete features into cluster centroids and build a prototype memory library. The prototype contrastive learning unit combines prototype contrastive loss and discriminative focusing loss to form a joint loss function; Industrial images are input into a pre-trained network to extract texture features at layer 2, local features at layer 3, and global features at layer 4. These multi-scale features are then input into corresponding feature adaptation units to perform reprojection and redundancy filtering. Feature soft alignment is achieved through a cross-scale attention interaction unit. Then, a prototype construction unit generates category feature prototypes. Finally, the feature distribution is constrained by a prototype contrast learning unit to complete anomaly detection and pixel localization.

7. The multi-category industrial anomaly detection and localization system based on a memory bank according to claim 6, wherein the feature adaptation unit performs reprojection mapping on the original pre-trained features, filters out task-irrelevant redundancies such as color and background, and maps the features to an industrial detection-specific subspace.

8. The multi-category industrial anomaly detection and localization system based on a memory bank according to claim 6, wherein the cross-scale attention interaction unit: uses high-level feature Q as the query, upsamples mid-level features and concatenates them with low-level features to obtain K and V, and the attention weights... d is the feature dimension, and the final aligned feature is obtained by splicing the residual features after fusion.

9. The multi-category industrial anomaly detection and location system based on a memory bank according to claim 6, characterized in that, The prototype construction unit performs K-means clustering on each class of normal features and selects the cluster centroids as the class prototypes; the prototype comparison learning unit updates the dynamic proxy centers using exponential moving average, with the momentum coefficient set to 0.8, and the joint loss function expression is as follows: ,in For prototype comparison loss, To discriminate and focus on loss, , This is the loss weighting coefficient.

Images