An unsupervised anomaly detection method and system based on memory expert guidance

Abstract

The present invention provides an unsupervised anomaly detection method and system based on memory expert guidance, which relates to the field of anomaly detection technology. The method comprises: obtaining a defect image to be detected; inputting the defect image to be detected into a trained defect detection model for detection, and obtaining a glass container surface defect detection result; the defect detection model comprises a feature distillation network for extracting a significant difference feature map and an anomaly refinement network for generating a defect detection result according to the significant difference feature map, which are connected in sequence; the feature distillation network helps a denoising student network learn normal samples based on a normal memory expert, and the normal memory expert stores a memory vector obtained by a teacher network according to normal sample features, and the denoising student network updates the query feature generated according to the defect image to be detected according to the memory vector. The present invention can improve the accuracy of glass container surface defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of anomaly detection, and in particular to an unsupervised anomaly detection method and system based on memory expert guidance. Background Art

[0002] In intelligent manufacturing, anomaly detection, as one of the key technologies, can help enterprises monitor the production process in real time, detect and deal with potential problems in a timely manner, and thus ensure product quality and production safety. Anomaly detection is the process of identifying abnormal or rare objects, events, and patterns in images or videos. Anomalies can be any situation that does not conform to the expected pattern or deviates significantly from the majority of data. In industrial defect detection, unsupervised anomaly detection methods have received widespread attention due to the high cost of data collection and annotation and the unpredictability of industrial product anomaly types.

[0003] Anomaly detection is particularly important in the production process of glass containers. During the production process of glass containers, tiny defects such as bubbles, cracks or uneven coatings may affect product quality or even cause product scrapping. Since these defects are of various types and difficult to fully label, unsupervised anomaly detection technology has become the key to glass container defect detection.

[0004] In the unsupervised anomaly detection task, knowledge distillation has become an important technology. The anomaly detection framework based on knowledge distillation can effectively utilize the advantages of the teacher network, guide the student network to learn the representation of normal patterns, and detect anomalies by identifying the differences between the teacher network and the student network. However, in this process, the following problems often occur:

[0005] (1) Insufficient refinement: When knowledge is transferred from the teacher network to the student network, it may be incompletely refined due to insufficient model capacity or training, which affects the student network’s learning of defect features.

[0006] (2) Insufficient feature extraction: The difference in feature extraction capabilities between the teacher network and the student network may cause the student network to be unable to fully learn abnormal features, especially when dealing with subtle defects.

[0007] (3) Weak teacher constraints: The guidance constraints of the teacher network on the student network may not be sufficient, resulting in poor performance of the student network in detecting glass container defects. Summary of the invention

[0008] In order to solve at least one of the deficiencies of the prior art, the object of the present invention is to provide an unsupervised anomaly detection method and system based on memory expert guidance to improve the accuracy of glass container surface defect detection.

[0009] To achieve the above object, according to some embodiments, a first aspect of the present invention provides an unsupervised anomaly detection method based on memory expert guidance, comprising:

[0010] Acquire an image of a defect to be detected;

[0011] The defect image to be detected is input into the trained defect detection model for detection to obtain the surface defect detection result of the glass container;

[0012] Among them, the defect detection model includes a feature distillation network for extracting a significant difference feature map and an anomaly refinement network for generating a defect detection result based on the significant difference feature map, which are connected in sequence. The significant difference feature map is obtained based on the denoising student network and the teacher network of the feature distillation network; the feature distillation network helps the denoising student network to learn normal samples based on the normal memory expert, and the normal memory expert stores the memory vector obtained by the teacher network according to the normal sample characteristics. The denoising student network updates the query features generated according to the defect image to be detected according to the memory vector.

[0013] A second aspect of the present invention provides an unsupervised anomaly detection system based on memory expert guidance, comprising:

[0014] An image acquisition module is configured to acquire an image of a defect to be detected;

[0015] A defect detection module is configured to input a defect image to be detected into a trained defect detection model for detection, and obtain a defect detection result of the glass container surface;

[0016] Among them, the defect detection model includes a feature distillation network for extracting a significant difference feature map and an anomaly refinement network for generating a defect detection result based on the significant difference feature map, which are connected in sequence. The significant difference feature map is obtained based on the denoising student network and the teacher network of the feature distillation network; the feature distillation network helps the denoising student network to learn normal samples based on the normal memory expert, and the normal memory expert stores the memory vector obtained by the teacher network according to the normal sample characteristics. The denoising student network updates the query features generated according to the defect image to be detected according to the memory vector.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] The present invention provides an unsupervised anomaly detection method and system based on memory expert guidance. The memory expert mechanism is introduced in the knowledge distillation process. The high-level normal features in the normal sample (i.e., normal sample) are extracted through the teacher network, and the high-level normal features are stored in the normal memory expert in the form of memory vectors and passed to the denoising student network. The denoising student network updates the query features according to the memory vectors stored in the normal memory expert, thereby accelerating the image compression process, enhancing the degree of knowledge distillation, and improving the accuracy of the denoising student network in identifying small defects. An adaptive perception state enhancement module is introduced into the denoising student network to fully extract global information and local information in the reconstruction process, so that the denoising student network can better focus on the accurate restoration of the surface details and structure of the glass container during denoising, thereby improving the reconstruction quality of the normal sample. A dual-domain comparison method is adopted to introduce frequency domain information on the basis of the spatial domain, thereby enhancing the constraints of the teacher network on the denoising student network, so that the teacher network can guide the denoising student network to reconstruct a higher quality normal image, thereby further improving the accuracy of the surface defect detection of the glass container.

[0019] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0021] Figure 1 is a flow chart of the method in Embodiment 1 of the present invention;

[0022] Figure 2 It is a schematic diagram of the overall architecture of the defect detection model;

[0023] Figure 3 It is a schematic diagram of the normal information storage and recall network;

[0024] Figure 4 Schematic diagram of the adaptive perception state enhancement module;

[0025] Figure 5 Schematic diagram of a lossless frequency domain feature encoder;

[0026] Figure 6 Schematic diagram of the abnormal refinement network. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] Embodiment 1

[0029] Embodiment 1 of the present invention provides an unsupervised anomaly detection method based on memory expert guidance, such as Figure 1-Figure 6 As shown, including:

[0030] Acquire an image of a defect to be detected;

[0031] The defect image to be detected is input into the trained defect detection model for detection to obtain the surface defect detection result of the glass container;

[0032] Among them, the defect detection model includes a feature distillation network for extracting a significant difference feature map and an anomaly refinement network for generating a defect detection result based on the significant difference feature map, which are connected in sequence. The significant difference feature map is obtained based on the denoising student network and the teacher network of the feature distillation network; the feature distillation network helps the denoising student network to learn normal samples based on the normal memory expert, and the normal memory expert stores the memory vector obtained by the teacher network according to the normal sample characteristics. The denoising student network updates the query features generated according to the defect image to be detected according to the memory vector.

[0033] In view of the problems existing in the existing unsupervised anomaly detection based on knowledge distillation, this embodiment constructs a defect detection model, and introduces a memory expert mechanism into the defect detection model to help the student network identify tiny defects in glass containers more accurately. First, a defect detection model consisting of a feature distillation network and anomaly refinement network is constructed, and then the model is trained and tested using the collected surface defect images of defective and non-defective industrial products. Finally, the trained model is deployed in an industrial application scenario to perform anomaly detection on glass containers in the production process.

[0034] The specific implementation process includes:

[0035] Step S1, collecting surface defect images of industrial products for training and verifying the model, and preprocessing the collected images to obtain a training set and a verification set;

[0036] Step S2: construct a feature distillation network;

[0037] Step S3, constructing an abnormal refinement network;

[0038] Step S4, connecting the feature distillation network and the anomaly refinement network to obtain a defect detection model, using the training set obtained in step S1 for training, and using the test set for testing to obtain a trained defect detection model;

[0039] Step S5: deploy the trained defect detection model.

[0040] In step S1, a systematic method is used to screen and divide the surface defect images of industrial products. Defect-free images are selected as training sets to cultivate the ability of the anomaly detection model to accurately recognize normal surface features. At the same time, defective images are used as test sets to evaluate the performance and accuracy of the model in actual detection tasks. In order to further improve the generalization ability and sensitivity to anomalies of the model, pseudo-anomalies are constructed for normal samples in the training set to generate pseudo-anomaly images corresponding to normal images, and mask images are created for these pseudo-anomaly images. These mask images not only define the abnormal areas, but also provide additional training information for the model, thus forming a pseudo-anomaly training set.

[0041] The generation of pseudo abnormal images aims to generate abnormal images in a more complex and realistic way. Specifically, in order to increase the diversity of generated abnormal images, multiple stages of nonlinear processing and noise transformation are introduced. a The process formula is as follows:

[0042] ;

[0043] in, represents element-by-element multiplication; T is a training set image without abnormalities, which retains the information of the original scene; A is a dataset for adding noise, which contains various abnormal shapes and textures; is an opacity factor randomly selected in the range (0,1) as a means of data enhancement; M is an anomaly mask generated by multi-level nonlinear transformation and activation function processing, representing the abnormal area; and are additional noise terms introduced, representing local and global complex noises, thereby increasing the diversity of abnormal images. The characteristics of the noise are determined by the parameters and Make adjustments.

[0044] Abnormal image P a In the generation, (1-M) is used to represent the non-abnormal area, which can make the distinction between abnormal and non-abnormal areas more direct and obvious, especially at the boundary of the abnormal area; the additional noise term introduced includes local and global complex noise, which can more intuitively control the characteristics of the noise; by introducing the opacity factor , the visibility of anomalies can be flexibly adjusted when generating abnormal images, which increases the flexibility of generating images and helps train models to identify abnormalities of different degrees.

[0045] The generation process of the anomaly mask M is as follows:

[0046] ;

[0047] Where * represents convolution operation; W i is the convolution kernel; X i is the input image; It is the nonlinear activation function RELU; is the Sigmoid activation function used to generate the mask; N is the number of convolutional layers, which determines the complexity of generating the anomaly mask; parameter W i and They represent the convolution kernel and activation function at each stage, respectively, and are used to adjust the transformation strength.

[0048] In step S2, the feature distillation network includes a denoising student network and a teacher network. A normal information storage and recall network and an adaptive perception state enhancement module are designed in the denoising student network, and the parameters of the feature distillation network are updated by a dual-domain comparison method.

[0049] Specifically, if Figure 2 As shown in Figure 1, the teacher network consists of three pre-trained Resnet18 blocks of corresponding scales, which are used to extract the features of normal samples. The denoising student network is an encoder-decoder architecture. The encoder contains four Resnet18 blocks and three normal information storage and recall networks. Each of the first three Resnet18 blocks is connected to a normal information storage and recall network; the decoder contains two Resnet18 blocks and two adaptive perception state enhancement modules connected after the Resnet18 blocks.

[0050] Inspired by the human anomaly recognition process, the embodiment of the present invention introduces a memory expert mechanism in the process of knowledge distillation to assist the denoising learning of the denoising student network. The core of the memory expert mechanism is the normal information storage and recall network. Figure 3 As shown in the figure, a normal memory expert is established to store high-level normal features extracted from normal samples. The high-level normal features are extracted by the pre-trained ResNet18 encoder (i.e., the teacher network) to form a set of memory information representing the normal state. The teacher network extracts high-level normal features from normal samples, which can accurately reflect the high-level information of normal samples and are stored in the normal memory expert. During the training process of the student network, the normal memory expert helps it to perform denoising learning. The student network not only needs to learn how to identify normal samples, but also improve its robustness to noise and anomalies through the high-level features provided by the memory expert.

[0051] A normal learning strategy is used to help memory experts master the prior knowledge in normal data during training. First, a randomly initialized normal memory expert is established to store a certain number of memory vectors m i In the training phase, the normal samples in the training set are fed into the teacher network, and the encoder of the teacher network encodes the k training samples into normal sample features. ,in , Representing a d-dimensional real vector space defined by d real coordinates, these normal sample features are stored in the normal memory expert to recall the learned normal knowledge from the query features.

[0052] In order to further improve the learning effect, an adaptive weighted update mechanism is introduced to adjust the weight based on the dynamic relationship between the memory vector and the normal sample features of the current input:

[0053] ;

[0054] in, is the value of the memory vector in the tth iteration; is the value of the memory vector in the t+1th iteration; It is a dynamic learning rate, which is adaptively adjusted according to the change of the loss function in each iteration step; is an appropriately selected nonlinear function, where the Sigmoid function is used to adjust the update amplitude; is the normal sample feature of the current input; · represents the element product. Adaptively adjust the learning rate according to the loss of each iteration to converge faster or avoid overfitting; Introducing nonlinear adjustment, we can use the memory vector m i With normal sample characteristics The similarity of the dynamic adjustment of the update amplitude enhances the flexibility and expression ability of the model; element product Preserving the relative direction information between the input normal sample features and the memory vector helps to more accurately capture the relationship between features. is flattened and combined with the memory vector m in the memory expert i Calculate the cosine similarity, obtain the similarity weight through the softmax activation function, and form the first similarity score matrix :

[0055] ;

[0056] Where K is the total number of vectors of normal memory experts. Then m i according to Aggregate to get the first normalized feature :

[0057] ;

[0058] In order to ensure that normal information is memorized from normal sample features, the normal memory loss function L is used. mem To minimize and The difference between:

[0059] ;

[0060] in, represents the sparsity loss function, represents the cross-sample consistency loss function, , are the weight coefficients of the sparsity loss function and the cross-sample consistency loss function, respectively.

[0061] In order to promote the sparsity of memory vectors and avoid overfitting, a sparse loss function is introduced to ensure that the memory vector m i With sparseness:

[0062] .

[0063] In order to ensure the consistency of features between different samples, cross-sample consistency loss is introduced to ensure that the structural relationship between samples remains consistent after reconstruction:

[0064] ;

[0065] in, Represents different normal samples and The difference between Represents different normal samples and The first normalized feature and The difference between.

[0066] After completing the training of the teacher network for the normal memory expert, the ultimate goal of the normal information storage and recall network is to adaptively adjust the normality of the features generated by the student network. To model the process of extracting memory prior knowledge using query features, the pseudo-abnormal images corresponding to the training set are input into the denoising student network. The query features are obtained through the denoising student network. , and call the normal information stored in the memory expert according to the query features.

[0067] Specifically, the pseudo-abnormal images corresponding to the training set are fed into the denoising student network, and the k training samples are encoded into query features through the denoising student network. ,in The query feature recalls the normal knowledge learned by the normal memory expert. Each query feature is flattened and combined with the memory vector m in the normal memory expert iCalculate the cosine similarity, obtain the similarity weight through the softmax activation function, and form the second similarity score matrix :

[0068] ;

[0069] Where K is the total number of vectors of memory experts.

[0070] The second similarity score matrix Controls how much normality is required to integrate the normalization. Then m i according to Aggregate to get the second normalized feature :

[0071] ;

[0072] Finally, Transformed to , and the original query features Further splicing forms the input of the student network in the next stage.

[0073] The encoder of the denoising student network includes four Resnet18 blocks and three normal information storage and recall networks, wherein the first Resnet18 block is connected to the first normal information storage and recall network NMR1, the first normal information storage and recall network NMR1 is connected to the second Resnet18 block, the second Resnet18 block is connected to the second normal information storage and recall network NMR2, the second normal information storage and recall network NMR2 is connected to the third Resnet18 block, the third Resnet18 block is connected to the third normal information storage and recall network NMR3, and the third normal information storage and recall network NMR3 is connected to the fourth Resnet18 block. The teacher network includes three Resnet18 blocks, corresponding to the first three Resnet18 blocks of the denoising student network encoder, and the features extracted by each Resnet18 block are input into the normal information storage and recall network after the corresponding Resnet18 block in the denoising student network to train the normal memory expert in the corresponding normal information storage and recall network.

[0074] The adaptive perceptual state enhancement module is based on the Mamba state space model and considers the needs of the model in this embodiment. It improves the post-processing process after the visual state space block to extract global and local information, so that the denoising student network can pay more attention to the accurate restoration of details and structures during denoising.

[0075] Specifically, the adaptive perception state enhancement module uses the visual state space block to extract spatial long-range dependencies, and on this basis adds the extraction of local information. After using the visual state space block to complete the extraction of spatial features, a strategy combining average pooling and maximum pooling is adopted to improve the ability to capture local area features, helping the model to more accurately locate and target objects of interest.

[0076] The structure of the adaptive perception state enhancement module EP-VSS is as follows: Figure 4 As shown in Figure 2, for the input feature X input to the adaptive perception state enhancement module, X is first sent to the visual state space block VSS for processing to obtain the output result of the visual state space block. . Then, Average pooling and maximum pooling are performed separately to capture the global average feature and the global maximum feature respectively, and then these two features are concatenated (cat) to enhance the model's global understanding of the input data and obtain the concatenated feature y. The calculation formula of the concatenated feature y can be expressed as:

[0077] ;

[0078] in, is the output result of the visual state space block, H, w represent the height and width of the feature map respectively, (H, 1) and (1, w) represent the window size of the pooling operation, and pooling is performed in the height and width directions respectively.

[0079] Then, the features are extracted through the convolution layer, and the nonlinear expression ability of the model is enhanced through batch normalization BN and Sigmoid activation function to obtain enhanced features. , the calculation formula is as follows:

[0080] ;

[0081] Among them, conv(y) means applying the convolution operation to y, Represents element-wise multiplication.

[0082] Afterwards, Split into two parts along the second dimension (channel dimension), each part has a size of H and W, and obtains the segmented feature map x w and x h The formula is as follows:

[0083] .

[0084] After that, the attention weights a in the width and height directions are calculated through the convolution layer and the Sigmoid activation function w 、a h .

[0085] ;

[0086] ;

[0087] in, is the Sigmoid activation function, For x w Permutate to accommodate the convolution operation. Finally, multiply the calculated attention weights in the width and height directions to obtain the final attention map, and multiply it with the input feature map to enhance important features:

[0088] ;

[0089] in, identity Represents the identity mapping of the input, that is, directly using the input feature X of the input; out is the feature map output by the adaptive perception state enhancement module.

[0090] like Figure 2 As shown, in order to improve the performance of the feature distillation network and optimize the network structure of the denoising student network, the embodiment of the present invention adopts a dual-domain comparison method between the denoising student network and the teacher network for extracting normal features. Based on the traditional spatial domain analysis, this method introduces a more efficient frequency domain information analysis strategy, and performs a dual-domain comparison between the output of the three scales of the teacher network and the output of the last Resnet18 block and two adaptive perception state enhancement modules of the denoising student network, thereby promoting the denoising student network to reconstruct a higher quality normal image through the guidance of the teacher network.

[0091] Specifically, the image obtained by the teacher network and the corresponding image processed by the denoising student network are first passed through the lossless frequency domain feature encoder LFDFE. The key to this step is to achieve feature conversion from the spatial domain to the frequency domain without losing any image information. After that, the redundant information is removed through a filter. On this basis, a dual-domain composite loss function is further constructed, which comprehensively considers the image restoration performance in the feature space and the image restoration efficiency in the wavelet frequency domain space.

[0092] The lossless frequency domain feature encoder first performs a Haar wavelet transform on the input feature map I. This process decomposes the feature map into four key components: the approximate (low-frequency) component A, and the detail (high-frequency) components in the horizontal C, vertical V, and diagonal D directions. The four components are concatenated and further refined through a filter. The filter consists of a standard 1×1 convolution layer, a batch normalization layer, and a RELU activation function, the purpose of which is to filter out redundant information and provide more effective representative features. The process of Haar wavelet transform can be expressed by the following formula:

[0093] ;

[0094] Among them, I Haar represents the features obtained after Haar wavelet transform, and H() represents Haar wavelet transform.

[0095] After concatenating the four key components obtained from the wavelet transform, they are further processed through filters:

[0096] ;

[0097] The four key components are concatenated by cat, further processed by conv1×1 convolution, and then the features after filter processing are obtained by batch normalization BN and activation function RELU. .

[0098] Wavelet transform can provide multi-scale and multi-directional representation of images, which helps to more accurately identify and remove noise while retaining important structural information of the image. By minimizing the difference between the wavelet transform of normal image features and the wavelet transform of denoised features through the frequency domain loss function, we can learn how to effectively restore image details in the wavelet domain. Frequency domain loss function It can be expressed as:

[0099] ;

[0100] Where W represents lossless frequency domain feature coding, is the normal image feature output by the teacher network, is the denoised image feature output by the denoising student network, r represents different scales, and R=3 represents the number of scales of three different scales. The differences in frequency domain between the output features of the first Resnet18 block of the teacher network and the last adaptive perceptual state enhancement module of the denoising student network decoder, the output features of the second Resnet18 block of the teacher network and the penultimate adaptive perceptual state enhancement module of the denoising student network decoder, and the output features of the third Resnet18 block of the teacher network and the last Resnet18 block of the denoising student network decoder are compared respectively. The frequency domain loss function is minimized. , the network is able to optimize its parameters in the wavelet domain to achieve better denoising performance.

[0101] Combining the loss in the feature domain and the loss in the frequency domain, we get the dual-domain composite loss function , so that the network can achieve optimization in both spaces:

[0102] ;

[0103] ;

[0104] in, represents the feature domain loss function, r represents different scales, R=3, , is the weight parameter used to balance the two loss terms, h and w represent the height and width of the feature map respectively. and denote the original and denoised eigenvalues ​​at position (i, j), respectively.

[0105] Three different scales and Fusion to a unified scale, and splicing to obtain a significant difference feature map , the formula is as follows:

[0106] ;

[0107] Among them, r=1, 2, 3 represent three different scales, U() represents the upsampling operation, and cat() represents the concatenation operation.

[0108] In step S3, an anomaly refinement network of the anomaly detection model is constructed to further refine the difference-significant feature map obtained by the feature distillation network to improve the accuracy of anomaly detection. Figure 6 As shown in the figure, the anomaly refinement network includes a multi-scale feature extraction module, a self-attention fusion module, a cascade convolution module, a global context perception module, an attention-guided feature fusion module, and a refinement output module, which are connected in sequence and finally generate anomaly detection results.

[0109] Among them, the multi-scale feature extraction module extracts information from the input feature map at different scales to enhance the ability to capture abnormal features at multiple scales. The feature extraction at each scale uses a combination of convolutional layers and pooling layers, and the formula is as follows:

[0110] ;

[0111] Among them, Conv represents convolution, Pooling represents pooling operation, Represents the multi-scale features extracted at the r-th scale.

[0112] Then, the features of different scales are fused through the self-attention fusion module, and the key areas are highlighted through the self-attention mechanism. The formula is as follows:

[0113] ;

[0114] in, Represents calculating self-attention weights and applying these weights to multi-scale features , Represents the output features of the self-attention fusion module.

[0115] After that, through the cascade convolution modules, each convolution layer further refines the features based on the previous convolution layer. The formula is as follows:

[0116] ;

[0117] Among them, BN represents normalization, RELU represents activation function, Represents the output features of the cascaded convolutional module.

[0118] Then, the global context awareness module enhances the capture of global information through the global context awareness mechanism. The formula is as follows:

[0119] ;

[0120] Among them, GAP represents global average pooling, FC represents fully connected layer, Represents the output features of the global context-aware module.

[0121] Afterwards, the attention-guided feature fusion module fuses multi-scale features through a multi-head attention mechanism to enhance feature expression capabilities. The formula is as follows:

[0122] ;

[0123] ;

[0124] in, represents the attention mechanism of the 1st, 2nd, ..., nth heads, cat represents concatenation, represents the fused multi-scale features, Represents the output features of the attention-guided feature fusion module. Finally, the refinement output module processes the final features to generate anomaly detection results.

[0125] ;

[0126] Among them, Upsample represents the upsampling operation, which is used to increase the resolution of the feature map. Represents the output features of the refined output module.

[0127] Fixed feature distillation network. The pseudo-anomaly images are fed into the teacher network and the denoising student network respectively, and the features extracted by the teacher network and the denoising student network are used to train the anomaly refinement network. An anomaly refinement loss function combining multi-scale weighted loss and adaptive fusion loss is designed to better capture anomaly features while balancing the influence of different categories, thereby improving the performance of the anomaly refinement network.

[0128] Multi-scale weighted loss function It aims to enhance the focus on abnormal areas through feature extraction and weighted processing at different scales. The formula is as follows:

[0129] ;

[0130] Where S is the number of scales, N s is the number of samples at the sth scale, is the weight of the sth scale, is the weighting parameter of the sample at the sth scale, which is used to balance the losses at different scales. is the true label of sample c at the sth scale, is the predicted value of sample c at the sth scale.

[0131] Adaptive fusion loss function By adaptively fusing the outputs of multiple feature layers, the anomaly detection performance is optimized, and the formula is as follows:

[0132] ;

[0133] Where N is the number of samples, is the adaptive weight of sample d, which is used to balance the loss of different samples, L is the number of feature layers, is the fusion weight of the lth feature layer, is the true label of sample d, is the predicted value of sample d at the lth feature layer.

[0134] The anomaly refinement loss function combines multi-scale weighted loss and adaptive fusion loss to optimize the performance of the anomaly refinement network. The formula is as follows:

[0135] ;

[0136] in, and They are the weight parameters of the multi-scale weighted loss and the adaptive fusion loss, respectively, used to balance the contribution of the two losses.

[0137] In step S4, the defect detection model is constructed and the defect detection model is trained using the training set, wherein the normal memory loss function is used to Training the normal information storage and recall network based on the dual-domain composite loss function Complete the training of the feature distillation network based on the anomaly refinement loss function The training of the abnormal refinement network is completed. Then the test set is used to verify the model effect.

[0138] The training process is divided into two stages. In the first stage, normal images and pseudo-abnormal images are used as inputs to the teacher network and the denoising student network, respectively. After the teacher network completes the training of the normal information storage and recall network, the intermediate features output by the denoising student network encoder are input into the normal information storage and recall network for reorganization, and then sent to the decoder part of the denoising student network. The goal of training is to make the feature representation generated by the denoising student network as similar as possible to that of the teacher network. Finally, the feature distillation network generates a significant difference feature map. In the second stage, the student network is fixed, and pseudo-abnormal images are used as inputs to the teacher network and the student network. The anomaly refinement network receives the significant feature map from the feature distillation network and generates the final anomaly detection result through multi-level refinement processing. The loss function of the model consists of three parts: the normal memory loss function used to train the normal information storage and recall network , Dual-domain composite loss function for training denoising student network , including the frequency domain loss function And the feature domain loss function , and the anomaly refinement loss function used to train the anomaly refinement network .

[0139] In the training of the defect detection model, the first 1000 rounds focus on memory enhancement and feature optimization, training the normal information storage and recall network and the feature distillation network, and cultivating the anomaly removal ability of the student network; the subsequent 4000 rounds focus on feature refinement and performance optimization, and are committed to the training and performance optimization of the anomaly refinement network to further improve the accuracy and generalization ability of the anomaly detection system. After every 1000 rounds, the model is evaluated using an independent test set, and the weight of the model with the best performance is retained. This strategy ensures that the best performing model can be selected for optimization at each stage of the training process, and finally the model with the best performance on the test set is selected for deployment, thereby ensuring that the defect detection model achieves the best performance in practical applications.

[0140] Embodiment 2

[0141] This embodiment provides an unsupervised anomaly detection system based on memory expert guidance, including:

[0142] An image acquisition module is configured to acquire an image of a defect to be detected;

[0143] A defect detection module is configured to input a defect image to be detected into a trained defect detection model for detection, and obtain a defect detection result of the glass container surface;

[0144] Among them, the defect detection model includes a feature distillation network for extracting a significant difference feature map and an anomaly refinement network for generating a defect detection result based on the significant difference feature map, which are connected in sequence. The significant difference feature map is obtained based on the denoising student network and the teacher network of the feature distillation network; the feature distillation network helps the denoising student network to learn normal samples based on a normal memory expert, and the normal memory expert stores the memory vectors obtained by the teacher network according to the normal sample characteristics. The denoising student network updates the query features generated according to the defect image to be detected according to the memory vectors.

[0145] It should be noted here that the various modules in this embodiment correspond one-to-one to the steps of the method in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An unsupervised anomaly detection method based on memory expert guidance, characterized in that: include: Acquire an image of a defect to be detected; The defect image to be detected is input into the trained defect detection model for detection to obtain the surface defect detection result of the glass container; The defect detection model includes a feature distillation network for extracting a significant difference feature map and an anomaly refinement network for generating a defect detection result according to the significant difference feature map, which are connected in sequence. The significant difference feature map is obtained according to a denoising student network and a teacher network of the feature distillation network. The feature distillation network helps the denoising student network learn normal samples based on a normal memory expert. The normal memory expert stores a memory vector updated by the teacher network according to the normal sample features. The denoising student network updates the query features generated according to the defect image to be detected according to the memory vector. The feature distillation network includes a teacher network and a denoising student network. The denoising student network is an encoder-decoder architecture. The encoder includes four Resnet18 blocks. The first three Resnet18 blocks of the encoder are connected to a normal information storage and recall network. Each of the normal information storage and recall networks includes a normal memory expert. The normal memory expert updates its own memory vector according to the normal sample features extracted from the normal sample by the corresponding layer of the teacher network. The updating process of the memory vector is expressed as: ； in, is the value of the memory vector in the tth iteration, is the value of the memory vector in the t+1th iteration, is the dynamic learning rate, is a nonlinear function, is the normal sample feature; · represents element-wise multiplication.

2. The unsupervised anomaly detection method based on memory expert guidance as claimed in claim 1, characterized in that: The loss function in the normal memory expert training process is the normal memory loss function. for: ； in, represents the sparsity loss function, represents the cross-sample consistency loss function, , is the weight coefficient, k is the number of training samples, is the first normalized feature, obtained by aggregating the memory vectors according to a first similarity score matrix, and the first similarity score matrix is ​​obtained according to the similarity between the normal sample features and the memory vectors.

3. The unsupervised anomaly detection method based on memory expert guidance as claimed in claim 1, characterized in that: The denoising student network updates the query features generated according to the defect image to be detected according to the memory vector, including: The normal information storage and recall network calculates the similarity between the input query features and the memory vectors stored in the normal memory expert to obtain a second similarity score matrix; aggregates the memory vectors according to the second similarity score matrix to obtain a second normalized feature; and concatenates the second normalized feature with the query feature to obtain the output of the normal information storage and recall network.

4. The unsupervised anomaly detection method based on memory expert guidance as claimed in claim 1, characterized in that: The decoder of the denoising student network includes two Resnet18 blocks and two adaptive perception state enhancement modules connected after the Resnet18 blocks. The adaptive perception state enhancement module uses the visual space state block to process the input features, and respectively obtains the global average feature and the global maximum feature of the output result of the visual state space block, splices the global average feature and the global maximum feature to obtain the spliced ​​feature, and enhances the spliced ​​feature to obtain the output feature map.

5. The unsupervised anomaly detection method based on memory expert guidance as claimed in claim 4, characterized in that: The enhancement of the splicing feature includes: The concatenated features are processed through convolution layer, batch normalization and activation function in turn to obtain enhanced features; Segmentation enhancement features, based on the segmented feature map, obtain the attention weights in the width and height directions respectively; The acquired attention weights in the width and height directions are multiplied with the input features of the adaptive perception state enhancement module to obtain the output feature map.

6. The unsupervised anomaly detection method based on memory expert guidance as claimed in claim 4, characterized in that: The feature maps output by the last Resnet18 block and the two adaptive perceptual state enhancement modules in the decoder of the denoising student network are compared with the feature maps output by the corresponding layers of the teacher network in the spatial and frequency domains, so as to guide the denoising student network to reconstruct the normal image through the teacher network.

7. The unsupervised anomaly detection method based on memory expert guidance as claimed in claim 1, characterized in that: The significant difference feature map is obtained by correspondingly fusion and splicing the output features of the last three layers of the denoising student network decoder and the output features of the three scales of the teacher network; the anomaly refinement network includes a multi-scale feature extraction module, a self-attention fusion module, a cascade convolution module, a global context perception module, an attention-guided feature fusion module, and a refinement output module connected in sequence. The anomaly refinement network further refines the significant difference feature map to generate anomaly detection results.

8. An unsupervised anomaly detection system based on memory expert guidance, characterized in that: include: An image acquisition module is configured to acquire an image of a defect to be detected; A defect detection module is configured to input a defect image to be detected into a trained defect detection model for detection, and obtain a defect detection result of the glass container surface; The defect detection model includes a feature distillation network for extracting a significant difference feature map and an anomaly refinement network for generating a defect detection result according to the significant difference feature map, which are connected in sequence. The significant difference feature map is obtained according to a denoising student network and a teacher network of the feature distillation network. The feature distillation network helps the denoising student network learn normal samples based on a normal memory expert. The normal memory expert stores a memory vector updated by the teacher network according to the normal sample features. The denoising student network updates the query features generated according to the defect image to be detected according to the memory vector. The feature distillation network includes a teacher network and a denoising student network. The denoising student network is an encoder-decoder architecture. The encoder includes four Resnet18 blocks. The first three Resnet18 blocks of the encoder are connected to a normal information storage and recall network. Each of the normal information storage and recall networks includes a normal memory expert. The normal memory expert updates its own memory vector according to the normal sample features extracted from the normal sample by the corresponding layer of the teacher network. The updating process of the memory vector is expressed as: ； in, is the value of the memory vector in the tth iteration, is the value of the memory vector in the t+1th iteration, is the dynamic learning rate, is a nonlinear function, is the normal sample feature; · represents element-wise multiplication.

Images