Image anomaly detection method and computer device

Abstract

The application relates to an image anomaly detection method and computer equipment. The method comprises the following steps: inputting a to-be-detected image into a first encoder and a second encoder respectively to extract a multi-level first feature set and a multi-level second feature set; inputting the multi-level first feature set into a first decoder to obtain an anomaly-enhanced feature set; performing embedding fusion processing on the multi-level second feature set to generate embedded features and input the embedded features into a second decoder for reconstruction to obtain a reconstructed feature set; calculating the similarity features between the reconstructed feature set and the multi-level first feature set and the anomaly-enhanced feature set respectively, and then screening target similarity features; performing cross-attention fusion processing on the multi-level first feature set and the target similarity features to obtain enhanced similarity features; processing the enhanced similarity features by using a segmentation head to generate an anomaly probability map, and calculating an image anomaly score according to the anomaly probability map. The method can improve the reliability of image anomaly detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image anomaly detection method and computer device. Background Technology

[0002] Unsupervised anomaly detection methods are widely used in industrial scenarios where anomaly samples are scarce, and models are typically trained using only normal images.

[0003] Existing methods primarily rely on reconstruction to train models on normal images, achieving anomaly localization by comparing the differences between the input and reconstructed images. However, the strong generalization ability of deep networks can lead to well-reconstructed anomaly regions, reducing detection performance. Embedding-based methods utilize pre-trained models to extract features, detecting anomalies by calculating the feature distance between test samples and normal samples. However, these methods require maintaining large feature libraries, incurring storage and computational burdens. Anomaly simulation-based methods transform unsupervised problems into supervised learning by generating pseudo-anomaly samples. However, their performance depends on the similarity between pseudo-anomalies and real anomalies. Knowledge distillation-based methods utilize frozen pre-trained networks to provide stable feature representations, training another network to learn normal patterns. However, existing knowledge distillation techniques suffer from limitations in anomaly localization accuracy due to over-generalization of trainable networks to anomaly features, lack of sensitivity of pre-trained networks to anomaly features, and a lack of autonomous discrimination ability in the segmentation module. Summary of the Invention

[0004] Therefore, it is necessary to provide an image anomaly detection method and computer equipment that can suppress the overgeneralization of abnormal features, enhance the ability to distinguish abnormal regions, and improve the accuracy of anomaly localization, in order to address the above-mentioned technical problems.

[0005] To address the aforementioned technical problems, firstly, an image anomaly detection method is provided, the method comprising: The image to be detected is acquired and input into the first encoder and the second encoder respectively to extract multi-level first feature sets and multi-level second feature sets; The multi-level first feature set is input into the first decoder for anomaly enhancement decoding processing to obtain the anomaly enhancement feature set; The multi-level second feature set is embedded and fused to generate embedded features. The embedded features are then input into the second decoder for reconstruction to obtain the reconstructed feature set. Calculate the first similarity feature between the features of the reconstructed feature set and the features of the first feature set of the multi-level system, and the second similarity feature between the features of the reconstructed feature set and the features of the anomaly enhancement feature set; Target similarity features are selected based on the first and second similarity features; The enhanced similarity features are obtained by cross-attention fusion processing of the multi-level first feature set and the target similarity features. The enhanced similarity features are processed using a segmentation head to generate an anomaly probability map, and the image anomaly score of the image to be detected is calculated based on the anomaly probability map.

[0006] In one embodiment, the first encoder and the second encoder each include a plurality of residual modules connected in sequence. The image to be detected is input to the first encoder and the second encoder respectively to extract a multi-level first feature set and a multi-level second feature set, including: The image to be detected is input into the first encoder. According to the connection order of each residual module of the first encoder, the image to be detected or the feature map output by the residual module of the previous sequence is convolved and downsampled in turn. Multiple feature maps of different levels with progressively decreasing resolution are output as the first feature set of multiple levels. The image to be detected is input into the second encoder. According to the connection order of each residual module of the second encoder, the image to be detected or the feature map output by the residual module of the previous sequence is convolved and downsampled in turn. Multiple feature maps of different levels with progressively decreasing resolution are output as a multi-level second feature set.

[0007] In one embodiment, the multi-level first feature set is input to the first decoder for anomaly enhancement decoding processing, resulting in an anomaly enhancement feature set including: The first feature set of multiple levels is input into the first decoder, and the convolutional layer in the first decoder performs channel adjustment on the first feature set of each level. According to the reverse order of the connection order of each residual module in the first decoder, each residual module upsamples the feature map with the lowest resolution in the first feature set of the multi-level layers or the feature map output by the residual module in the previous order. The upsampling result is then concatenated with the feature map in the first feature set after channel adjustment corresponding to each residual module level. The concatenated result is then input into the convolutional layer for residual learning, and the decoding features of each residual module level are output. The decoding features output by the last residual module are used as the anomaly enhancement feature set.

[0008] In one embodiment, the multi-level second feature set is subjected to embedding fusion processing to generate embedded features. The embedded features are then input to a second decoder for reconstruction, resulting in a reconstructed feature set including: The multi-level second feature set is input into the multi-level feature bottleneck embedding module. The multi-level feature bottleneck embedding module fuses the feature maps of each level from bottom to top through multiple cascaded bottleneck fusion units, and suppresses abnormal information during each fusion process to generate embedded features. The embedded features are input into the second decoder, which performs progressive upsampling and residual processing on the embedded features through multiple upsampling layers and multiple residual modules, and outputs the reconstructed feature set.

[0009] In one embodiment, calculating a first similarity feature between the reconstructed feature set and features in the first feature set, and a second similarity feature between the reconstructed feature set and features in the anomaly enhancement feature set, and filtering target similarity features based on the first similarity feature and the second similarity feature includes: After normalizing the reconstructed feature set and the feature maps of the same level in the first multi-level feature set, they are multiplied element by element to obtain the first set of multi-level similarity features. The first set of multi-level similarity features is summed along the channel dimension and then inverted to obtain the first distance map. After normalizing the feature maps of the same level in the reconstructed feature set and the anomaly enhancement feature set respectively, they are multiplied element by element to obtain the second set of multi-level similarity features. The second set of multi-level similarity features is summed along the channel dimension and then inverted to obtain the second distance map. Pixel-by-pixel comparison of the first distance map and the second distance map: if the value of the first distance map is greater than the value of the second distance map, then the feature vector in the first group of multi-level similarity features for that pixel is selected as the target similarity feature; if the value of the first distance map is less than the value of the second distance map, then the feature vector in the second group of multi-level similarity features is selected as the target similarity feature. Arrange the selected feature vectors from all pixels according to the pixels to form a target similarity feature map and output it.

[0010] In one embodiment, the multi-level first feature set and the target similarity features are cross-attention fused to obtain the enhanced similarity features, including: After upsampling the feature maps of each level in the first feature set of multiple levels to the same spatial resolution, channel stitching is performed to obtain the teacher guidance features; After upsampling the feature maps of each level in the target similarity feature map to the same spatial resolution, the channels are stitched together to obtain the similarity fusion feature. The teacher guidance features and similarity fusion features are divided into several image blocks, and each image block is linearly mapped to obtain the teacher embedding vector and the similarity embedding vector. Generate query vectors and key vectors from teacher embedding vectors, and generate value vectors from similarity embedding vectors; Attention weights are calculated based on the query vector and key vector. The value vectors are then weighted and fused using these attention weights, and finally added to the similarity embedding vector to obtain the global guiding features. The global guiding features are restored to the same spatial resolution as the similarity fusion features, and then adaptively fused with the similarity fusion features to obtain the enhanced similarity features.

[0011] In one embodiment, the enhanced similarity features are processed using a segmentation head to generate an anomaly probability map. The calculation of the image anomaly score of the image to be detected based on the anomaly probability map includes: The segmentation head extracts features from the enhanced similarity features to obtain the first feature map; The first feature map is subjected to multi-scale convolution, and the multi-scale outputs are concatenated along the channel dimension to obtain the second feature map. The second feature map is fused and its dimensionality reduced to obtain the third feature map; The third feature map is restored to the spatial resolution of the input image to obtain the fourth feature map; The value of each pixel in the fourth feature map is converted into a probability value to generate a pixel-level anomaly probability map. Select a preset number of pixels with the largest pixel values ​​from the pixel-level anomaly probability map, and calculate the average value of the preset number of pixels as the image anomaly score of the image to be detected.

[0012] In one embodiment, the first encoder is pre-trained and its parameters are frozen, and the method further includes training the first decoder: Obtain the first training sample set, which includes normal images and pseudo-anomaly images; Normal images and pseudo-abnormal images from the first training sample set are input into the first encoder to extract multi-level normal feature maps and multi-level pseudo-abnormal feature maps. The difference between each level of normal feature map and pseudo-abnormal feature map is calculated as a perturbation term. The perturbation term is weighted and fused with the downsampled pseudo-anomaly mask to construct a supervisory signal; The pseudo-anomaly images in the first training sample set are input into the first decoder. The cosine distance loss between the output features of the first decoder and the supervision signal is calculated. The parameters of the first decoder are updated based on the cosine distance loss.

[0013] In one embodiment, the method further includes training a second encoder and a second decoder: Obtain a second training sample set, which includes normal images; For each normal image in the second training sample set, construct the corresponding pseudo-abnormal image and pseudo-abnormal mask. Normal images from the second training sample set are input into the first encoder to extract multi-level normal feature maps of the target; pseudo-abnormal images are input into the second encoder, and after embedding and fusion processing and reconstruction by the second decoder, the target reconstruction feature set is obtained. Calculate the target cosine distance loss between the target reconstructed feature set and the target multi-level normal feature map, and update the parameters of the second encoder and the second decoder based on the target cosine distance loss.

[0014] To address the aforementioned technical problems, a second aspect provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: the processor executes the computer program to perform the steps of the method described in the first aspect.

[0015] Unlike existing technologies, this application extracts multi-level features from the image to be detected using a first encoder and a second encoder. The first encoder provides a stable feature benchmark, while the second encoder extracts the features to be detected. The multi-level first features extracted by the first encoder are input into a first decoder for anomaly enhancement decoding, enhancing the feature response of the abnormal region. The multi-level second features extracted by the second encoder are then fused and anomaly suppressed through a multi-level feature bottleneck embedding module to generate clean embedded features, which are then input into the second decoder for reconstruction, outputting reconstructed features. Similarity features between the reconstructed features and the first feature, and between the reconstructed features and the anomaly enhancement features, are calculated to capture anomaly information from different perspectives. By comparing similarity features pixel by pixel, features with stronger anomaly responses are adaptively selected to reduce missed detections. The selected target similarity features are fused with the first feature through cross-attention, using the global semantic relationship of the first feature to guide the similarity feature fusion, enhancing spatial context modeling capabilities. Finally, a segmentation head is used for multi-scale feature extraction and fusion to highlight local abnormal regions and generate accurate image-level anomaly scores. Through the above steps, this application effectively solves the problems of insufficient anomaly feature discrimination capability and limited positioning accuracy in existing technologies, achieving accurate identification and positioning of abnormal regions. Attached Figure Description

[0016] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating an image anomaly detection method in one embodiment; Figure 2 This is a schematic diagram of the teacher-student model in one embodiment; Figure 3 This is a schematic diagram of the structure of a multi-level feature bottleneck embedding module in one embodiment; Figure 4 This is a flowchart illustrating a strong anomaly selection method in one embodiment; Figure 5 This is a schematic diagram of the segmentation network structure in one embodiment; Figure 6 This is a flowchart illustrating a teacher perception guidance method in one embodiment; Figure 7 This is a schematic flowchart of a discriminative residual distillation method in one embodiment; Figure 8 This is a schematic diagram illustrating the location of an abnormal region in one embodiment; Figure 9 This is a schematic diagram illustrating the location of abnormal regions in another embodiment; Figure 10 This is a schematic diagram of locating abnormal areas in yet another embodiment; Figure 11 This is a structural block diagram of an image anomaly detection device in one embodiment; Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] Unsupervised anomaly detection and localization methods are widely used in industrial scenarios where anomaly samples are scarce. These methods typically train models using only normal images, without relying on any anomaly samples, thus reducing data acquisition and annotation costs. Existing anomaly detection methods can be mainly divided into four categories: reconstruction-based methods, embedding-based methods, anomaly simulation-based methods, and knowledge distillation-based methods.

[0020] Reconstruction-based methods typically train models on normal images to achieve pixel-level reconstruction capabilities. When the input image contains anomalous regions, the model struggles to effectively reconstruct these regions, thus achieving anomaly localization by comparing the differences between the input and reconstructed images. These methods primarily include models based on autoencoders, generative adversarial networks (GANs), and Transformers. However, due to the strong generalization ability of deep networks, some anomalous regions may still be reconstructed well, leading to a decrease in detection performance. To mitigate this issue, some methods introduce noise or pseudo-anomaly masks into the input image, transforming the reconstruction task into an image inpainting task, thereby suppressing the model's ability to reconstruct anomalous regions. Furthermore, other methods introduce attention mechanisms, constrain the reconstruction process, or optimize the differences between normal and anomalous distributions to avoid the model learning simple identity mappings and reduce computational overhead.

[0021] Embedding-based methods typically utilize models pre-trained on large-scale datasets to extract image features, constructing a feature distribution space from normal samples, and achieving anomaly detection and localization by calculating the feature distance between test samples and normal samples. These methods offer good expressive power and inference efficiency, but often require maintaining a large feature library, resulting in storage and computational burdens. To improve efficiency, some methods construct representative feature subsets to reduce redundant information, thereby achieving more efficient memory utilization.

[0022] Anomaly simulation-based methods transform unsupervised problems into supervised learning problems by generating pseudo-anomaly samples, thereby significantly improving detection performance. These methods typically employ image patch copying and pasting, texture replacement, noise perturbation, or utilize generated pseudo-anomaly regions to enable the model to learn to distinguish between normal and abnormal features. While these methods effectively enhance the model's discriminative ability, their performance depends on the similarity between pseudo-anomalies and real anomalies; when the difference is significant, the model's generalization ability may still be affected.

[0023] In recent years, the knowledge distillation paradigm based on teacher-student models has proven effective in anomaly detection. During the training of the teacher-student model, the teacher, pre-trained on a large dataset, is frozen, while the student only learns the pre-trained teacher's feature representations under normal images. Therefore, during inference, the student will generate feature representations different from the teacher's in abnormal regions of the image. By comparing the feature similarity between the two, the abnormal pixels can be located. Since the student network may overgeneralize to abnormal features, a reverse knowledge distillation paradigm has been proposed. This paradigm uses only a pre-trained encoder as the teacher and the decoder as the student, preventing the student from generating the same representation as the teacher network in abnormal regions due to the same or similar network structure and data flow. Building on this, DeSTSeg (segmentation-guided denoising student-teacher network) and DMDD (mutually exclusive multipath directional diffusion routing) introduce pseudo-anomalies as constraints to optimize the student network's reconstruction of normal feature representations under pseudo-anomaly input, further avoiding overgeneralization. Furthermore, it proposes using a separately trained segmentation network to supervise the fusion of features at different levels of the teacher-student model, resulting in more accurate anomaly segmentation results. However, existing research shows that students have a weak ability to reconstruct normal feature representations under abnormal image inputs, and teachers lack the ability to output sufficiently discriminative abnormal features. Furthermore, the segmentation networks in existing methods are heavily reliant on teacher-student models and lack autonomous discrimination capabilities.

[0024] To solve the above problems, such as Figure 1 As shown, an image anomaly detection method is provided, which includes the following steps: Step 101: Obtain the image to be detected, and input the image to be detected into the first encoder and the second encoder respectively to extract the multi-level first feature set and the multi-level second feature set.

[0025] Specifically, the first encoder and the second encoder each include multiple residual modules connected in sequence. The image to be detected is input into the first encoder. According to the connection order of the residual modules in the first encoder, convolution and downsampling are performed on the image to be detected or the feature map output by the previous residual module in sequence, outputting multiple feature maps of different levels with progressively decreasing resolution, serving as a multi-level first feature set. The image to be detected is input into the second encoder. According to the connection order of the residual modules in the second encoder, convolution and downsampling are performed on the image to be detected or the feature map output by the previous residual module in sequence, outputting multiple feature maps of different levels with progressively decreasing resolution, serving as a multi-level second feature set. A feature map is a multi-dimensional tensor carrying spatial structural information in a convolutional neural network. It can be understood as a set of two-dimensional matrices containing C channels, where each channel (size H×W) corresponds to the response intensity distribution map of a feature pattern on the image. Different levels of feature maps correspond to semantic information with different levels of abstraction.

[0026] The image anomaly detection method of this application is applied to an image anomaly detection system. The system includes an unsupervised anomaly detection model, which comprises a teacher-student network. The teacher-student network (teacher-student model) includes a teacher network and a student network. The teacher network includes a teacher encoder (first encoder) and a teacher decoder (first decoder). The student network includes a student encoder (second encoder) and a student decoder (second decoder).

[0027] like Figure 2 As shown, in this application, the teacher encoder and student encoder extract multi-level features in parallel. In stage 1 (encoding stage), the image to be detected is input into the teacher encoder and student encoder in parallel. The teacher encoder and student encoder have similar structures, both consisting of convolutional layers, multi-level residual modules, hierarchical spatial downsampling, and hierarchical channel attention stacking. The student encoder receives the image to be detected, and the image is processed by the student encoder's convolutional layers, multi-level residual modules, and... , , After hierarchical downsampling, the shallow features of students are output layer by layer. Mid-layer characteristics High-level characteristics ; , , Together, they constitute multi-level student features (multi-level second feature set); the extracted three-level features are uniformly input into the multi-level feature bottleneck embedding MFBE module, and the MFBE completes multi-scale feature fusion from bottom to top and suppresses abnormal information during the fusion process, and finally outputs single-scale compact embedded features.

[0028] The teacher encoder receives the image to be detected, which is then processed through the teacher encoder's convolutional layers, multi-level residuals, and... , , , Hierarchical downsampling generates native teacher features layer by layer without attention optimization. , , Teacher's original characteristics , , via Channel attention advantage Channel attention optimization, Channel attention optimization yields multi-level teacher features (multi-level first feature set).

[0029] In phase 2 (decoding phase), the teacher decoder receives multi-level teacher features. The teacher decoder adopts a reverse decoding architecture, through T... D (Illustration) , , Upsampling operations combined with cross-level skip connections splice features from the corresponding levels. / / The decoding operation is completed layer by layer through residual learning, and the final output is the anomaly enhancement feature. The compact embedded feature output by the MFBE module is input into the student decoder. The student decoder performs upsampling and residual reconstruction operations layer by layer, and outputs the reconstructed student features. The reconstructed student features are matched with the first feature set of the multi-level layer by layer to generate the first set of multi-level similarity features. The reconstructed student features are operated with the anomaly enhancement feature to generate the second set of multi-level similarity features. The above two sets of similarity features are input into the subsequent SAS strong anomaly selection module to complete the feature selection.

[0030] In anomaly detection methods using teacher-student models, the teacher network's ability to represent anomalies is crucial to overall detection performance. However, existing teacher networks often struggle to effectively distinguish between normal and anomalous features, resulting in indistinct responses in anomalous regions and missed detections. Based on the idea that "anomalous features can be considered as the result of superimposing perturbation terms onto normal features," this application proposes that the poor discriminability of anomalous features stems from the weak amplitude and limited distribution of perturbation terms, which are masked by normal features. Therefore, this application proposes a Discriminative Residual Distillation Teacher (DRDT) to enhance the teacher network's ability to perceive anomalies. The teacher network in this application employs an encoder-decoder structure. The first encoder uses a ResNet-18 backbone network pre-trained on a large-scale dataset, with its last module removed to retain multi-scale features. The first decoder consists of multiple residual modules, whose core function is to extract normal components from anomalous features, thereby extracting information representing anomalous perturbations. This perturbation is defined as a discriminative residual, which has a more significant difference from the normal characteristics of the student network reconstruction, thereby improving the overall teacher-student framework's responsiveness to anomalous regions.

[0031] Specifically, the teacher encoder (first encoder) of this application can employ a pre-trained ResNet-18 network to provide stable feature representations. Generally, the ResNet-18 network includes 5 functional layers.

[0032] Specifically, the first functional layer (conv1+bn+relu+maxpool) is the input preprocessing layer, responsible for compressing the original image (e.g., 3×256×256) into 64 feature maps, reducing the size by 1 / 4 (from 256 to 64). conv1 refers to the convolutional layer, sometimes specifically the first convolutional layer or a single convolutional operation. bn refers to the batch normalization layer, used to accelerate network convergence and prevent gradient vanishing or exploding. Relu refers to the rectified linear unit activation function, a commonly used activation function. Maxpool refers to the max pooling layer, used for downsampling, extracting the maximum value in local regions to reduce the feature map size. It is a standard combination of convolution and pooling.

[0033] The second functional layer (Layer 1) is the first residual stage, which can contain two residual modules. It does not change the spatial resolution and number of channels, but only performs nonlinear transformation and deep semantic extraction on the features.

[0034] The third functional layer (Layer 2) is the second residual stage. It can contain two residual blocks. Downsampling is performed (spatial size changes from 64 to 32), and the number of channels doubles (from 64 to 128). A shallow feature map is also derived from this layer. (128×32×32).

[0035] The fourth functional layer (Layer 3) is the third residual stage. It contains two residual blocks. It performs further downsampling (from 32 to 16), doubling the number of channels (from 128 to 256). This leads to the mid-layer feature map. (256×16×16).

[0036] The fifth functional layer (Layer 4) is the fourth residual stage, containing 2-3 residual blocks. It downsamples the feature map (256×16×16) output from the fourth layer (spatial size from 16 to 8), doubles the number of channels (from 256 to 512), and finally outputs a feature map of 512×8×8.

[0037] The teacher encoder in this application can be based on a pre-trained ResNet-18 network, but with the fifth layer removed, retaining only the first four stages (conv1 + Layer1 to Layer3), ultimately outputting a 256×16×16 feature map. This design aims to avoid excessively small feature map sizes while preserving a 16×16 spatial resolution, which is beneficial for subsequent feature alignment or spatial awareness tasks. Simultaneously, keeping the output channels to 256 reduces computational cost and memory usage, avoiding an excessively high number of channels, and preserves mid-level semantic information. This is because the features of Layer3 are closer to the input than those of Layer4, making them more suitable as teacher features in multi-scale or knowledge distillation processes. Inputting the image to be detected into the first encoder yields a shallow feature map. Mid-layer feature map Top-level feature map (Here, the high-level feature map reuses the mid-level feature map) These features constitute a multi-level first feature set.

[0038] The student encoder (second encoder) in this application can employ a pre-trained ResNet-18 network to provide stable feature representations. Typically, a ResNet-18 network includes five functional layers. The student encoder in this application, based on a pre-trained ResNet-18 network, removes the fifth layer, retaining only the first four stages (conv1 + Layer1 to Layer3). Similar to the teacher encoder, the student encoder processes the image to be detected and can obtain shallow feature maps. Mid-layer feature map High-rise feature map (Here, the high-level feature map reuses the mid-level feature map) The second feature set is composed of multiple layers. The purpose of this design is to maintain consistency with the feature space of the teacher encoder. The teacher encoder has removed Layer 4 and outputs 256×16×16 features. The student encoder uses the same output size and can directly perform pixel-by-pixel or region-level feature distillation without the need for an additional dimension alignment module.

[0039] Furthermore, removing Layer 4 significantly reduces the number of parameters and computational cost of the student encoder, lowering the computational overhead of the student network and making it more suitable for deployment on resource-constrained terminal devices, while retaining sufficient receptive field and semantic information. It preserves higher spatial resolution, with an output feature map size of 16×16, compared to the original ResNet-18's 8×8 output, avoiding spatial information loss caused by excessively small feature maps, which is beneficial for subsequent dense prediction tasks (such as segmentation and detection) or spatially aware distillation. Controlling the number of output channels to 256 (instead of 512) prevents overfitting, reduces memory usage and computational complexity, and the mid-level semantic features of Layer 3 are more general than the high-level features of Layer 4, making them more suitable as intermediate representations for the student network's learning.

[0040] Furthermore, the teacher network and student network employ the same deletion strategy, enabling a symmetric distillation structure that eliminates the need for an adaptation layer, resulting in more stable training.

[0041] Step 102: Input the multi-level first feature set into the first decoder for anomaly enhancement decoding processing to obtain the anomaly enhancement feature set.

[0042] The multi-level first feature set is input into the first decoder. The convolutional layers in the first decoder adjust the channels of the first feature set at each level. According to the reverse order of the connection order of each residual module in the first decoder, each residual module upsamples the feature map with the lowest resolution in the multi-level first feature set or the feature map output by the residual module in the previous order. The upsampled result is then concatenated with the feature map in the first feature set with channel adjustment corresponding to each residual module level. The concatenated result is then input into the convolutional layer for residual learning. The decoded features of each residual module level are output, and the decoded features output by the last residual module are used as the anomaly enhancement feature set.

[0043] The teacher decoder (first decoder) of this application adopts a top-down progressive upsampling structure. The teacher decoder of this application can be composed of multiple residual modules connected in sequence. The number of residual modules is the same as the number of feature levels output by the teacher encoder.

[0044] For example, let's describe a teacher decoder consisting of three sequentially connected residual modules (first residual module, second residual module, and third residual module). Assume the teacher encoder outputs three levels of feature maps: shallow features... Mid-layer characteristics Highest level features .

[0045] Specifically, will , downsampling to At a resolution of [resolution value], features are obtained by fusion using 1×1 convolution. The first residual module can be responsible for processing the fused features. When the first residual module receives the fused features output by the teacher encoder (For example, with dimensions of 16×16×512) Then, the features are first obtained through the first residual module. (Size 16×16×256), which is then upsampled and used as the input to the second residual module (size 32×32×256). Simultaneously, the middle-layer features from the multi-layer first feature set are... (Dimensions 32×32×128) As a jump connection, due to The dimensions (32×32×128) do not match the current feature (32×32×256), and need to be adjusted first. The dimensions are increased to 256 using 1×1 convolutions to obtain the skip connection feature S3 (size 32×32×256). The input feature and the skip connection feature S3 are concatenated along the channel dimension to obtain the input D1 of the second residual module (size 32×32×512).

[0046] The second residual module is responsible for processing the mid-level features. Specifically, the second residual module receives and outputs feature D1 (32×32×512) from the first residual module, and obtains the feature through two residual blocks. (Size is 32×32×128). Then, an upsampling operation is performed to enlarge its spatial resolution to 64×64. Upsampling can be achieved using bilinear interpolation or transposed convolution (stride=2). This process is then applied to the middle-layer features of the first feature set across multiple layers. (Dimensions 64×64×64) as a jump connection. Because... The dimensions (64×64×64) do not match the current feature (64×64×128), so it needs to be adjusted first. The dimensions are increased (using 1×1 convolution) to 64×64×128, resulting in the skip connection feature S2. The input feature D1 and the skip connection feature S2 are then concatenated. Finally, the input D2 of the third residual module (size 64×64×256) is obtained.

[0047] The third residual module is responsible for processing shallow features. Specifically, it receives the output feature D2 (64×64×256) from the second residual module. Instead of using skip connections, the third residual module directly passes the input features sequentially through two residual blocks within itself for feature extraction and nonlinear transformation. Each residual block contains two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function, along with identity skip connections. Finally, the third residual module outputs the features... (Dimensions: 64×64×64).

[0048] Obtain the output characteristics of each residual module , , Then, the output features of each residual module are passed through the output convolutional layer to generate the final multi-scale discriminative residual features, which constitute the anomaly enhancement feature set.

[0049] The residual modules in the teacher decoder can include upsampling sub-layers, fusion sub-layers, first residual blocks, second residual blocks, skip connection sub-layers, and activation function sub-layers. The upsampling sub-layer performs bilinear interpolation or transposed convolution to enlarge the feature map space. The fusion sub-layer performs concatenation or element-wise addition to fuse the upsampled features with the encoder's skip connection features. The first residual block includes 3×3 convolutions + BN + ReLU for feature extraction and nonlinear transformation. The second residual block includes 3×3 convolutions + BN for further feature extraction. The skip connection sub-layer includes identity mapping or 1×1 convolution to directly add the module's input features to the output of the second residual block, achieving residual learning. When the number of channels or spatial size of the input and output are inconsistent, 1×1 convolution is used for dimension matching; when the dimensions are completely consistent, identity mapping is used for direct addition. The activation function sub-layer includes ReLU to perform nonlinear activation before the final output.

[0050] The residual module in the teacher decoder in this application does not correspond independently to a certain layer of the encoder. Instead, it establishes a cross-layer correspondence with the multi-level features of the encoder through a combination of step-by-step upsampling and skip fusion.

[0051] Step 103: The multi-level second feature set is embedded and fused to generate embedded features. The embedded features are then input into the second decoder for reconstruction to obtain the reconstructed feature set.

[0052] The multi-level second feature set is input to the multi-level feature bottleneck embedding module. The multi-level feature bottleneck embedding module fuses the feature maps of each level from bottom to top through multiple cascaded bottleneck fusion units, and suppresses abnormal information during each fusion process to generate embedded features. The embedded features are input to the second decoder. The second decoder performs progressive upsampling and residual processing on the embedded features through multiple upsampling layers and multiple residual modules, and outputs the reconstructed feature set.

[0053] In this application, a multi-level feature bottleneck embedding (MFBE) module is set between the student encoder and the student decoder. The multi-level feature bottleneck embedding module can adopt a bottom-up pyramid fusion structure, which consists of multiple cascaded bottleneck fusion units (BFUs). Each bottleneck fusion unit can correspond to a functional layer of the student encoder. Figure 3 The structure of a multi-level feature bottleneck embedding module is illustrated, which can be composed of multiple BFU bottleneck fusion units stacked together from top to bottom. The multi-level feature bottleneck embedding module receives multi-level student features output from the graph student encoder, and each BFU integrates embedded channel attention (illustrated). , , ) and feature fusion operator (illustrated) , The output features of the previous level unit are fused with the input features of the current level. , , During the fusion process, abnormal feature suppression is achieved. After bottleneck compression, the fused features G1, G2, and G3 of this level are output. The third-level unit outputs G3 as the overall embedding feature of MFBE and is sent to the student second decoder to complete feature reconstruction. The shallow features are first fused with the empty input, and after upsampling, they are successively fused with the middle-level features and high-level features. During each fusion process, a bottleneck structure and anomaly suppression mechanism are used to generate pure embedding features.

[0054] Specifically, each bottleneck fusion unit may include an input component, a preprocessing component, an upsampling component, a fusion component, a bottleneck compression component, an anomaly suppression component, and an output component.

[0055] The input component consists of two parts: the first part is the feature map output by the current level encoder. The first part, i, is 1, 2, or 3, corresponding to the features of the shallow, middle, and highest levels, respectively; the second part is the high-level fusion feature G output by the previous BFU. i-1For the first-level BFU (corresponding to the shallowest layer of the encoder), its higher-level fusion input can be set to an empty or zero tensor.

[0056] The two types of preprocessed features are then fused. Specifically, initial fusion can be achieved by element-wise addition or channel concatenation, followed by cross-channel information interaction through a 3×3 convolutional layer, thereby enhancing the expressive power of the fused features.

[0057] To reduce computational complexity and extract compact feature representations, the bottleneck fusion unit introduces a bottleneck structure after fusion. This bottleneck structure consists of: a compressed convolutional layer that reduces the number of channels to one-quarter of the original number; a non-linear activation layer (such as ReLU); and an expanded convolutional layer that restores the number of channels to the original dimension. This design effectively preserves core semantic information while reducing the number of parameters.

[0058] In the aforementioned bottleneck structure, this application further integrates an anomaly suppression mechanism. Specifically, a gating mechanism or attention module (including channel attention and / or spatial attention) is introduced on the intermediate features of the bottleneck layer. This module learns a set of suppression weights and selectively attenuates neurons in the feature map that have weak responses and may correspond to abnormal regions, thereby suppressing the propagation of abnormal information.

[0059] Finally, the current BFU outputs the fused feature map G. i The output serves two purposes: firstly, it propagates the fusion result of the current level upwards to the next level BFU; secondly, it participates in multi-scale interactions of higher-level features. By stacking these BFUs layer by layer, the MFBE module can fuse multi-level features from the bottom up while filtering out abnormal interference layer by layer, generating robust and compact feature embeddings.

[0060] For example, the multi-level feature bottleneck embedding module is described as consisting of three cascaded bottleneck fusion units (BFUs): The first-level BFU (BFU_1) uses shallow features output from the second layer of the encoder. (Size 64×64×64) is used as the sole input. After bottleneck fusion and anomaly suppression, the output fusion feature G1 (size 32×32×128, number of channels 128) is generated.

[0061] The second-level BFU (BFU_2) receives two types of inputs: one is the mid-level features output from the third layer of the encoder. (Size 32×32×128); the second is the fusion feature G1 output from the previous BFU. The two are fused, bottleneck compressed and anomaly suppressed, and finally the fusion feature G2 (size 16×16×256) is output.

[0062] The third-level BFU (BFU_3) also receives two types of inputs: one is the highest-level feature output from the fourth layer of the encoder. (Size 16×16×256); the second is the fusion feature G2 output from the previous BFU. The two are fused, bottleneck compressed and anomaly suppressed, and finally the fusion feature G3 (size 8×8×512) is output as the final feature embedding of the MFBE module.

[0063] By cascading multiple levels of BFUs, progressive feature interaction and fusion from shallow to deep layers can be achieved. Optionally, skip connections can be introduced between BFUs at each level to enhance gradient flow and information preservation.

[0064] In this application, when the student network receives the input image, the input image is first processed by the student encoder, which uses a pre-trained ResNet-18 and removes the fifth layer (Layer 4). It is assumed that the output consists of three layers of feature maps. (64×64×64) (128×32×32) and (256×16×16).

[0065] The three feature maps mentioned above are fed into the MFBE module in parallel. The MFBE module can be integrated into the student decoder as a pre-processing component of the decoder to preprocess the multi-level features output by the encoder before decoding and reconstruction. The MFBE module fuses these multi-level features step by step from bottom to top, and suppresses anomalous information during each fusion process, finally outputting a single compact feature embedding G3 (8×8×512).

[0066] This feature embedding is then fed into the student decoder. The decoder uses this embedding as its sole input and reconstructs the original input image through an upsampling operation.

[0067] Features of the encoder's second, third, and fourth layers (64×64×64) (128×32×32) and (256×16×16) is used as the multi-level input of MFBE and directly fed into the corresponding BFU unit. This connection method skips the early processing of the decoder, allowing feature fusion and anomaly suppression to be completed independently at the encoding end.

[0068] The MFBE employs a bottom-up, hierarchical connection method. Shallow features are first fused with an empty input (or a zero tensor) to generate G1; G1 is then fused with middle-layer features... The fusion produces G2; G2 and high-level features The fusion process produces the final embedding G3. Feature interaction and anomaly suppression are performed simultaneously at each fusion stage.

[0069] The final output G3 (8×8×512) of MFBE is fed into the student decoder as the sole feature embedding. Unlike the traditional U-Net architecture where the decoder needs to receive encoder skip connections at multiple scales, the decoder in this application only needs to process this single-scale compact feature, thereby simplifying the decoder design and reducing the computational burden.

[0070] Traditional skip connections directly pass features from each layer of the encoder to the corresponding layer of the decoder. This application, however, uses MFBE (Multi-Fold Feature Embedding) to fuse and suppress features at each layer before uniformly passing them to the decoder. Traditional skip connections do not filter features, allowing anomalous information to propagate directly to the decoder, leading to reconstruction bias. This application, however, selectively suppresses anomalous responses through gating or attention mechanisms during the fusion process of each BFU (Browser Function Unit), effectively blocking the propagation path of anomalous information. Traditional skip connections only transmit features from a single layer, lacking cross-layer interaction. This application's MFBE achieves information complementarity across multiple layers through step-by-step fusion, enhancing the diversity of feature representations. Traditional decoders need to receive multiple skip connection features of different scales, increasing design complexity. This application's decoder receives only a compact feature embedding, resulting in a simpler and more efficient structure.

[0071] Assume the output of the i-th layer encoder is The output of the previous BFU is G i-1 The current BFU output is: ; Among them, G i This represents the output feature map of the i-th Bottleneck Fusion Unit (BFU), which is the result of fusing the features of the current layer's encoder and information from all previous layers. It is then passed up to the next BFU. BN stands for Batch Normalization, which standardizes the feature map to have a mean of 0 and a variance of 1. This can accelerate network convergence, prevent gradient vanishing or exploding, and provide a slight regularization effect. 3×3 This indicates a 3×3 convolutional layer. It performs spatial information interaction and transformation on the fused feature maps to extract higher-level features. 3×3 is the most commonly used receptive field size for extracting local features in convolutional neural networks. 1×1 This represents a 1×1 convolutional layer. It's primarily used to change the number of channels in the feature map. Here, its function is to... The number of channels is transformed to be comparable to G. i-1 The same properties are used for subsequent addition or concatenation. A 1×1 convolution does not change the spatial dimensions, only the channel dimensions. This represents the original feature map output by the encoder at layer i. As another input source for the current BFU, it provides the fusion process with the most direct, unfiltered semantic or spatial information of the current layer.

[0072] G i-1 This represents the output feature map of the previous BFU layer, representing the fused information passed from shallower (higher resolution) layers of the network. For the first-layer BFU, this value is usually zero or empty. ⊕ represents the fusion operator, used to merge two feature maps. Depending on the design, this can be element-wise addition or concatenation along the channel dimension. Addition saves computation, while concatenation retains more original information. ⊙ represents the Hadamard Product, i.e., element-wise multiplication. This is a key operation for anomaly suppression. It combines the fused feature map with the weight matrix A. i Multiplying each corresponding element in the feature map enhances or weakens the response at a specific location in the feature map. A i This represents the attention weight matrix / suppression weight matrix. It is dynamically generated by the channel attention module or spatial attention module. The values ​​in the matrix typically range from 0 to 1. The closer the value is to 0, the stronger the suppression of anomalous information at the corresponding position; the closer the value is to 1, the more completely important information is preserved. G i This represents the output of the i-th level BFU. i represents the level of the BFU; in this case, i is at most 3. i It is the output of the deepest BFU, which is assigned to the final embedding feature G3.

[0073] Traditional feature bottleneck fusion only performs multi-scale feature stitching and channel compression, failing to distinguish between normal textures and abnormal pixels. Abnormal features are fused across layers along with normal features and ultimately flow into the student decoder, causing abnormal regions to be reconstructed normally by the model and anomaly detection to fail. This application introduces an attention weight matrix A in BFU. i By relying on the attention mechanism to adaptively identify abnormal region features in the feature space, and by using the Hadamard product to weighted attenuate the abnormal feature response, abnormal information is filtered out layer by layer in the feature fusion stage, ensuring that the final output embedded features retain only the normal feature representation of the image. This blocks the transmission of abnormal information to the student decoder from the encoding front end, and constrains the second decoder to only learn the distribution of normal samples, thus overcoming the defect of existing reconstruction-type anomaly detection algorithms that overgeneralize the reconstruction of anomalies.

[0074] This module progressively fuses features from different levels of the encoder in a bottom-up approach, suppressing anomalous information during the fusion process to generate richer and cleaner feature embeddings for subsequent decoding. This design offers two advantages: firstly, the information interaction between multi-level features promotes the learning of diverse feature representations; secondly, MFBE uses a bottleneck structure as its basic unit, enabling it to filter redundant information and anomalous interference during the fusion process. Unlike methods that fuse features first and then process them uniformly, this module selectively retains features at each layer during fusion, effectively suppressing the propagation of anomalous information and resulting in more robust, anomalous feature representations.

[0075] The student decoder can specifically include multiple upsampling layers and multiple residual modules. Specifically, the first residual module receives the output of the MFBE module and performs residual processing on it. The residual processing includes two 3×3 convolutions, batch normalization, and ReLU activation. The input and output are then added through skip connections to obtain the first residual feature. The second upsampling layer receives the first residual feature output from the first residual module and performs upsampling processing on it, increasing the spatial resolution from 8×8 to 16×16 to obtain the second upsampling feature. The second residual module receives the second upsampling feature output from the second upsampling layer, performs residual processing on it, and reduces the number of feature channels from 512 to 256 to obtain the second residual feature. The third upsampling layer receives the second residual feature output from the second residual module, upsamples it to increase the spatial resolution from 16×16 to 32×32, and obtains the third upsampling feature. The third residual module receives the third upsampling feature output from the third upsampling layer, performs residual processing on it, and reduces the number of feature channels from 256 to 128, and obtains the third residual feature. The fourth upsampling layer receives the third residual feature output from the third residual module, upsamples it to increase the spatial resolution from 32×32 to 64×64, and reduces the number of feature channels from 128 to 64, and obtains the fourth upsampling feature. The output convolutional layer receives the fourth upsampling feature output from the fourth upsampling layer and obtains the fourth residual feature through the residual module. The second, third, and fourth residual features constitute the reconstructed feature set.

[0076] In this application, the embedded features output by the MFBE module are then fed into subsequent network layers of the student decoder for decoding and reconstruction. The student decoder restores the embedded features to the original image resolution through progressive upsampling and convolution operations, ultimately outputting a reconstructed feature set with the same resolution as the teacher encoder. Thus, through its internally integrated MFBE module, the student decoder achieves progressive fusion of multi-layer features from the encoder and suppression of anomalous information, thereby providing clean and rich feature embeddings for subsequent decoding and reconstruction.

[0077] Step 104: Calculate the first similarity feature between the features of the reconstructed feature set and the features of the multi-level first feature set, and the second similarity feature between the features of the reconstructed feature set and the features of the anomaly enhancement feature set.

[0078] Figure 4 The process of a strong anomaly selection strategy is illustrated, using three-layer multi-scale features as the computational basis to reconstruct student features (shown as F in the figure). S ) respectively with the first feature set of multiple layers, the second feature set of multiple layers, and (illustrated as F) TE F TD Perform normalization and element-wise multiplication to generate two sets of multi-level similarity features X. E and X D Calculate the corresponding distance map M E and M D Then compare M pixel by pixel E and M D The set of features with stronger abnormal responses is selected as the output.

[0079] Specifically, firstly, based on the correspondence between the student-reconstructed feature set output by the student network and the multi-level first feature set extracted by the teacher encoder from the image to be detected, a first set of multi-level similarity features is constructed. Based on the correspondence between the student reconstruction feature set output by the student network and the abnormal enhancement feature set output by the teacher decoder, a second set of multi-level similarity features is constructed. .

[0080] Calculate the first similarity feature between the student reconstructed feature set and features at the same level in the first feature set, and the second similarity feature between the reconstructed feature set and features at the same level in the anomaly enhancement feature set. Summate the first similarity features along the channel dimension and then invert the result to obtain the first distance map. Summate the second similarity features along the channel dimension and then invert the result to obtain the second distance map. Compare the first and second distance maps pixel by pixel. If the first distance map is greater than the second distance map, retain the first similarity feature; otherwise, retain the second similarity feature. Output the filtered first similarity feature / second similarity feature as the target similarity feature.

[0081] , The details are as follows: ; ; in, This represents the first group of multi-level similarity features. This represents the second group of multi-level similarity features. Let represent the similarity feature (a three-dimensional tensor) between the reconstructed features of the k-th student and the first feature set of multiple levels. This represents the similarity feature between the student reconstruction feature set and the abnormal enhancement feature set at layer k. `sim()` represents the similarity calculation function, which performs L2 normalization on the two input features and then multiplies them element-wise. `K` is the total number of layers, and `k` is the feature at layer k. Represents the reconstructed feature set. This represents the k-th layer features extracted from the image to be detected by the first encoder. This represents the abnormal enhancement features output by the first decoder of the image to be detected.

[0082] By aggregating the channel dimensions, the corresponding distance map is further calculated. and This is used to measure the degree of anomaly at each spatial location. The larger the distance value, the more obvious the characteristic difference at that location, and the higher the probability of its potential anomaly. and The calculation formula is as follows: ; ; in, This represents the first group of multi-level distance maps, by The aggregation yields a measure of the difference between the features reconstructed by students at each spatial location and the features of the first feature set at multiple levels. This represents the second group of multi-level distance maps, by The aggregation yields a measure of the difference between the features reconstructed by students at each spatial location and the features of the first feature set at multiple levels. Let c represent the total number of channels in the k-th layer feature map, where c = 1, 2, ... , This represents the first distance map of the k-th layer (based on the teacher encoder). A larger value indicates a higher probability of anomalies. Representing similarity features The value on the c-th channel, This represents the second distance map of the k-th layer (based on the teacher decoder). Representing similarity features The value on the c-th channel.

[0083] Step 105: Filter target similarity features based on the first similarity feature and the second similarity feature.

[0084] At each feature layer, the two sets of distance maps are compared pixel-by-pixel, and a binary selection mask is constructed using an indicator function: if the anomalous response at a certain location is stronger in one set of representations, the corresponding similarity feature is retained; otherwise, the other set of features is selected. The calculation formula is expressed as follows: ; Where X represents the filtered multi-level similarity features, compared pixel by pixel. and ,reserve or A set of characteristics that indicate a stronger abnormal response. This indicates that the first distance map is greater than the second distance map. At that time, it was believed The abnormal response is stronger, and its corresponding similarity features are preserved. This indicates that the first distance map is smaller than the second distance map. At that time, it was believed The abnormal response is stronger, and its corresponding similarity features are preserved. This represents the similarity feature after the k-th layer of filtering. This represents an exponential function, where the condition within the parentheses is true and the value is 1, otherwise it is 0.

[0085] Through this position-by-position adaptive filtering mechanism, the fused feature representation X is finally obtained. This representation highlights the information with the strongest anomaly indication capabilities, thereby reducing missed detections and improving the accuracy and stability of subsequent anomaly segmentation.

[0086] Step 106: Perform cross-attention fusion processing on the multi-level first feature set and the target similarity feature to obtain the enhanced similarity feature.

[0087] The feature maps of each level in the multi-level first feature set are upsampled to the same spatial resolution and then concatenated to obtain the teacher guidance feature. The feature maps of each level in the target similarity feature map are upsampled to the same spatial resolution and then concatenated to obtain the similarity fusion feature. The teacher guidance feature and the similarity fusion feature are divided into several image blocks, and each image block is linearly mapped to obtain the teacher embedding vector and the similarity embedding vector. The query vector and the key vector are generated from the teacher embedding vector, and the value vector is generated from the similarity embedding vector. The attention weight is calculated based on the query vector and the key vector, and the value vector is weighted and fused using the attention weight. This weight is then added to the similarity embedding vector to obtain the global guidance feature. The global guidance feature is restored to the same spatial resolution as the similarity fusion feature and adaptively fused with the similarity fusion feature to obtain the enhanced similarity feature.

[0088] like Figure 5As shown, for segmentation networks to achieve accurate localization of abnormal regions in complex industrial scenarios, it is necessary to effectively fuse and discriminate abnormal information under the condition that the multi-level feature representations of teachers and students are not entirely reliable. To this end, this application introduces the proposed Teacher-perception Guided Segmentation Network (TGSN) into the knowledge distillation-based image anomaly detection method, enabling it to fuse and optimize the multi-level similarity features generated by the teacher-student network. Figure 5 The complete structure of the segmentation network is illustrated exemplarily. The improved segmentation network includes three key components: a Strong Anomaly Selection (SAS) component, used to filter multi-level similarity features and retain features with stronger anomaly responses; a Teacher-Perception Guided Mechanism (TGM) component, used to guide the global fusion of similarity features using the relational distribution of pre-trained features; and a segmentation head, used to receive the enhanced features and generate a pixel-level anomaly probability map. The segmentation network input is fed in parallel with F... TE F TD F S Three hierarchical feature maps are first fed into the SAS module for feature optimization; the resulting target similarity features are then compared with the original F... TE The data are fed into the TGM teacher perception guidance module, where cross-attention feature enhancement is achieved using the global semantics of teacher features. After enhancement, the features undergo ASPP multi-scale feature extraction and multi-layer residual block deepening of feature representation before being fed into the SegHead segmentation head. The segmentation head restores the original image resolution through multi-scale convolutional fusion and feature upsampling, generates a pixel-level anomaly probability map through an activation function, and calculates the anomaly score for each image based on the probability map. This segmentation network overcomes the single reliance on teacher-student similarity, achieving refined anomaly localization through autonomous feature discrimination. Based on the above components, the segmentation network of this application not only fully utilizes the multi-scale information provided by the teacher-student network but also possesses a certain degree of autonomous discrimination capability, thus outputting stable and accurate anomaly segmentation results even when faced with unreliable feature inputs, ultimately achieving refined localization of anomaly regions. The following is a detailed description of the key components: The strong anomaly selection strategy component of this application can be a computation module used to capture anomaly region information more comprehensively. The core idea of ​​this component is to prioritize retaining the location features with more significant anomaly responses among multi-layer similarity features, thereby enhancing the expressive power of anomaly regions.

[0089] like Figure 6As shown, this application proposes a Teacher-perception Guided Mechanism (TGM), which uses the inter-regional relationship distribution information contained in pre-trained features to guide the fusion process of teacher-student similarity features, thereby achieving global context modeling. Figure 6 The process of the teacher perception guidance mechanism is illustrated by example: the teacher guidance features and similarity fusion features are embedded in blocks respectively, Q and K are generated from the teacher embedding, V is generated from the similarity embedding, and the global guidance features are obtained by cross-attention calculation.

[0090] Specifically, the teacher perception guidance mechanism components may include linear mapping layers, MLP, convolutional layers, etc. In the specific implementation, the multi-level first features output by the teacher encoder are first upsampled to the same spatial resolution (e.g., 64×64), and then concatenated along the channel dimension to obtain the teacher guidance features. At least one target similarity feature obtained from the screening is upsampled to the same spatial resolution (e.g., 64×64), and then concatenated along the channel dimension to obtain the similarity fusion feature. ,Will and These two sets of features are input into the TGM for processing. Within the module, the two sets of input features are first segmented and embedded to obtain corresponding patch-level representations; specifically, ... The data is divided into several non-overlapping patches (e.g., each patch is 8×8 in size). The pixel values ​​within each patch are flattened into a one-dimensional vector, and then mapped to a fixed-dimensional embedding vector through a linear mapping (fully connected layer). The embedding vectors of all patches constitute the teacher patch embedding P, which has a size of N×d, where N is the number of patches and d is the embedding dimension. Similarly, ... Perform the same block embedding operation to obtain the similarity patch embedding S, which is of size N×d and serves as the starting point for the residual join. Generate the query vector Q(P) and key vector K(P) from the teacher patch embedding P, and generate the value vector V(S) from the similarity patch embedding S. Here, Q(), K(), and V() are learnable linear mapping functions. That is, from... Generate query and key vectors from Numerical vectors are generated, and attention weights are obtained by calculating the similarity between the query and the key, thereby modeling the dependencies between different regions. Based on this, the numerical vectors are weighted and fused using the attention weights to achieve global guidance and enhancement of similarity features. The calculation formula is as follows: ; in, Let S represent the patch-level representation of the similarity fusion features, S+ represent the residual connection (adding the original S to the attention output), Softmax represent the ratio normalization exponential function that maps values ​​to [0,1] and sums to 1, and P is... The patch-level representation is as follows: Upsample each sample to the same spatial resolution, then concatenate them along the channel dimension to obtain... ,Will The algorithm is divided into several patches (e.g., each patch is 8×8 in size). The pixel values ​​within each patch are flattened into a one-dimensional vector, and then mapped to a fixed-dimensional embedding vector through a linear mapping (fully connected layer). The embedding vectors of all patches together constitute P, where Q(P) represents the query vector corresponding to P, K(P) represents the key vector corresponding to P, and T represents the matrix transpose. Indicates the scaling factor. Let V(S) be the dimension of the key vector, and let V(S) represent the value vector generated from the similarity feature S.

[0091] The attention-fused features are refined using a multilayer perceptron (MLP) and convolutional operations to further enhance their expressive power. The refined features are then restored to the fused features with the original similarity. The same spatial resolution (e.g., C×64×64) is then used to fuse features with the original similarity. Perform adaptive fusion (e.g., weighted summation by learning weights) to output enhanced similarity features. Its dimensions are C×64×64.

[0092] Step 107: Process the enhanced similarity features using the segmentation head to generate an anomaly probability map, and calculate the image anomaly score of the image to be detected based on the anomaly probability map.

[0093] The segmentation head extracts features from the enhanced similarity features to obtain the first feature map; the first feature map is then subjected to multi-scale convolution and the multi-scale outputs are concatenated along the channel dimension to obtain the second feature map; the second feature map is then fused and dimensionality reduced to obtain the third feature map; the third feature map is restored to the spatial resolution of the input image to obtain the fourth feature map; the value of each pixel in the fourth feature map is converted into a probability value to generate a pixel-level anomaly probability map; a preset number of pixels with the largest pixel values ​​are selected from the pixel-level anomaly probability map, and the average value of the preset number of pixels is calculated as the image anomaly score of the image to be detected.

[0094] Enhanced similarity features The input segmentation head's residual module is processed. This module can consist of two 3×3 convolutional layers, batch normalization (BN), and a ReLU activation function. The input and output are added together via skip connections, maintaining the output feature size at C×64×64. Multi-scale dilated convolutions capture contextual information from different receptive fields. Specifically, three parallel dilated convolutions are used, each with a 3×3 kernel and dilation rates of 1, 2, and 4, respectively. The outputs of the three convolutions are concatenated along the channel dimension to obtain a 3C×64×64 feature. A 1×1 convolution is then used to fuse and reduce the dimensionality of the multi-scale features, resulting in an output feature size of C′×64×64, where C′ is the number of channels after dimensionality reduction (e.g., 128). An upsampling operation (e.g., bilinear interpolation) restores the feature space resolution to the input image resolution of 256×256, resulting in an output feature size of C′×256×256. The number of channels is reduced to 1 through 1×1 convolution, and then the Sigmoid activation function is used to map the value of each pixel to the [0,1] interval, finally generating a pixel-level anomaly probability map M. seg Its dimensions are 1×256×256. Each pixel value in this probability map represents the probability that the location is predicted to be an anomaly.

[0095] Considering that anomalies in industrial scenarios are usually only distributed in local areas, directly averaging the entire image could easily diminish the importance of high-response areas. Therefore, a Top-K statistical strategy is adopted, selecting the top 100 pixels with the highest response values ​​from the anomaly map and averaging them to obtain the final image-level anomaly score.

[0096] Specifically, the image-level anomaly score is calculated based on the pixel-level anomaly probability map. Specifically, the pixel-level anomaly probability map is flattened into a one-dimensional array, sorted in descending order of pixel values, and the first Z pixel values ​​are selected. The average of the first Z pixel values ​​is calculated as the image-level anomaly score, where Z is a preset positive integer, preferably 100.

[0097] The formula for calculating image-level anomaly scores is: ; Where score is the image-level anomaly score, M seg This is a pixel-level anomaly probability map. This indicates that the top Z pixels with the highest response values ​​are selected from the anomaly probability map, where Z is the preset number of pixels, for example, 100, and β is the anomaly probability value of the selected pixels.

[0098] In one embodiment, the first encoder is pre-trained and its parameters are frozen. The method further includes training the first decoder: acquiring a first training sample set, which includes normal images and pseudo-anomaly images; inputting the normal images and pseudo-anomaly images in the first training sample set into the first encoder to extract multi-level normal feature maps and multi-level pseudo-anomaly feature maps, and calculating the difference between each level of normal feature map and pseudo-anomaly feature map as a perturbation term; performing weighted fusion of the perturbation term and the downsampled pseudo-anomaly mask to construct a supervision signal; inputting the pseudo-anomaly images in the first training sample set into the first decoder, calculating the cosine distance loss between the output features of the first decoder and the supervision signal, and updating the parameters of the first decoder according to the cosine distance loss.

[0099] This application also includes training a teacher network. During the training phase, the teacher network employs a strategy of freezing the encoder and optimizing only the decoder. Pre-training is performed before knowledge distillation by constructing discriminative residual supervision signals to enhance the teacher network's ability to perceive anomalies. During the training phase, the difference between normal features and pseudo-anomaly features is first calculated as a perturbation term. Then, a supervision signal is constructed by combining it with a pseudo-anomaly mask. Finally, the teacher decoder parameters are optimized using cosine distance loss. Figure 7 The discriminative residual distillation process of the teacher encoder is illustrated. During the training phase, normal images and synthetic pseudo-anomaly images are input separately. The normal image is processed by the teacher encoder to extract baseline features (shown as F in the figure). TE The pseudo-anomaly image is propagated forward with the encoder to obtain features, as illustrated in the figure. S After extracting features from the pseudo-anomaly image, the difference between these features and the baseline features extracted from the normal image is calculated to obtain the perturbation term. This perturbation term is then calculated by combining the pseudo-anomaly mask to divide the normal and anomalous regions. The normal regions retain their original features, while the anomalous regions have the perturbation term superimposed, collectively constructing the training supervision label for the decoder. The pseudo-anomaly features are fed into the first decoder for decoding operations, outputting discriminative residual features (shown as F in the diagram). TD The feature distance between the discriminative residual features and the pre-constructed supervision labels is calculated using the feature distance function, i.e., the θ() function shown in the figure. Based on this distance result, a cosine distance loss function is constructed, and the parameters of the first decoder are iteratively optimized by backpropagation of the loss. After training under this loss constraint, the teacher decoder has the ability to strengthen the representation of abnormal perturbations, solving the problem of insufficient sensitivity to anomalies in traditional teacher networks.

[0100] Specifically, a first training sample set is obtained, which includes normal images and pseudo-anomaly images. Pseudo-anomaly images can be generated by adding anomalies to normal images, such as pasting image patches at random locations in a normal image, adding texture noise, or using a generative model to generate pseudo-anomaly regions, while simultaneously generating corresponding anomaly masks. The anomaly masks are used to label the locations of pseudo-anomalies and can be represented by 0 or 1, where 1 represents an anomaly region and 0 represents a normal region.

[0101] Normal images and pseudo-anomaly images from the first training sample set are input into the first encoder. The encoder parameters are frozen during training. Through forward propagation of the encoder, multi-level normal feature maps corresponding to normal images and multi-level pseudo-anomaly feature maps corresponding to pseudo-anomaly images are extracted. For example, the multi-level features corresponding to normal images are represented as follows: Where k = 1, 2, or 3, corresponding to the shallow, middle, and highest layers, respectively. The multi-layer feature representation corresponding to the pseudo-anomaly image is as follows: , where k=1 or 2 or 3, corresponding to the shallow layer, middle layer and the highest layer, respectively.

[0102] For each level k, the difference between the pseudo-anomaly features and the normal features is calculated as a perturbation representation. The perturbation representation reflects the direction of feature change in the anomaly region relative to the normal region, as detailed below: ; in, Let K represent the set of perturbations, containing K levels of perturbation features, where K is the total number of multi-level features. The perturbation representation of layer k is the characteristic change of the abnormal region in layer k relative to the normal region. This represents the k-th layer output of the teacher encoder under pseudo-abnormal input. This represents the k-th layer output of the teacher encoder under normal input.

[0103] The real anomaly mask M gt The spatial resolution of the downsampled feature map to the k-th layer is obtained. Then, a supervisory signal is constructed through weighted fusion. : ; ; in, This represents the set of monitoring signals, containing three levels of monitoring signals. The feature map represents the supervision signal of the k-th layer, constructed by the function f, where f() represents the weighted fusion function, and M gt This represents the true anomaly mask, with the same resolution as the input image; 1 indicates an anomaly, and 0 indicates normal. This indicates that the downsampled anomaly mask is downsampled to the k-th layer feature map resolution H. k ×W k ,1- ⊙ represents the normal region mask, and ⊙ represents the Hadamard product (element-by-element multiplication). Indicates contribution from the normal region. This represents the contribution from the abnormal region, that is, in the normal region, i.e., M. gt At the position of =0, normal features are preserved. In the abnormal region, i.e., M gt The position where =1 is represented using a perturbation. By using the above element-wise weighted fusion, the contributions of normal and abnormal regions are merged to construct a target feature representation for supervising the training of the teacher decoder.

[0104] Then, the pseudo-anomaly image features Input the teacher decoder to generate discriminative residual features. The parameters of the teacher decoder are trainable during training. It should be noted that the multi-scale features output by the first decoder serve as discriminative residual features, acting as the supervisory target for knowledge distillation to guide the learning of the student network. These discriminative residual features strip normal components from anomalous features, extracting information representing anomalous perturbations, and exhibit a more significant difference compared to the normal features reconstructed by the student network. For each spatial location (i,j) and each level k, the supervisory signal is calculated. With discriminative residual characteristics Cosine distance between them: ; in, The value of is in the range of {0,2}. The smaller the value, the more similar the features are.

[0105] The total loss of the teacher network is obtained by averaging the cosine distances over all levels and all spatial locations, as shown below: ; in, This represents the total loss of the teacher network. This represents the height of the feature map at layer k. This represents the width of the feature map at layer k. Represents all spatial locations in the k-th layer. The average value, This indicates that the loss is accumulated across K levels.

[0106] Based on the calculated total loss, the parameters of the teacher decoder are backpropagated and updated, while the parameters of the teacher encoder remain frozen. The optimization objective is to minimize the total loss, enabling the decoder to extract discriminative residuals in abnormal regions and recover non-abnormal features in normal regions. The teacher network can learn more sensitive and discriminative abnormal representations, effectively reducing the false negative rate and improving the overall detection performance.

[0107] In the teacher decoder, it can determine whether the output features of each residual module are consistent with the number of channels in the corresponding layer of the student network. If they are consistent, the output features of each residual module are transformed through the output convolutional layer to generate the final discriminative residual features. The output convolutional layer is a 3×3 convolutional layer (including batch normalization and ReLU activation), with the same 64 input and output channels. This convolutional layer does not change the spatial size and number of channels of the feature map, but maps the features to a more discriminative feature space through learnable convolutional kernels. If they are inconsistent, the output features of each residual module are adjusted through the output convolutional layer to generate discriminative residual features with a channel number matching the student network. The output convolutional layer can be an independent 3×3 convolutional layer (keeping the number of channels unchanged) or a 1×1 convolutional layer (used to adjust the number of channels to the specific dimension required for the distillation task).

[0108] Understandably, this assumes the teacher decoder has a uniform output channel count of 64, while the supervisory signal... The number of channels varies with the layer (e.g., 64, 128, 256). Before calculating the cosine distance, both may need to be processed by a 1×1 convolution. The number of channels is adjusted to 64 to match the number of output channels of the teacher decoder, or it can be kept as is and calculated by channel dimension. The specific adjustment can be made according to the actual implementation.

[0109] Because perturbations have complex and irregular distributions, direct modeling them presents certain challenges. Therefore, drawing on the idea of ​​skip connections, feature information from different levels of the encoder is introduced during the decoding process. Specifically, after mapping the anomalous features of the lower and middle layers using 1×1 convolutions, these features are concatenated with the input of the corresponding decoding layer. This shifts the learning objective from directly recovering the perturbation to modeling residual information in the latent space, effectively improving the stability and expressive power of the distillation process. Furthermore, to avoid insufficient response to anomalous features from a single level, multi-layer encoded features are uniformly adjusted to the same resolution before convolutional fusion, enhancing the overall representation of the anomalous signal.

[0110] At this point, the teacher decoder has completed the transformation from multi-scale encoder features to multi-scale discriminative residual features. The output multi-scale discriminative residual features will serve as the supervision target in knowledge distillation to guide the learning of student networks.

[0111] In one embodiment, the method further includes training a second encoder and a second decoder: acquiring a second training sample set, which includes normal images; constructing corresponding pseudo-abnormal images and pseudo-abnormal masks for each normal image in the second training sample set; inputting the normal images in the second training sample set into a first encoder to extract target multi-level normal feature maps; inputting the pseudo-abnormal images into a second encoder, which then performs embedding fusion processing and reconstruction with the second decoder to obtain a target reconstructed feature set; calculating the target cosine distance loss between the target reconstructed feature set and the target multi-level normal feature maps, and updating the parameters of the second encoder and the second decoder based on the target cosine distance loss.

[0112] In this embodiment, the training and inference phases share the same network structure (second encoder, second decoder, MFBE module). The difference lies in that the training phase requires calculating the loss and updating the parameters, while the inference phase only performs forward operations, and the network parameters remain fixed. The training loss is calculated using cosine distance, which differs from the inference similarity calculation logic described in the inference phase, which uses normalized multiplication and channel summation and inversion to generate the distance map. The two calculation methods have clearly distinct applicable scenarios: cosine distance is used for backpropagation optimization of network parameters during training, while the normalized multiplication and inversion method is used to generate the distance map during the inference phase.

[0113] Obtain a second training sample set, which includes normal images. Construct a corresponding pseudo-anomaly image and pseudo-anomaly mask for each normal image in the second training sample set.

[0114] Normal images from the second training sample set are input into the first encoder to extract multi-level normal feature maps of the target. The pseudo-anomaly image is input into the second encoder, and after embedding fusion processing and reconstruction by the second decoder, the target reconstruction feature set is obtained. Where k = 1, 2, or 3, corresponding to the shallow, middle, and highest layers, respectively.

[0115] For each level k, calculate the cosine distance between the reconstructed features and the normal features: ; in, The value of is in the range of {0,2}. The smaller the value, the more similar the features are.

[0116] In the inference phase, similarity is calculated using element-wise multiplication followed by channel summation and inversion, while in this training phase, cosine distance is used. Both essentially measure the similarity between features, but their forms of expression differ.

[0117] The total loss of the student network is obtained by averaging the cosine distances across all levels and spatial locations. in, The value of is in the range of {0,2}. The smaller the value, the more similar the features are.

[0118] The total loss of the teacher network is obtained by averaging the cosine distances over all levels and all spatial locations, as shown below: ; in, This represents the total loss of students' network access. This represents the height of the feature map at layer k. This represents the width of the feature map at layer k. Represents all spatial locations in the k-th layer. The average value, This indicates that the loss is accumulated across K levels.

[0119] Based on the calculated total loss The parameters of the second encoder and the second decoder are backpropagated and updated so that the reconstructed features output by the second decoder are spatially aligned with the normal features output by the first encoder. This enables the student network to recover a near-normal distribution of features from pseudo-abnormal image inputs.

[0120] In this application, the training phase and the inference phase share the same segmentation head structure. The difference is that the training phase needs to calculate the loss and update the parameters, while the inference phase only involves forward propagation and parameter freezing.

[0121] This application also includes training the segmentation network. During the segmentation network training process, the parameters of the teacher-student network (first encoder, first decoder, second encoder, second decoder) remain frozen, and only the segmentation network is trained. The segmentation network uses L1 loss and focal loss to jointly constrain the prediction results, and the total loss function is: ; in, This represents the total loss of the segmentation network. This represents the L1 loss value. This represents the focus loss value.

[0122] The L1 loss is used to constrain the absolute error between the pixel-level prediction and the true mask. ; in, H represents the L1 loss value, and W represents the height of the input image and the width of the input image. This represents summing over all pixel positions, where i represents the row position of the pixel and j represents the column position of the pixel. This represents the probability of an anomaly predicted by the segmentation head at position (i,j). Represents the true anomaly mask at position (i,j); Focus loss is used to enhance the ability to focus on hard-to-classify samples: ; in, Indicates the focal loss value. p represents the modulation factor, with its value decreasing as p approaches 1. γ represents the focusing parameter, which controls the weights of difficult samples. The logarithm of the accuracy rate, when When the value is close to 1 (correct prediction), ≈0, small loss, when When close to 0 (prediction error), ≈1, the loss is large, the larger γ is, the stronger the focus is on samples that the model has difficulty classifying or predicting correctly.

[0123] ; This represents the accuracy of the segmentation network's prediction at pixel (i,j), i.e., the degree to which the predicted result at that position matches the true mask. This represents the true anomaly mask at pixel (i,j), used to determine whether the location is actually an anomaly. This represents the probability of an anomaly predicted by the segmentation network at pixel (i,j), i.e., the probability that this position is predicted as an anomaly. This indicates the actual normal area, meaning whether the location is actually normal. This indicates the probability that the prediction is normal, that is, the probability that this position is predicted to be normal.

[0124] The training objective of the segmentation network is to minimize the total loss, enabling the segmentation network to approximate the true anomaly distribution at the pixel level (L1 loss constraint), focus on hard-to-classify samples (focus loss constraint), and output stable and accurate anomaly segmentation results when faced with unreliable feature inputs.

[0125] It is understandable that in this application, "unsupervised" refers to the reasoning phase not relying on any anomaly labels. The training phase employs pseudo-anomaly synthesis technology, which belongs to the self-supervised learning paradigm and does not violate the basic premise of unsupervised detection.

[0126] The image anomaly detection method provided in this application will be clearly and completely described below, combining the experimental data, evaluation indicators, experimental setup, experimental results and analysis, and ablation experiments.

[0127] Dataset: This application uses MVTec AD to evaluate the proposed method. This dataset is one of the widely used benchmark datasets in the field of unsupervised anomaly detection, covering 15 industrial categories, including 5 texture types and 10 concrete object types. Regarding data partitioning, the training set consists only of normal samples, containing 3629 images; the test set contains 1725 images, composed of both normal and anomaly samples, used to evaluate the model's performance in anomaly detection and localization tasks.

[0128] Evaluation metrics: In image-level anomaly detection tasks, the area under the receiver operating characteristic curve (AUROC) is used as the primary evaluation metric to measure the model's overall ability to distinguish between normal and abnormal samples. For pixel-level anomaly localization, pixel-level AUROC, average precision (AP), and region overlap index (PRO) are used to comprehensively evaluate model performance, reflecting the model's performance in fine-grained anomaly detection.

[0129] Furthermore, considering that practical applications prioritize the ability to locate the entire region of anomalies rather than the prediction results of individual pixels, Instance-Level Mean Precision (IAP) is introduced as an evaluation metric to assess the model's detection performance at the instance level. Under this definition, an anomaly instance is considered to consist of connected ground truth labeled regions, and an instance is considered successfully detected when more than 50% of the pixels in the model's predictions are identified as anomalies. IAP is calculated by combining pixel-level precision and instance-level recall at different thresholds, thus reflecting the model's overall performance under different detection conditions. For applications requiring high recall, the IAP metric is further employed, which is the precision value corresponding to a recall rate reaching a specified threshold k%. In this experiment, k is set to 90 to evaluate the model's detection capability and stability under high recall conditions.

[0130] Experimental Setup: During the experiment, independent models were trained for each category in the dataset. All input images were uniformly adjusted to a resolution of 256×256 during both training and inference phases. The overall training process consisted of 6000 iterations, with 1000 iterations for the student network, 1000 iterations for the teacher decoder, and 4000 iterations for the segmentation network. A phased optimization strategy was employed to improve the performance of each module. For optimization, independent stochastic gradient descent (SGD) optimizers were used for different sub-networks, with differentiated hyperparameters. Specifically, the learning rate was set to 0.4 for the student network, 0.1 for the teacher decoder, 0.15 for the TGM module, and 0.01 for the segmentation head. The momentum parameter of all optimizers was uniformly set to 0.9, and the weight decay coefficient was 1×10⁻⁴ to ensure the stability and convergence of the training process. All experiments were implemented using the PyTorch 2.4 deep learning framework and completed on a single NVIDIA GeForce RTX 4090 graphics card for both training and testing.

[0131] Experimental Results and Analysis: To verify the effectiveness of the proposed method, several representative anomaly detection algorithms were selected for comparative experiments. These methods cover different technical approaches, including knowledge distillation-based methods, reconstruction-based methods, and memory-based methods. Specifically, DeSTSeg, RD++, and URD were selected as knowledge distillation-based methods; DRAEM and RealNet (DTD) were selected as reconstruction-based methods; and PatchCore was selected as a memory-based method. Through comparative analysis with these mainstream methods, the performance of the proposed method in anomaly detection tasks was comprehensively evaluated. The comparison results between the proposed method and the aforementioned methods in pixel-level, image-level, and instance-level tasks will be reported and analyzed below.

[0132] In the pixel-level anomaly localization task, the proposed method was systematically evaluated on the MVTec AD dataset, and quantitative comparisons were made with several mainstream methods. The results are reported in Table 1. Experimental results show that the proposed method achieves leading performance in all evaluation metrics, with a pixel-level AUROC of 99.2%, an average precision (AP) of 81.5%, and a region overlap index (PRO) of 96.2%. In particular, the overall average AP index is improved by 4.4% compared with the current best-performing method DeSTSeg, demonstrating a significant advantage. Figure 8 The anomaly detection visualization results further validated the effectiveness of the model in locating anomaly regions.

[0133] In image-level anomaly detection, the model performance was also evaluated on the MVTec AD dataset, and the results are shown in Table 2. Experimental results show that the proposed method exhibits strong detection capabilities across multiple categories, with overall performance on par with state-of-the-art methods, demonstrating good stability and generalization ability.

[0134] In the instance-level anomaly detection task, as shown in Table 3 of the detailed evaluation results on the MVTec AD dataset, the method provided in this application achieves an instance-level average precision (IAP) of 79.8% and an IAP@90 score of 65.9%, both outperforming other comparative methods. These results demonstrate that the proposed method possesses excellent detection capabilities under different recall requirements, effectively meeting the needs of anomaly instance identification in industrial scenarios, and is particularly significant in reducing the risk of missed detections.

[0135] Table 1 shows the AUC / AP / PRO (%) results for pixel-level anomaly localization on MVTec AD, with the best results shown in bold.

[0136] Table 2 shows the AUC (%) results for image-level anomaly detection on MVTec AD. The results are the average of all categories, and the best results are shown in bold.

[0137] Table 3 shows the instance-level anomaly detection IAP / IAP@90 (%) results on MVTec AD, with the best results shown in bold.

[0138] Ablation experiment analysis: To verify the effectiveness of each key module in the proposed method, this application conducted systematic ablation experiments on the MVTec AD dataset and carried out in-depth analysis of the teacher-student model structure and segmentation network components.

[0139] Regarding the teacher-student model architecture, the ablation study results are shown in Table 4. By comparing the performance of different teacher model designs, it can be found that the proposed Discriminative Residual Distillation Teacher (DRDT) has a significant advantage over the Baseline Teacher model (Base-T), mainly due to its stronger ability to capture anomalous features. Building on this, further combining DRDT with a student network incorporating a Multi-Level Feature Bottleneck Embedding Module (MFBE) further improves model performance. This improvement is primarily reflected in its ability to more effectively suppress false detections, thereby enhancing overall detection accuracy. Figure 9 and Figure 10The visualization results show that after introducing the MFBE module and DRDT, the false positives and false negatives generated by the model are significantly reduced, which indirectly verifies the effectiveness of the two in optimizing feature reconstruction and comprehensively locating anomalies.

[0140] In the segmentation network section, the ablation study results are reported in Table 5. A comparison between the improved segmentation network TGSN (w / TGSN) and the method of directly using teacher-student cosine distance maps for fusion (w / o TGSN) shows that the segmentation network significantly improved performance across all tasks. Further ablation analysis of the various modules within TGSN revealed that removing the teacher-perceived guidance mechanism (-TGM) significantly reduced the model's performance in anomaly localization and instance-level detection tasks; while removing the strong anomaly selection strategy (-SAS) led to an overall degradation of all evaluation metrics. These experimental results fully demonstrate the crucial role of each module in improving model performance and validate the rationality and effectiveness of the overall method design.

[0141] Table 4 shows the ablation results of the teacher-student model (teacher-student network) components.

[0142] Table 5 shows the ablation results of the segmentation network components.

[0143] It should be understood that, although Figure 1 , Figure 4 , Figure 6 , Figure 7 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 , Figure 4 , Figure 6 , Figure 7 At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0144] In one embodiment, such as Figure 11 As shown, an image anomaly detection device is provided, comprising: a determination module 20 and a generation module 21, wherein: The determination module 20 is used to acquire the image to be detected, and input the image to be detected into the first encoder and the second encoder respectively to extract multi-level first feature sets and multi-level second feature sets; input the multi-level first feature sets into the first decoder for anomaly enhancement decoding processing to obtain anomaly enhancement feature sets; perform embedding and fusion processing on the multi-level second feature sets to generate embedded features, and input the embedded features into the second decoder for reconstruction to obtain reconstructed feature sets; The generation module 21 is used to calculate the first similarity feature between the features of the reconstructed feature set and the features of the multi-level first feature set, and the second similarity feature between the features of the reconstructed feature set and the features of the anomaly enhancement feature set; to filter the target similarity feature based on the first similarity feature and the second similarity feature; to perform cross-attention fusion processing on the multi-level first feature set and the target similarity feature to obtain the enhanced similarity feature; to process the enhanced similarity feature using the segmentation head to generate an anomaly probability map, and to calculate the image anomaly score of the image to be detected based on the anomaly probability map.

[0145] Specific limitations regarding the image anomaly detection device can be found in the limitations of the image anomaly detection method described above, and will not be repeated here. Each module in the aforementioned image anomaly detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0146] In one embodiment, this application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the image anomaly detection methods provided by the above methods.

[0147] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data used in the image anomaly detection method. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements an image anomaly detection method.

[0148] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Step 101: Obtain the image to be detected, and input the image to be detected into the first encoder and the second encoder respectively to extract the multi-level first feature set and the multi-level second feature set.

[0150] Step 102: Input the multi-level first feature set into the first decoder for anomaly enhancement decoding processing to obtain the anomaly enhancement feature set.

[0151] Step 103: The multi-level second feature set is embedded and fused to generate embedded features. The embedded features are then input into the second decoder for reconstruction to obtain the reconstructed feature set.

[0152] Step 104: Calculate the first similarity feature between the features of the reconstructed feature set and the features of the first feature set of the multi-level system, and the second similarity feature between the features of the reconstructed feature set and the features of the anomaly enhancement feature set.

[0153] Step 105: Filter target similarity features based on the first similarity feature and the second similarity feature.

[0154] Step 106: Perform cross-attention fusion processing on the multi-level first feature set and the target similarity feature to obtain the enhanced similarity feature.

[0155] Step 107: Process the enhanced similarity features using the segmentation head to generate an anomaly probability map, and calculate the image anomaly score of the image to be detected based on the anomaly probability map.

[0156] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Step 101: Obtain the image to be detected, and input the image to be detected into the first encoder and the second encoder respectively to extract the multi-level first feature set and the multi-level second feature set.

[0157] Step 102: Input the multi-level first feature set into the first decoder for anomaly enhancement decoding processing to obtain the anomaly enhancement feature set.

[0158] Step 103: The multi-level second feature set is embedded and fused to generate embedded features. The embedded features are then input into the second decoder for reconstruction to obtain the reconstructed feature set.

[0159] Step 104: Calculate the first similarity feature between the features of the reconstructed feature set and the features of the first feature set of the multi-level system, and the second similarity feature between the features of the reconstructed feature set and the features of the anomaly enhancement feature set.

[0160] Step 105: Filter target similarity features based on the first similarity feature and the second similarity feature.

[0161] Step 106: Perform cross-attention fusion processing on the multi-level first feature set and the target similarity feature to obtain the enhanced similarity feature.

[0162] Step 107: Process the enhanced similarity features using the segmentation head to generate an anomaly probability map, and calculate the image anomaly score of the image to be detected based on the anomaly probability map.

[0163] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image anomaly detection method, characterized in that, include: The image to be detected is acquired and input into the first encoder and the second encoder respectively to extract multi-level first feature sets and multi-level second feature sets; The multi-level first feature set is input into the first decoder for anomaly enhancement decoding processing to obtain the anomaly enhancement feature set; The multi-level second feature set is embedded and fused to generate embedded features. The embedded features are then input into the second decoder for reconstruction to obtain the reconstructed feature set. Calculate a first similarity feature between the features of the reconstructed feature set and the features of the multi-level first feature set, and a second similarity feature between the features of the reconstructed feature set and the features of the anomaly enhancement feature set; Target similarity features are selected based on the first similarity feature and the second similarity feature; The multi-level first feature set and the target similarity feature are subjected to cross-attention fusion processing to obtain the enhanced similarity feature; The enhanced similarity features are processed using a segmentation head to generate an anomaly probability map, and the image anomaly score of the image to be detected is calculated based on the anomaly probability map.

2. The method according to claim 1, characterized in that, The first encoder and the second encoder each include multiple residual modules connected in sequence. The image to be detected is input into the first encoder and the second encoder respectively to extract multi-level first feature sets and multi-level second feature sets, including: The image to be detected is input into the first encoder. According to the connection order of each residual module of the first encoder, the image to be detected or the feature map output by the residual module of the previous order is convolved and downsampled in turn. Multiple feature maps of different levels with progressively decreasing resolution are output as a multi-level first feature set. The image to be detected is input into the second encoder. According to the connection order of each residual module of the second encoder, the image to be detected or the feature map output by the residual module of the previous sequence is convolved and downsampled in turn, and multiple feature maps of different levels with progressively decreasing resolution are output as a multi-level second feature set.

3. The method according to claim 1, characterized in that, The step of inputting the multi-level first feature set into the first decoder for anomaly enhancement decoding processing to obtain the anomaly enhancement feature set includes: The multi-level first feature set is input to the first decoder, and the convolutional layer in the first decoder performs channel adjustment on each level of the first feature set. According to the reverse order of the connection order of each residual module in the first decoder, each residual module upsamples the feature map with the lowest resolution in the first feature set of the multi-level layers or the feature map output by the residual module in the previous order. The upsampled result is then concatenated with the feature map in the first feature set after channel adjustment corresponding to each residual module level. The concatenated result is then input into the convolutional layer for residual learning, and the decoding features of each residual module level are output. The decoding features output by the last residual module are used as the anomaly enhancement feature set.

4. The method according to claim 1, characterized in that, The process of embedding and fusing the multi-level second feature set to generate embedded features, and inputting the embedded features into the second decoder for reconstruction, to obtain the reconstructed feature set includes: The multi-level second feature set is input to the multi-level feature bottleneck embedding module. The multi-level feature bottleneck embedding module fuses the feature maps of each level from bottom to top through multiple cascaded bottleneck fusion units, and suppresses abnormal information during each fusion process to generate embedded features. The embedded features are input to a second decoder, which performs progressive upsampling and residual processing on the embedded features through multiple upsampling layers and multiple residual modules, and outputs a reconstructed feature set.

5. The method according to claim 1, characterized in that, The calculation of a first similarity feature between the reconstructed feature set and features in the first feature set, and a second similarity feature between the reconstructed feature set and features in the anomaly enhancement feature set, and the selection of target similarity features based on the first similarity feature and the second similarity feature, includes: After normalizing the reconstructed feature set and the feature maps of the same level in the first multi-level feature set, they are multiplied element by element to obtain the first set of multi-level similarity features. The first distance map is obtained by summing the first set of multi-level similarity features along the channel dimension and then inverting the sum. After normalizing the feature maps of the same level in the reconstructed feature set and the abnormal enhancement feature set, they are multiplied element by element to obtain the second set of multi-level similarity features. The second set of multi-level similarity features is summed along the channel dimension and then inverted to obtain the second distance map. The first distance map and the second distance map are compared pixel by pixel. If the value of the first distance map is greater than the value of the second distance map, the feature vector in the first group of multi-level similarity features for that pixel is selected as the target similarity feature; if the value of the first distance map is less than the value of the second distance map, the feature vector in the second group of multi-level similarity features is selected as the target similarity feature. Arrange the selected feature vectors from all pixels according to the pixels to form a target similarity feature map and output it.

6. The method according to claim 1, characterized in that, The step of performing cross-attention fusion between the multi-level first feature set and the target similarity feature to obtain the enhanced similarity feature includes: After upsampling the feature maps of each level in the multi-level first feature set to the same spatial resolution, channel stitching is performed to obtain the teacher guidance features. After upsampling the feature maps of each level in the target similarity feature map to the same spatial resolution, the channels are stitched together to obtain the similarity fusion feature. The teacher guidance feature and the similarity fusion feature are divided into several image blocks, and each image block is linearly mapped to obtain the teacher embedding vector and the similarity embedding vector. Generate a query vector and a key vector from the teacher embedding vector, and generate a value vector from the similarity embedding vector; The attention weights are calculated based on the query vector and the key vector. The value vector is then weighted and fused using the attention weights and added to the similarity embedding vector to obtain the global guidance feature. The global guiding feature is restored to the same spatial resolution as the similarity fusion feature, and then adaptively fused with the similarity fusion feature to obtain the enhanced similarity feature.

7. The method according to claim 1, characterized in that, The step of processing the enhanced similarity features using a segmentation head to generate an anomaly probability map, and calculating the image anomaly score of the image to be detected based on the anomaly probability map includes: The segmentation head extracts features from the enhanced similarity features to obtain a first feature map; The first feature map is subjected to multi-scale convolution, and the multi-scale outputs are concatenated along the channel dimension to obtain the second feature map. The second feature map is fused and its dimensionality reduced to obtain the third feature map; The third feature map is restored to the spatial resolution of the input image to obtain the fourth feature map; The value of each pixel in the fourth feature map is converted into a probability value to generate a pixel-level anomaly probability map. A preset number of pixels with the largest pixel values ​​are selected from the pixel-level anomaly probability map, and the average value of the preset number of pixels is calculated as the image anomaly score of the image to be detected.

8. The method according to claim 1, characterized in that, The first encoder is pre-trained and its parameters are frozen, and the method further includes training the first decoder: Obtain a first training sample set, which includes normal images and pseudo-abnormal images; Normal images and pseudo-abnormal images from the first training sample set are input into the first encoder to extract multi-level normal feature maps and multi-level pseudo-abnormal feature maps. The difference between each level of normal feature map and pseudo-abnormal feature map is calculated as a perturbation term. The perturbation term is weighted and fused with the downsampled pseudo-anomaly mask to construct a monitoring signal; The pseudo-anomaly images in the first training sample set are input into the first decoder. The cosine distance loss between the output features of the first decoder and the supervision signal is calculated. The parameters of the first decoder are updated according to the cosine distance loss.

9. The method according to claim 1, characterized in that, The method also includes training a second encoder and a second decoder: Obtain a second training sample set, which includes normal images; For each normal image in the second training sample set, construct the corresponding pseudo-abnormal image and pseudo-abnormal mask. The normal images in the second training sample set are input into the first encoder to extract the target multi-level normal feature map; The pseudo-anomaly image is input into the second encoder, and after embedding fusion processing and reconstruction by the second decoder, the target reconstruction feature set is obtained; Calculate the target cosine distance loss between the target reconstructed feature set and the target multi-level normal feature map, and update the parameters of the second encoder and the second decoder based on the target cosine distance loss.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images