An image anomaly detection method based on local perception knowledge distillation network

Abstract

The application discloses an image anomaly detection method based on a local perception knowledge distillation network. An anomaly detection network with a feature local perception module and a difficult example perception loss function is constructed. Non-anomalous images are input for training, and real-time test images are input into the trained anomaly detection network to obtain an abnormal image for judging and detecting anomalies. The anomaly detection network includes a structure teacher network and a student network, and a feature local perception operation module is used to obtain a feature corresponding graph containing local information. The student network is trained under a multi-level difficult example perception feature loss, while the teacher network remains unchanged. The feature loss of the two networks is large for abnormal positions. The application can fully perceive context feature information and local spatial information, improve the network's structured feature extraction capability and the role of structured information in the anomaly detection task, enhance the network's ability to perceive small abnormal areas, the network's adaptability to noise and the detection accuracy of abnormal areas, and effectively improve the network's generalization performance.

Description

Technical Field

[0001] This invention relates to an image detection method in the field of computer vision, and particularly to an image anomaly detection method based on a local perceptual knowledge distillation network. Background Technology

[0002] Anomaly detection primarily aims to identify patterns that differ from the norm, and it is widely used in video surveillance, product quality control, and medical diagnosis. Traditional anomaly detection methods rely on manually generated features, which are ineffective in handling complex and diverse detection scenarios. Deep learning methods, capable of autonomously extracting high-dimensional features from defective images, have been extensively explored in recent years. However, due to the diversity of anomaly data types and the scarcity of anomaly samples, supervised deep learning methods cannot effectively handle anomaly detection scenarios. Although knowledge distillation models can achieve unsupervised training based solely on positive samples, this network model relies on feature maps extracted by teacher and student networks, determining anomaly locations only through the difference in the position of a single pixel corresponding to the two feature maps. This fails to effectively utilize the contextual information of the image, limiting the network's ability to extract structured features. Furthermore, knowledge distillation network models rely solely on the distance between corresponding pixels to construct the loss function, obscuring the differences between abnormal and normal regions. This fails to effectively exploit the impact of difficult samples on network optimization, hindering the improvement of anomaly detection localization accuracy. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of the prior art by proposing an anomaly detection method based on a locally perceptive knowledge distillation network. This method enhances the network's ability to extract structured features by constructing a feature-locally perceptive module to extract contextual information from the knowledge distillation network. Furthermore, the constructed hard-example perceptual loss function widens the gap between the loss in anomaly regions and the loss in normal regions, thereby increasing the contribution of hard samples to the network loss.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] Step 1: Construct an anomaly detection network with a feature-local perception module and a hard-example perception loss function;

[0006] Step 2: Input historical images without anomalies into the anomaly detection network for training;

[0007] Step 3: Acquire real-time images of the scene to be tested, input them into the trained anomaly detection network, detect and output anomaly maps, and determine anomalies based on the anomaly maps.

[0008] Both the historical images and the real-time images are two-dimensional images.

[0009] The aforementioned anomalies refer to situations that differ from the norm most of the time, and can be defined according to specific needs.

[0010] The anomaly detection network mentioned above is as follows: Figure 4 As shown, it includes a teacher network, a student network, multiple feature local perception modules, and multiple hard example perception loss functions;

[0011] The teacher network is mainly composed of four consecutive convolutional modules Tconv1 to Tconv4 connected in sequence, and the student network is mainly composed of four consecutive convolutional modules Sconv1 to Sconv4 connected in sequence. The convolutional modules Tconv1 to Tconv4 correspond to the convolutional modules Sconv1 to Sconv4. Each convolutional module is composed of multiple convolutional blocks connected in sequence, and each convolutional block is composed of multiple consecutive convolutional operations.

[0012] The input images are fed into the teacher network and the student network respectively. The high-dimensional feature maps output by the second convolutional module Tconv2 of the teacher network and the second convolutional module Sconv2 of the student network are passed through their respective feature local perception modules to obtain their respective local perception feature maps, which are then input into the first hard example perception loss function. The first hard example perception loss function outputs the first feature distance map.

[0013] The input images are fed into the teacher network and the student network respectively. The high-dimensional feature maps output by the third convolutional module Tconv3 of the teacher network and the third convolutional module Sconv3 of the student network are passed through their respective feature local perception modules to obtain their respective local perception feature maps, which are then input into the second hard example perception loss function. The second hard example perception loss function outputs the second feature distance map.

[0014] The input images are fed into the teacher network and the student network respectively. The high-dimensional feature maps output by the fourth convolutional module Tconv4 of the teacher network and the fourth convolutional module Sconv4 of the student network are processed by their respective feature local perception modules to obtain their respective local perception feature maps, which are then input into the third hard example perception loss function. The third hard example perception loss function outputs the third feature distance map.

[0015] Multiplying the three feature distance maps O2, O3 and O4 together yields the anomaly map O.

[0016] The method of detecting anomalies based on anomaly graphs specifically involves: traversing each feature position in anomaly graph O; if the value of a feature position is greater than a preset anomaly threshold, then an anomaly is considered to exist in anomaly graph O, i.e., an anomaly exists in the real-time image.

[0017] In step 2, during the training process, the hard example perception loss is calculated for each feature distance map, and the three hard example perception losses corresponding to the three feature distance maps are directly added together to obtain the total loss. The anomaly detection network is trained with the goal of minimizing the total loss, thereby optimizing the parameters of the anomaly detection network.

[0018] The feature local perception module is specifically:

[0019] S1. Using each channel of the high-dimensional feature map as a feature channel map, iterate through each feature channel map in the high-dimensional feature map:

[0020] S2. Traverse each feature position (i,j) in the feature channel map and perform the following processing at each feature position (i,j): establish a search range around the feature position (i,j) to search for neighboring points. The height of the search range is h and the width is w. Take the maximum value of all neighboring points in the search range as the local perceptual feature of the feature position (i,j).

[0021] S3. Repeat the above steps for each feature position on the feature channel map to traverse each feature position (i,j) in the feature channel map, thereby obtaining the local perceptual features of each feature position (i,j) in the feature channel map.

[0022] S4. Repeat the steps to traverse each feature channel map in the high-dimensional feature map for each channel of the network high-dimensional feature map, thereby obtaining the local perceptual features of each feature position (i,j) in each feature channel map of the high-dimensional feature map, and finally forming the local perceptual feature map of the high-dimensional feature map.

[0023] When the feature location (i,j) is located at an edge position, add 0 pixels to the areas within the search range that do not have a feature location, so that the entire feature map can use a search range of the same size.

[0024] The processing for outputting the first feature distance map from the first hard example perceptual loss function, the second feature distance map from the second hard example perceptual loss function, and the third feature distance map from the third hard example perceptual loss function is all the same. Specifically, the feature distance maps output by the hard example perceptual loss function are as follows:

[0025] S1. Based on the high-dimensional feature maps obtained from the teacher network and student network, perform regularization processing on the feature vectors at each feature position (i,j) to obtain the regularized vectors at the corresponding feature positions (i,j) of the teacher network and student network. and

[0026]

[0027]

[0028] Among them, F t (I) represents the high-dimensional feature map obtained by the teacher network, F s (I) represents the high-dimensional feature map obtained from the student network, F t (I) ij F represents the teacher feature vector at feature position (i,j) of the high-dimensional feature map obtained by the teacher network. s (I) ij Let represent the student feature vector at feature position (i,j) in the high-dimensional feature map obtained by the student network. and Let i and j represent the regularization vectors at feature positions (i,j) corresponding to the teacher network and student network, respectively, and I represent the input image.

[0029] S2. Based on the regularization vector at each feature position (i,j) corresponding to the teacher network and student network. and The feature distance dis(I) at each feature position (i,j) is calculated using the following formula. ij Furthermore, the feature distance dis(I) at all feature positions (i,j) is used. ij Composition to obtain the feature distance map:

[0030]

[0031] Where, dis(I) ij This represents the feature distance at feature position (i,j). This represents the L2 norm.

[0032] The calculation of the hard-example perception loss for each feature distance map is specifically as follows:

[0033] S1. Traverse each feature position (i,j) on the feature distance map, first using the distance dis(I) at the same feature position (i,j) between the teacher network and the student network. ij Calculate the hard example perception loss value l(I) at the feature location (i,j). ij :

[0034]

[0035] Where e represents the natural constant;

[0036] S2. The hard case perception loss value l(I) is calculated based on the hard case perception value at each feature location (i,j). ij The hard-example perceptual loss l(I) of the feature distance map generated between the teacher network and the student network is calculated by traversing each feature location (i,j) according to the following formula:

[0037]

[0038] Where H represents the height of the high-dimensional feature map, and W represents the width of the high-dimensional feature map.

[0039] The method of this invention can be applied to images in fields such as video surveillance, product quality control, and medical diagnosis.

[0040] When applied to video surveillance images, it can detect abnormal behavior of pedestrians or abnormal movement of moving objects in the images.

[0041] When applied to product quality control images, it can detect defects in the product within the image.

[0042] When applied to medical diagnostic images, it can detect diseases in the images.

[0043] This invention designs a local perception operation module for high-dimensional features of images. A complex teacher network and a simple student network are constructed. Simultaneously, feature maps corresponding to the teacher and student networks are constructed, and the designed local perception operation module is applied to these feature maps to obtain feature correspondence maps containing local information. A hard-example perception feature loss is constructed between the two networks based on the multi-level feature correspondence maps. The student network is trained under the guidance of the multi-level feature loss, while keeping the teacher network model unchanged. The locations with larger feature losses in both networks are identified as anomaly locations in the image.

[0044] The beneficial effects of this invention are:

[0045] The method of this invention can fully perceive contextual feature information during feature construction, improve the network's ability to extract structured features, and enhance the anomaly detection network's ability to perceive small anomaly regions.

[0046] The method of this invention can effectively utilize local spatial information in two-dimensional images, enhance the role of structured information in anomaly detection, and improve the network's adaptability to noise and the detection accuracy of anomaly regions.

[0047] Meanwhile, the network loss error established by this invention can perceive difficult examples online and increases the weight of difficult examples in the network loss, which is beneficial to improving the network's ability to detect abnormal regions of the same category but different sizes, and effectively improves the network's generalization performance. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of high-dimensional image features;

[0049] Figure 2 This is a schematic diagram of the process of constructing the local perceptual feature map at position (i,j);

[0050] Figure 3 This is a diagram showing the addition of 0 pixels at the outer edge of the feature map;

[0051] Figure 4 It is an anomaly detection network based on local perception knowledge distillation;

[0052] Figure 5 This is a flowchart of the anomaly detection method of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0054] It should be understood that the embodiments described in this invention are exemplary, and the specific parameters used in the embodiment description are only for the purpose of describing this invention and are not intended to limit this invention.

[0055] like Figure 5 As shown, embodiments of the present invention include the following steps:

[0056] Step 1: Construct a feature local perception module.

[0057] The constructed feature local perception module is mainly designed for extracting high-dimensional feature maps from the network. For example... Figure 1 As shown, the high-dimensional feature map of the image has dimensions H×W×C, representing the image height H, width W, and number of channels C, respectively. The feature local perception module operates on each channel of the high-dimensional feature map, extracting contextual information for each feature location to obtain a local perception feature map. The specific steps are as follows:

[0058] (1) Iterate through and select a feature channel map;

[0059] (2) Establish a search range search neighborhood around the feature position (i,j) of the selected feature channel map. The height of the search range is h and the width is w.

[0060] (3) Take the maximum value of all neighboring points within the search range as the local perceptual feature map of the feature location (i,j).

[0061] (4) Repeat steps (2)-(3) for each feature location on the selected feature channel map to obtain the local perceptual feature map of the corresponding location.

[0062] (5) Perform steps (1)-(4) for each channel of the high-dimensional features of the network to obtain the local perceptual feature map of the high-dimensional features.

[0063] Taking the feature position of the i-th row and j-th column of a certain channel as an example, the construction process of steps (2)-(3) is introduced, such as Figure 2As shown in the diagram. Black dots represent locations where a local perceptual feature map needs to be constructed, and the height h and width w of the neighborhood search range are both 3 pixels. It should be noted that when solving for the local perceptual feature map at the edge of the feature channel map, 0 pixels need to be added to the outer edge of the edge location to ensure that the entire feature map can use a search range of the same size, such as... Figure 3 As shown.

[0064] For the high-dimensional feature maps TF2, TF3, TF4 and SF2, SF3, SF4, refer to Figure 2 and Figure 3 Local perceptual feature maps for these high-dimensional features are constructed. The corresponding local perceptual feature maps are TL2, TL3, TL4 and SL2, SL3, SL4. In the specific implementation, the height h and width w of the neighborhood search range are both set to 3 pixels, and 0 pixels are added to the outer edge of the feature map.

[0065] Step 2: Construct the hard example-aware loss function.

[0066] During training, the main goal of the network is to obtain the feature distribution of normal samples by matching the high-dimensional features extracted by the teacher network and the student network. Given an input image I, the feature map obtained by the teacher network is... The corresponding feature map obtained by the student network is Let H represent the real number field, with dimensions H×W×C. The teacher feature vector and student feature vector at feature position (i,j) are respectively... and Let F represent the real number field, with dimension C. When constructing the loss at feature location (i,j), first, the feature vector F... t (I) ij and F s (I) ij Regularization is applied, and then a loss function is constructed based on the L2 distance between the two regularized vectors. This is done using the teacher network feature map. Student Network Feature Map For example, the steps to construct a hard-example perceptual loss function for a corresponding feature map are as follows:

[0067] (1) Construct regularized vectors at the feature positions (i,j) corresponding to the teacher network and student network. and

[0068]

[0069]

[0070] (2) Calculate the L2 distance dis(I) between the two regularized vectors obtained in the previous step.ij .

[0071]

[0072] (3) Calculate the hard example perception loss l(I) at the feature positions (i,j) corresponding to the teacher network and the student network. ij .

[0073]

[0074] (4) Calculate the hard example perceptual loss l(I) of the corresponding feature maps generated by the two networks.

[0075]

[0076] Based on the above formulas (1)-(5), construct the difficult example perception losses L2, L3, and L4, and then sum the three difficult example perception losses L2, L3, and L4 to obtain the total network loss L.

[0077] Step 3: Construct an anomaly detection network.

[0078] The constructed anomaly detection network based on local perception knowledge distillation is as follows: Figure 4 As shown in the diagram. Here, Tconv1, Tconv2, Tconv3, and Tconv4 represent convolutional modules at different stages in the teacher network. Sconv1, Sconv2, Sconv3, and Sconv4 represent convolutional modules at different stages in the student network, corresponding to the convolutional modules in the teacher network.

[0079] The anomaly detection network takes a two-dimensional image as input. The convolutional modules Tconv1, Tconv2, Tconv3, and Tconv4 in the teacher network extract teacher features TF1, TF2, TF3, and TF4 from the image, respectively. Similarly, the convolutional modules Sconv1, Sconv2, Sconv3, and Sconv4 in the student network extract student features SF1, SF2, SF3, and SF4 from the image, respectively. The corresponding teacher and student features have the same dimensionality. Teacher features TF2, TF3, and TF4 are processed by a feature local perception module to obtain local perception feature maps TL2, TL3, and TL4. Similarly, student features SF2, SF3, and SF4 are processed by the same module to obtain local perception feature maps SL2, SL3, and SL4.

[0080] The specific input image size is 256×256.

[0081] In the teacher network, the operation corresponding to Tconv1 is (7×7,64)×1, and the operation corresponding to Tconv2 is... The operation corresponding to Tconv3 is: The operation corresponding to Tconv4 is:

[0082] In the student network, the operation corresponding to Sconv1 is (7×7,64)×1, and the operation corresponding to Sconv2 is... The operation corresponding to Sconv3 is... The operation corresponding to Sconv4 is:

[0083] Taking Tconv2 as an example, the corresponding operation for Tconv2 is: The numbers inside the parentheses represent convolutional blocks, and the numbers outside the parentheses indicate that the number of convolutional blocks stacked is 3. The 1×1 and 3×3 numbers inside the convolutional blocks represent the kernel size, and 64 and 256 represent the number of convolutional kernels.

[0084] Step 4: Train the constructed anomaly detection network.

[0085] The local perceptual feature maps obtained by the teacher network and the student network are processed by the hard example perceptual loss module to obtain corresponding hard example perceptual losses L2, L3, and L4. The three hard example perceptual losses are summed to obtain the total loss L of the network. The abnormal network completes the training process under the guidance of the total loss L.

[0086] In practice, the anomaly detection network was trained using the MVTec anomaly detection dataset. This dataset contains over 5000 high-resolution images covering 15 different object types. The initial learning rate was set to 0.4, the training batch size was 32, and the maximum number of iterations was 200 epochs.

[0087] Step 5: After training, the anomaly detection network directly predicts the abnormal regions in the input object image.

[0088] After the anomaly detection network is trained, anomaly detection tests can be performed on the input image. The main goal of the testing phase is to obtain an anomaly region map O of size Rh×Rw. The specific steps are as follows:

[0089] (1) Input the test image into the teacher network and the student network to obtain the local perceptual feature maps TL2, TL3, and TL4 of the teacher network and the local perceptual feature maps SL2, SL3, and SL4 of the student network.

[0090] (2) Calculate the distances between the three feature pairs TL2 and SL2, TL3 and SL3, and TL4 and SL4 at corresponding positions (i,j) according to formulas (1)-(3). The (i,j) positions represent each position on the local sensing feature map. After the distances at each position on the local sensing feature map are calculated, the distance map between each pair of features can be obtained. The feature distance map between TL2 and SL2 is denoted as O2, the feature distance map between TL3 and SL3 is denoted as O3, and the feature distance map between TL4 and SL4 is denoted as O4.

[0091] (3) Multiply the feature distance maps O2, O3 and O4 calculated in step (2) by their corresponding elements to obtain the anomaly map O of size Rh×Rw, as shown below:

[0092] O = O2 × O3 × O4

[0093] It should be noted that when the dimensions of feature maps O2, O3, and O4 are not Rh×Rw, bilinear interpolation is used to adjust their dimensions to Rh×Rw, and then they are multiplied by their corresponding elements.

[0094] Compared with existing technologies, the method of this invention constructs a feature-local perception module for capturing feature context information and a hard-example perception loss function for hard-example mining. The proposed method can effectively utilize the unstructured features of images, improve the network's perception ability of local and global features, effectively handle complex situations such as noise and unknown anomalies, and mine hard examples in the dataset, thus improving the model's generalization performance. On the MVTec anomaly detection dataset, this invention achieved a pixel-level anomaly detection accuracy of 97.2% and an image-level anomaly detection accuracy of 91.3%. The anomaly detection performance surpasses similar anomaly detection methods such as SSIM-AE, AnoGAN, CNN-Dic, Cut-Paste, Patch-SVDD, PaDiM-R18, and SPADE.

[0095] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited to the above embodiment. Any equivalent substitutions or changes made by those skilled in the art within the technical scope described in the present invention, based on the technical solution and inventive concept of the present invention, should be included within the scope of protection of the present invention.

Claims

1. An anomaly detection method based on a local perceptual knowledge distillation network, characterized in that: Step 1: Construct an anomaly detection network with a feature-local perception module and a hard-example perception loss function; Step 2: Input historical images without anomalies into the anomaly detection network for training; Step 3: Acquire real-time images of the scene to be tested, input them into the trained anomaly detection network, detect and output anomaly maps, and determine anomalies based on the anomaly maps; In step 2, during the training process, the hard example perception loss of each feature distance map is calculated separately, and the total loss is obtained by combining the three hard example perception losses corresponding to the three feature distance maps. The anomaly detection network is trained with the goal of minimizing the total loss, and the parameters of the anomaly detection network are optimized. The feature local perception module is specifically: S1. Use each channel of the high-dimensional feature map as a feature channel map: S2. At each feature position (i,j), perform the following processing: establish a search range around the feature position (i,j) to search for neighboring points. The height of the search range is h and the width is w. Take the maximum value of all neighboring points in the search range as the local perceptual feature of the feature position (i,j). S3. Repeat the above steps to traverse each feature position (i,j) in the feature channel map, thereby obtaining the local perceptual features of each feature position (i,j) in the feature channel map. S4. Repeat the steps to traverse each feature channel map in the high-dimensional feature map, thereby obtaining the local perceptual features of each feature position (i,j) in each feature channel map of the high-dimensional feature map, and finally forming the local perceptual feature map of the high-dimensional feature map. The calculation of the hard-example perception loss for each feature distance map is specifically as follows: S1. Traverse each feature position (i,j) on the feature distance map, first using the distance dis(I) at the same feature position (i,j) between the teacher network and the student network. ij Calculate the hard example perception loss value l(I) at the feature location (i,j). ij : Where e represents the natural constant; S2. The hard case perception loss value l(I) is calculated based on the hard case perception value at each feature location (i,j). ij The hard-example perceptual loss l(I) of the feature distance map generated between the teacher network and the student network is calculated by traversing each feature location (i,j) according to the following formula: Where H represents the height of the high-dimensional feature map, and W represents the width of the high-dimensional feature map.

2. The anomaly detection method based on a local perceptual knowledge distillation network according to claim 1, characterized in that: The anomaly detection network includes a teacher network, a student network, multiple feature local perception modules, and multiple hard example perception loss functions; The teacher network is mainly composed of four consecutive convolutional modules Tconv1 to Tconv4 connected in sequence, and the student network is mainly composed of four consecutive convolutional modules Sconv1 to Sconv4 connected in sequence. Each convolutional module is composed of multiple convolutional blocks connected in sequence, and each convolutional block is composed of multiple consecutive convolutional operations. The input images are fed into the teacher network and the student network respectively. The high-dimensional feature maps output by the second convolutional module Tconv2 of the teacher network and the second convolutional module Sconv2 of the student network are passed through their respective feature local perception modules to obtain their respective local perception feature maps, which are then input into the first hard example perception loss function. The first hard example perception loss function outputs the first feature distance map. The input images are fed into the teacher network and the student network respectively. The high-dimensional feature maps output by the third convolutional module Tconv3 of the teacher network and the third convolutional module Sconv3 of the student network are passed through their respective feature local perception modules to obtain their respective local perception feature maps, which are then input into the second hard example perception loss function. The second hard example perception loss function outputs the second feature distance map. The input images are fed into the teacher network and the student network respectively. The high-dimensional feature maps output by the fourth convolutional module Tconv4 of the teacher network and the fourth convolutional module Sconv4 of the student network are processed by their respective feature local perception modules to obtain their respective local perception feature maps, which are then input into the third hard example perception loss function. The third hard example perception loss function outputs the third feature distance map. Multiplying the three feature distance maps O2, O3 and O4 together yields the anomaly map O.

3. The anomaly detection method based on a local perceptual knowledge distillation network according to claim 2, characterized in that: The method of detecting anomalies based on anomaly graphs specifically involves: traversing each feature position in anomaly graph O; if the value of a feature position is greater than a preset anomaly threshold, then an anomaly is considered to exist in anomaly graph O, i.e., an anomaly exists in the real-time image.

4. The anomaly detection method based on a local perceptual knowledge distillation network according to claim 1, characterized in that: When the feature position (i,j) is located at an edge position, add 0 pixels to the areas within the search range that do not have a feature position.

5. The anomaly detection method based on a local perceptual knowledge distillation network according to claim 2, characterized in that: The specific feature distance maps output by the hard example perception loss function are as follows: S1. Based on the high-dimensional feature maps obtained from the teacher network and student network, perform regularization processing on the feature vectors at each feature position (i,j) to obtain the regularized vectors at the corresponding feature positions (i,j) of the teacher network and student network. and Among them, F t (I) represents the high-dimensional feature map obtained by the teacher network, F s (I) represents the high-dimensional feature map obtained from the student network, F t (I) ij F represents the teacher feature vector at feature position (i,j) of the high-dimensional feature map obtained by the teacher network. s (I) ij Let represent the student feature vector at feature position (i,j) in the high-dimensional feature map obtained by the student network. and Let i and j represent the regularization vectors at feature positions (i,j) corresponding to the teacher network and student network, respectively, and I represent the input image. S2. Based on the regularization vector at each feature position (i,j) corresponding to the teacher network and student network. and The feature distance dis(I) at each feature position (i,j) is calculated using the following formula. ij Furthermore, the feature distance dis(I) at all feature positions (i,j) is used. ij Composition to obtain the feature distance map: Where, dis(I) ij This represents the feature distance at feature position (i,j). This represents the L2 norm.

Images