An image anomaly detection method based on deep feature histogram

Abstract

The application discloses an image anomaly detection method based on a deep feature histogram, comprising the following steps: collecting normal images to construct a training set and performing pretreatment; performing feature extraction on the pretreated training set images; performing feature selection on the extracted features; constructing a histogram anomaly detection model for the selected features; obtaining an anomaly score map of a to-be-detected image through the anomaly detection model; performing postprocessing on the anomaly score map to obtain an anomaly detection and positioning result of the to-be-detected image. Different from the prior art, the histogram anomaly detection model is constructed in the channel dimension of the image depth feature, the dependence on the pixel position is avoided, the detection effect of the algorithm in the image misalignment scene is improved, and the storage space occupation is greatly reduced. The experimental results show that the application has good performance and robustness in the image-level anomaly detection and pixel-level anomaly positioning tasks.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image anomaly detection, and particularly relates to an image anomaly detection method based on deep feature histogram. BACKGROUND

[0002] Image anomaly detection technology is an important field of computer vision, and is often applied to industries such as industry and medical treatment. It mainly includes two tasks of image-level anomaly detection and pixel-level anomaly positioning. Traditional methods usually rely on manual labeling of anomalies and use supervised learning algorithms for identification. This way is often limited by high labor costs, scarce abnormal samples and complex and variable abnormal patterns.

[0003] With the development of deep learning, more and more image anomaly detection algorithms only use normal samples to train the model, and detect anomalies by learning the commonalities between normal samples. In the benchmark test of the image anomaly detection public dataset MVTec AD proposed in "Bergmann P, Fauser M, Sattlegger D, et al. MVTec AD--A comprehensive real-world dataset for unsupervised anomaly detection [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 9592-9600", some algorithms such as PaDiM proposed in "Defard T, Setkov A, Loesch A, et al. Padim: a patch distribution modeling framework for anomaly detection and localization [C] / / Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10-15, 2021, Proceedings, Part IV. Cham: Springer International Publishing, 2021: 475-489" use pre-trained models to extract the deep features of images and model the distribution of the features by position, reaching the level of SOTA (state-of-the-art). However, this kind of algorithm usually models the distribution by position, so it is very dependent on the alignment of the images in the dataset. Because the images in real scenes are usually non-aligned, this kind of algorithm performs poorly in practical applications. At the same time, this kind of algorithm needs to occupy a large amount of storage space, which is difficult to realize in practical applications.

[0004] Therefore, it is crucial to invent an image anomaly detection algorithm with high robustness and low storage space occupation. SUMMARY

[0005] The present application aims to provide an image anomaly detection method based on deep feature histogram, to solve the technical problems of low robustness and high storage space occupation of existing algorithms.

[0006] To solve the above technical problems, the specific technical solutions of the present application are as follows:

[0007] An image anomaly detection method based on deep feature histogram, comprising the following steps:

[0008] Step S1: collecting normal images to construct a training set and performing preprocessing;

[0009] Step S2: performing feature extraction on the preprocessed training set images;

[0010] Step S3: performing feature selection on the extracted features;

[0011] Step S4: constructing a histogram anomaly detection model for the selected features;

[0012] Step S5: obtaining an anomaly score map of the to-be-tested image through the anomaly detection model;

[0013] Step S6: performing post-processing on the anomaly score map to obtain the anomaly detection and positioning results of the to-be-tested image.

[0014] Further, in step S1, the preprocessing operation is to adjust the picture size to HxW, H is the height of the transformed picture, and W is the width of the transformed picture.

[0015] Further, the specific process of step S2 is:

[0016] Step S21: using a pre-trained model as a feature extractor, inputting each image i of the training set into the pre-trained model to obtain feature maps output by multiple intermediate layers of the pre-trained model; wherein the feature map output by the intermediate layer j is c j is the channel number of the feature map F ij , h j is the height of the feature map F ij , and w j is the width of the feature map F ij ; is a mathematical symbol representing a real number;

[0017] Step S22: for each selected intermediate layer j, upsample the feature map F ij to c j x h0 x w0, h0 is the height of the selected shallowest layer feature map, and w0 is the width of the selected shallowest layer feature map;

[0018] Step S23: concatenate all the upsampled feature maps in the channel dimension to obtain a multi-scale feature map wherein c = ∑ j c j , indicating the channel number of the multi-scale feature map F i ;

[0019] Step S24: keeping the channel dimension unchanged, the multi-scale feature map F i The feature of each position (h, w) is stored in the feature set M; for a training set containing n normal images, the sample number of the feature set M is n x h0 x w0, and each sample m is a one-dimensional vector with c channel dimensions wherein h and w represent the coordinates on the multi-scale feature map F i .

[0020] Further, the specific process of step S3 is as follows:

[0021] Step S31: using the PCA algorithm on the feature set M ([n x h0 x w0, c]), obtaining the eigenvalue diagonal matrix D = diag (λ1, λ2, λ3, …, λ c ) and the eigenvector matrix V = [V1, V2, V3, …, V c ]; wherein λ1≥λ2≥λ3≥…≥λ c .

[0022] Step S32: selecting a feature segmentation point s to obtain the segmented eigenvector matrix For each sample m in the feature set M, the feature-selected vector T is the matrix transpose symbol, and the feature-selected feature set

[0023] Further, in step S32, the specific process of selecting the feature segmentation point s is as follows:

[0024] Step a, taking the median λ r of λ1, λ2, λ3, …, λ m , wherein λ r is the last non-zero eigenvalue;

[0025] Step b, the feature segmentation point wherein μ is a set parameter.

[0026] Further, the specific process of step S4 is as follows:

[0027] Step S41: performing histogram statistics on the feature-selected feature set according to the channel dimension, obtaining c histograms; wherein the histogram of each channel is independent, and the set bin number k is equal;

[0028] Step S42: normalizing each channel histogram so that the maximum height is 1.

[0029] Further, the specific process of step S5 is as follows:

[0030] Step S51: pre-processing the to-be-tested image t, and the size is changed to HxW;

[0031] Step S52: feature extraction is performed on the pre-processed to-be-tested image t, and a multi-scale feature map F is obtained

[0032] Step S53: feature selection is performed on the multi-scale feature map F t The feature F of each position (h, w) t (h, w) is selected to obtain a feature-selected vector

[0033] Step S54: the feature-selected vector is sent to a histogram anomaly detection model to obtain an anomaly score map S The anomaly score of each position (h, w) Wherein, hist i is The probability density of the value on the channel i in the corresponding histogram i.

[0034] Further, the specific process of step S6 is as follows:

[0035] Step S61: the anomaly score map S obtained in step S54 is up-sampled to the size HxW;

[0036] Step S62: the up-sampled anomaly score map is smoothed by using a Gaussian filter to obtain a final anomaly score map S

[0037] Step S63: a threshold image_threshold is set, if the maximum value S of the anomaly score map S is greater than image_threshold, the to-be-tested image is determined to be abnormal, otherwise, it is determined to be normal; a threshold pixel_threshold is set, and the anomaly score map S is binarized according to the threshold to obtain an abnormal positioning result. max

[0038] The image anomaly detection method based on the deep feature histogram has the following advantages:

[0039] 1. The image anomaly detection method based on the deep feature histogram can well adapt to the non-aligned image scene, and the distribution of the deep feature is not modeled according to the position, but the distribution is non-parametrically estimated in the channel dimension in the form of a histogram, so that the adverse effects caused by the non-aligned image are avoided.

[0040] ​​2、The application does not need to store feature information for each pixel block, but only needs to store a histogram model with a limited number of boxes, greatly reducing the occupation of storage space and improving the running efficiency of the algorithm.

[0041] 3、The feature selection step in the application removes the feature subspace with large variance after principal component analysis on the feature set, and the remaining feature subspace with small variance can better express the common features of normal samples, improving the effect of anomaly detection. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a flowchart of the method of the application;

[0043] Figure 2 is an abnormality detection result schematic diagram of the application on the MVTec AD dataset;

[0044] Figure 3 is an abnormality detection result schematic diagram of the application on the Non-aligned MVTec AD dataset;

[0045] Figure 4 is an abnormality detection result schematic diagram of the PaDiM algorithm of the application on the Non-aligned MVTec AD dataset. DETAILED DESCRIPTION

[0046] In order to better understand the purpose, structure and function of the application, the application of a deep feature histogram-based image anomaly detection method will be further described in detail below in combination with the drawings.

[0047] According to the deep feature histogram-based image anomaly detection method provided by the application, as shown in Figure 1 , the method comprises the following steps:

[0048] Step S1: Collect normal images to construct a training set and perform preprocessing, and the operation of the preprocessing is to adjust the collected images to HxW, H is the height of the transformed picture, and W is the width of the transformed picture.

[0049] Step S2: Feature extraction is performed on the preprocessed training set images, and the specific process is as follows:

[0050] Step S21: Use a pre-trained model as a feature extractor, input each image i of the training set into the pre-trained model, and obtain the feature maps output by multiple intermediate layers of the pre-trained model; wherein the feature map output by the intermediate layer j is c j is the number of channels of the feature map F ij , h j is the height of the feature map F ij , and wj For feature map F ij The width;

[0051] Step S22: For each selected intermediate layer j, transfer the feature map F ij Upsampling to c j ×h0×w0, where h0 is the height of the selected shallowest feature map and w0 is the width of the selected shallowest feature map;

[0052] Step S23: Concatenate all the upsampled feature maps along the channel dimension to obtain a multi-scale feature map. Where, c = ∑ j c j , representing the multi-scale feature map F i The number of channels;

[0053] Step S24: Keeping the channel dimension unchanged, convert the multi-scale feature map F i Features of each position (h, w) Store the data in feature set M. For a training set containing n normal images, the number of samples in feature set M is n×h0×w0, and each sample m is a one-dimensional vector with c channels. Where h and w represent the multi-scale feature map F i The coordinates on the [space].

[0054] Step S3: Perform feature selection on the extracted features. The specific process is as follows:

[0055] Step S31: Apply the PCA algorithm to the feature set M([n×h0×w0, c]) to obtain the eigenvalue diagonal matrix D = diag(λ1, λ2, λ3, ..., λ c ) and the eigenvector matrix V = [V1, V2, V3, ..., V c ]; where λ1≥λ2≥λ3≥…≥λ c ;

[0056] Step S32: Select feature segmentation points s to obtain the segmented feature vector matrix. For each sample m in the feature set M, the feature-selected vector is obtained. And obtain the feature set after feature selection.

[0057] In step S32, the specific process of selecting the feature segmentation point s is as follows:

[0058] a) Take λ1, λ2, λ3,…,λ r The median λ m , where λ r It is the last non-zero eigenvalue;

[0059] b) feature segmentation points wherein μ is a set parameter.

[0060] Step S4: constructing a histogram anomaly detection model for the selected features, and the specific process is as follows:

[0061] Step S41: performing feature selection on the feature set after feature selection performing histogram statistics according to the channel dimension to obtain c histograms; wherein the histogram of each channel is independent, and the set number of boxes k is equal;

[0062] Step S42: normalizing the histogram of each channel so that the maximum height is 1;

[0063] Step S5: obtaining an anomaly score map of the to-be-tested image through the anomaly detection model, and the specific process is as follows:

[0064] Step S51: pre-processing the to-be-tested image t, and changing the size to HxW;

[0065] Step S52: performing feature extraction on the pre-processed to-be-tested image t to obtain a multi-scale feature map

[0066] Step S53: performing feature selection on the multi-scale feature map F t the feature F t of each position (h, w) to obtain a feature-selected vector

[0067] Step S54: inputting the feature-selected vector into the histogram anomaly detection model to obtain an anomaly score map the anomaly score of each position (h, w) wherein, hist i is the probability density of the value on the channel i in the corresponding histogram i.

[0068] Step S6: post-processing the anomaly score map to obtain the anomaly detection and positioning result of the to-be-tested image, and the specific process is as follows:

[0069] Step S61: upsampling the anomaly score map obtained in step S54 to the size HxW;

[0070] Step S62: using Gaussian filtering to smooth the upsampled anomaly score map to obtain the final anomaly score map

[0071] Step S63: Set a threshold image_threshold, if the maximum value S of the anomaly score map S is greater than image_threshold, the image to be tested is determined as abnormal; otherwise, it is determined as normal; set a threshold pixel_threshold, and perform binaryzation processing on the anomaly score map S according to the threshold to obtain an abnormal positioning result. max Step S63: Set a threshold image_threshold, if the maximum value S of the anomaly score map S is greater than image_threshold, the image to be tested is determined as abnormal; otherwise, it is determined as normal; set a threshold pixel_threshold, and perform binaryzation processing on the anomaly score map S according to the threshold to obtain an abnormal positioning result.

[0072] Embodiment 1

[0073] In order to prove the effectiveness of the method, the MVTec AD dataset [1] is used as the image data source; the MVTec AD dataset contains 5354 high-resolution color images of different target and texture types, and provides more than 70 different types of anomalies, simulating the real-world industrial anomaly detection scene; the running environment of this embodiment is GPU: RTX 3090 (24GB), CPU: 14 vCPU Intel(R) Xeon(R) Gold 6330 CPU@2.00GHz, operating system: Ubuntu 20.04.4 LTS.

[0074] Step S1: select a normal sample of a category of the MVTec AD dataset to construct a training set, and adjust the size of the training set image to 256x256.

[0075] Step S2: perform feature extraction on the preprocessed training set image, and the specific process is as follows:

[0076] Step S21: use the ImageNet pre-trained model wide_resnet50_2 as a feature extractor, input the training set image into wide_resnet50_2, batch_size=32, and obtain the feature maps output by layer1, layer2 and layer3; wherein the (channel, height, width) of the feature map output by layer1 is (256, 64, 64), the (channel, height, width) of the feature map output by layer2 is (512, 32, 32), and the (channel, height, width) of the feature map output by layer3 is (1024, 16, 16);

[0077] The pre-trained model here can be replaced by other models such as resnet18, vgg19, etc.; generally, the features extracted by the shallow layer contain more texture detail information, and the receptive field is smaller, while the features extracted by the deep layer contain more semantic information, and the receptive field is larger; the first three layers of the pre-trained model wide_resnet50_2 are used in the embodiment, and the last layer is discarded, because the pre-trained model is trained on ImageNet, and the semantic features of the deep layer are biased towards ImageNet, which deviates from the semantic features of the actual data, and affects the performance of anomaly detection;

[0078] Step S22: use the nearest up-sampling method to up-sample layer2 and layer3 to (512, 64, 64) and (1024, 64, 64) respectively;

[0079] Step S23: concatenate the channel dimensions of layer1, layer2 and layer3 feature maps to obtain a multi-scale feature map with (channel, height, width) of (1792, 64, 64);

[0080] Step S24: keep the channel dimension unchanged, and store the features of each position (h, w) of the multi-scale feature map in the feature set M; for a training set containing n normal images (the number of normal samples in each category of MVTec AD is different, which is described as n here), the sample number of the feature set M is n x 64 x 64, and each sample is a one-dimensional vector with a channel number of 1792.

[0081] Step S3: feature selection is performed on the extracted features, and the specific process is as follows:

[0082] Step S31: use the PCA algorithm on the feature set M ([n x 64 x 64, 1792]) to obtain the eigenvalue diagonal matrix D = diag (λ1, λ2, λ3, …, λ c ) and the eigenvector matrix V = [V1, V2, V3, …, V c ]; wherein λ1≥λ2≥λ3≥…≥λ c ;

[0083] Step S32: select a feature segmentation point s to obtain the segmented eigenvector matrix For each sample m in the feature set M, the vector after feature selection is obtained And the feature set after feature selection is obtained

[0084] Here, the feature selection is performed to obtain features with small variance of normal samples, which have high similarity and are key features for constructing normal samples, and can usually be well distinguished from abnormal samples.

[0085] In the step S32, the specific process of selecting the feature segmentation point s is as follows:

[0086] a) Take λ1, λ2, λ3, …, λ r The median λ m of the feature values, where λ r is the last non-zero eigenvalue.

[0087] b) The feature segmentation point The μ in this embodiment is set to 1.

[0088] The method of selecting the feature segmentation point here can refer to the literature “Lin J, Chen S, Lin E, et al. Deep Feature Selection for Anomaly Detection Based on Pretrained Network and Gaussian Discriminative Analysis [J]. IEEE Open Journal of Instrumentation and Measurement, 2022, 1: 1-11”.

[0089] Step S4: Construct a histogram anomaly detection model for the selected features, and the specific process is as follows:

[0090] Step S41: For the feature set after feature selection Perform histogram statistics by channel dimension to obtain 1792 histograms; each histogram of each channel is independent, and the number of boxes is set to 10;

[0091] Step S42: Normalize the histogram of each channel so that the maximum height is 1.

[0092] The histogram anomaly detection model here can refer to the literature “Goldstein M, Dengel A. Histogram-based outlier score (hbos): A fast unsupervised anomaly detection algorithm [J]. KI-2012: poster and demo track, 2012, 1: 59-63”. This model has the characteristics of low computational complexity and is suitable for large data scenarios.

[0093] Step S5: Obtain the anomaly score map of the to-be-tested image through the anomaly detection model, and the specific process is as follows:

[0094] Step S51: Preprocess the to-be-tested image t, and change the size to 256*256. The to-be-tested image here comes from the test set of the same category as the training set in MVTec AD, which includes both normal and abnormal images;

[0095] Step S52: Feature extraction is performed on the preprocessed to-be-tested image t, and a multi-scale feature map F with a size of (1792, 64, 64) is obtained t ;

[0096] Step S53: Feature selection is performed on the test multi-scale feature map F t , and a feature-selected vector t (h, w) is obtained at each position (h, w)

[0097] Step S54: The feature-selected vector is sent to the histogram anomaly detection model, and an anomaly score map S is obtained , wherein each position (h, w) of the anomaly score map S has an anomaly score S (h, w) i , wherein hist is the probability density of the value on the channel i in the corresponding histogram i.

[0098] Step S6: Post-processing is performed on the anomaly score map, and the anomaly detection and positioning result of the to-be-tested image is obtained, and the specific process is as follows:

[0099] Step S61: The anomaly score map S obtained in step S54 is up-sampled to a size of 256*256 by using a bilinear interpolation method;

[0100] Step S62: The up-sampled anomaly score map is smoothed by using a Gaussian filter (σ=4), and a final anomaly score map S is obtained

[0101] Step S63: Set a threshold image_threshold, if the maximum value S max of the anomaly score map S is greater than image_threshold, the to-be-tested image is determined to be abnormal; otherwise, it is determined to be normal; set a threshold pixel_threshold, and perform binarization processing on the anomaly score map S according to the threshold, and obtain an abnormal positioning result;

[0102] ​The area under the receiver operating characteristic curve (AUROC) is commonly used to evaluate the performance of algorithms in the field of image anomaly detection. Image-level anomaly detection uses image AUROC, and pixel-level anomaly localization uses pixel AUROC. In addition, pixel-level anomaly localization also uses AUPRO [5] which can reduce the impact of AUROC's preference for large-area anomalies.

[0103] This embodiment was tested on the MVTec AD dataset, where both image_threshold and pixel_threshold were obtained by maximizing the F1-score, Figure 2 The anomaly detection results of the present application on the MVTec AD dataset are shown in the figure. At the same time, the performance of the present application and the SOTA performance algorithm on MVTec AD is compared: Table 1 shows the results of image-level anomaly detection; Table 2 shows the results of pixel-level anomaly localization.

[0104] Table 1 Comparison of the present application and SOTA performance algorithms for image-level anomaly detection on MVTec AD (image AUROC%)

[0105]

[0106]

[0107] Table 2 Comparison of the present application and SOTA performance algorithms for pixel-level anomaly localization on MVTec AD (pixel AUROC%, AUPRO%)

[0108]

[0109]

[0110] As can be seen from the results in Table 1, the present application has SOTA performance in the image-level anomaly detection task, and is only second to the PaDiM algorithm in overall performance. The results in Table 2 also show that the present application has good competitiveness in the pixel-level anomaly localization task, and is only second to the PaDiM and SPADE. At the same time, compared with other algorithms, the present application has the characteristics of high computational efficiency and relatively low resource consumption.

[0111] Embodiment 2:

[0112] To verify the effectiveness of the present application in the image non-aligned scene, the present embodiment performs non-aligned processing on the MVTec AD dataset to obtain a Non-aligned MVTec AD dataset, and compares the present application and the PaDiM algorithm on the obtained Non-aligned MVTec AD dataset. The specific operation of non-aligned processing is as follows: randomly rotate the image to the left or right by 0-30°, and then randomly crop it with a size ratio of 0.5-1.0.

[0113] The specific steps and running environment are the same as those described in Embodiment 1, Figure 3 Fig. 3 is a schematic diagram of the anomaly detection result of the present application on the Non-aligned MVTec AD dataset. Figure 4 Fig. 4 is a schematic diagram of the anomaly detection result of the PaDiM algorithm on the Non-aligned MVTec AD dataset. Table 3 is the result of image-level anomaly detection, and Table 4 is the result of pixel-level anomaly localization.

[0114] Table 3 Comparison of the present application and the PaDiM algorithm in image-level anomaly detection on the Non-aligned MVTec AD (image AUROC%)

[0115] PaDiM-WR50-Rd550[2] Ours Bottle 97.0 97.5 Cable 76.3 84.6 Capsule 78.1 79.2 Hazelnut 97.4 99.8 Metal_nut 87.0 87.7 Pill 77.5 85.2 Screw 59.8 71.3 Toothbrush 86.1 90.0 Transistor 75.9 70.8 Zipper 70.1 78.3 All10ObjectClasses 80.5 84.4 Carpet 97.7 94.9 Grid 96.0 88.1 Leather 98.2 96.2 Tile 94.9 97.5 Wood 99.7 99.6 All5TextureClasses 97.3 95.3 Averageof15Classes 86.1 88.1

[0116] Table 4 Comparison of the present application and the PaDiM algorithm in pixel-level anomaly localization on the Non-aligned MVTec AD (pixel AUROC%, AUPRO%)

[0117]

[0118]

[0119] From the results of Table 3 and Table 4, it can be seen that in the image non-aligned scene (dataset Non-aligned MVTec AD), the average image AUROC of the present application is 2% higher than that of the PaDiM algorithm, and 11 out of 15 categories perform better than the PaDiM algorithm; and compared with the image aligned scene (dataset MVTec AD), the average image AUROC of the PaDiM algorithm decreases by 8.8%, while the average image AUROC of the present application only decreases by 4.2%.

[0120] In the pixel-level anomaly localization task, the present application performs better than the PaDiM algorithm in 8 out of 15 categories in terms of pixel AUROC; in terms of AUPRO, the present application performs better than the PaDiM algorithm in 10 categories; and compared with the image alignment scene (dataset MVTec AD), the average pixel AUROC and average AUPRO of the PaDiM algorithm decrease by 4.4% and 8.2% respectively, while the present application only decreases by 3.5% and 5.2% respectively; and the average pixel AUROC and average AUPRO of the two are not much different, which is mainly caused by the poor detection effect of the present application in the Transistor category, which is not suitable for using the present application for image features.

[0121] In summary, the present application has better anomaly detection and localization effect than the PaDiM algorithm in the image non-alignment scene, and has higher robustness.

[0122] [1] refers to the reference Bergmann P, Fauser M, Sattlegger D, et al. MVTec AD--A comprehensive real-world dataset for unsupervised anomaly detection [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 9592-9600.

[0123] [2] refers to the reference Defard T, Setkov A, Loesch A, et al. Padim: a patch distribution modeling framework for anomaly detection and localization [C] / / Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10-15, 2021, Proceedings, Part IV. Cham: Springer International Publishing, 2021: 475-489.

[0124] [3] refers to the reference Lin J, Chen S, Lin E, et al. Deep Feature Selection for Anomaly Detection Based on Pretrained Network and Gaussian Discriminative Analysis [J]. IEEE Open Journal of Instrumentation and Measurement, 2022, 1 : 1-11.

[0125] [4] refers to the reference Goldstein M, Dengel A. Histogram-based outlier score (hbos): A fast unsupervised anomaly detection algorithm [J]. KI-2012: poster and demo track, 2012, 1 : 59-63.

[0126] [5] refers to the reference Bergmann P, Fauser M, Sattlegger D, et al. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020: 4183-4192.

[0127] [6] refers to the reference Akcay S, Atapour-Abarghouei A, Breckon T P. Ganomaly: Semi-supervised anomaly detection via adversarial training [C] / / Computer Vision - ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Part III 14. Springer International Publishing, 2019: 622-637.

[0128] [7] refers to the reference Andrews J, Tanay T, Morton E J, et al. Transfer representation-learning for anomaly detection [C]. JMLR, 2016.

[0129] [8] refers to the reference Chaoqing Huang, Jinkun Cao, Fei Ye, Maosen Li, Ya Zhang, and Cewu Lu. Inverse-transform autoencoder for anomaly detection. CoRR, abs / 1911.10676, 2019.

[0130] [9] refers to the reference Cohen N, Hoshen Y. Sub-image anomaly detection with deep pyramid correspondences [J]. arXiv preprint arXiv:2005.02357, 2020.

[0131]

[10] refers to the reference Yi J, Yoon S. Patch svdd: Patch-level svdd for anomaly detection and segmentation [C] / / Proceedings of the Asian Conference on Computer Vision. 2020.

[0132] It can be understood that the present application is described through some embodiments, and those skilled in the art know that various changes or equivalent replacements can be made to the features and embodiments without departing from the spirit and scope of the present application. In addition, under the guidance of the present application, the features and embodiments can be modified to adapt to specific conditions and materials without departing from the spirit and scope of the present application. Therefore, the present application is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application are within the scope of the present application.

Claims

1. An image anomaly detection method based on depth feature histograms, characterized in that, Includes the following steps: Step S1: Collect normal images to construct a training set and perform preprocessing; Step S2: Extract features from the preprocessed training set images; Step S3: Perform feature selection on the extracted features; Step S4: Construct a histogram anomaly detection model for the selected features; Step S5: Obtain the anomaly score map of the image to be tested using the anomaly detection model; Step S6: Post-process the anomaly scoring map to obtain the anomaly detection and localization results of the image under test; The specific process of step S2 is as follows: Step S21: Using a pre-trained model as a feature extractor, input each image i from the training set into the pre-trained model to obtain feature maps output by multiple intermediate layers of the pre-trained model; wherein, the feature map output by intermediate layer j is... c j For feature map F ij The number of channels, h j For feature map F ij height, w j For feature map F ij The width; Step S22: For each intermediate layer j selected, upsample the feature map F ij to c j x h0 x w0, h0 being the height of the selected shallowest layer feature map, w0 being the width of the selected shallowest layer feature map; Step S23: Concatenate all the upsampled feature maps along the channel dimension to obtain a multi-scale feature map. Where, c = ∑ j c j , representing the multi-scale feature map F i The number of channels; Step S24: Keeping the channel dimension unchanged, convert the multi-scale feature map F i Features of each position (h, w) Store the data in feature set M; for a training set containing n normal images, the number of samples in feature set M is n×h0×w0, and each sample m is a one-dimensional vector with c channels. Where h and w represent the multi-scale feature map F i coordinates on; The specific process of step S4 is as follows: Step S41: Select the feature set after feature selection Histogram statistics are performed along the channel dimension to obtain c histograms; among them, the histogram of each channel is independent and the set number of bins k is equal. Step S42: Normalize the histogram of each channel so that the maximum height is 1; The specific process of step S5 is as follows: Step S51: Preprocess the image to be tested t, and change its size to H×W, where H is the height of the transformed image and W is the width of the transformed image; Step S52: Perform feature extraction on the preprocessed image t to obtain a multi-scale feature map. Step S53: Process the multi-scale feature map F t The feature F of each position (h,w) t Perform feature selection on (h,w) to obtain the feature-selected vector. s is the feature segmentation point; Step S54: Select the vector after feature selection The data is fed into a histogram anomaly detection model to obtain an anomaly score map. Anomaly score for each location (h, w) Among them, hist i for The probability density of the value in channel i on the corresponding histogram i.

2. The image anomaly detection method based on depth feature histograms according to claim 1, characterized in that, In step S1, the preprocessing operation is to adjust the image size to H×W.

3. The image anomaly detection method based on depth feature histograms according to claim 2, characterized in that, The specific process of step S3 is as follows: Step S31: Apply the PCA algorithm to the feature set M([n×h0×w0,c]) to obtain the eigenvalue diagonal matrix D = diag(λ1,λ2,λ3,…,λ c ) and the eigenvector matrix V = [V1, V2, V3, ..., V c ]; where λ1≥λ2≥λ3≥…≥λ c ; Step S32: Select feature segmentation points s to obtain the segmented feature vector matrix. For each sample m in the feature set M, the feature-selected vector is obtained. And obtain the feature set after feature selection.

4. The image anomaly detection method based on depth feature histograms according to claim 3, characterized in that, In step S32, the specific process of selecting the feature segmentation point s is as follows: Step a, take λ1, λ2, λ3,…,λ r The median λ m , where λ r It is the last non-zero eigenvalue; Step b, Feature segmentation points Where μ is a set parameter.

5. The image anomaly detection method based on depth feature histograms according to claim 4, characterized in that, The specific process of step S6 is as follows: Step S61: Calculate the anomaly score map obtained in step S54. Upsampled to size H×W; Step S62: Smooth the upsampled anomaly rating map using Gaussian filtering to obtain the final anomaly rating map. Step S63: Set a threshold image_threshold. If the maximum value S of the anomaly rating image S is exceeded... max If the value is greater than image_threshold, the image under test is judged as abnormal; otherwise, it is judged as normal. A threshold pixel_threshold is set, and the abnormal scoring image S is binarized according to this threshold to obtain the abnormal location result.

Images