Supermarket cargo outer package anomaly detection method based on multi-branch feature extraction and foreground mining

Abstract

The invention relates to a supermarket cargo outer package anomaly detection method based on multi-branch feature extraction and foreground mining, and belongs to the field of image anomaly detection. According to the method, an anomaly detection task is decomposed into three stages of anomaly synthesis, feature extraction and knowledge distillation. Firstly, a third-party data set is introduced, local texture anomalies are generated on normal samples, the normal samples and pseudo-abnormal samples collaboratively optimize the network, and the problem of consistency expression of a teacher-student network in an abnormal region is relieved. Secondly, constructing a multi-branch feature extractor as a teacher network and a student network, fusing feature details of different scales by crossing up-sampling and down-sampling, and integrating high-level semantic information and low-level texture information; and finally, taking foreground mining loss as distillation loss, and enabling the model to focus on foreground target mining so as to increase the difference between abnormal and normal foreground in teacher-student characteristics. According to the method, the sensitivity and the feature discrimination capability of the model to anomalies are remarkably enhanced.

Description

Technical Field

[0001] The invention belongs to the field of image anomaly detection, and in particular relates to a supermarket goods outer packaging anomaly detection method based on multi-branch feature extraction and foreground mining. Background Art

[0002] The anomaly detection of the outer packaging of goods in traditional supermarkets belongs to the field of image anomaly detection. This task includes image-level anomaly detection and pixel-level anomaly localization tasks. Existing methods mainly distinguish normal samples from abnormal samples by extracting feature information of images, thereby achieving accurate detection of anomalies.

[0003] In recent years, deep learning-based image anomaly detection methods have been widely used in this field, especially the introduction of knowledge distillation technology has significantly improved the detection accuracy. The knowledge distillation-based method constructs a teacher-student network architecture, uses the rich representation capabilities of the teacher network to provide guidance for the student network, and compares the differences in the output features of the two to locate anomalies. This method can effectively capture subtle abnormal patterns in images and is particularly suitable for high-precision detection tasks in complex scenarios.

[0004] In order to improve the detection effect, MKD[1] uses Euclidean distance and cosine similarity as distillation loss between the output features of teacher-student networks at different levels to strengthen the consistency between teacher-student networks. MRKD[2] introduces image-level and pixel-level masks in the reverse distillation structure to transform the reconstruction task into an image restoration task, which helps to capture global information and enhance the fine-grained local representation capability to improve the sensitivity to abnormal areas. CDO[3] introduces edge optimization modules and overlap optimization modules to optimize the edges and overlaps between normal samples and abnormal samples, thereby improving the reliability of prediction. In order to adaptively fuse multi-level features, Destseg[4] proposes to use a segmentation network to optimize the feature difference map, thereby improving the accuracy of positioning.

[0005] At present, the anomaly detection algorithm based on knowledge distillation mainly relies on the transfer of knowledge from the teacher model to the student model, and detects anomalies by comparing the differences in the output features of the teacher-student network. Therefore, the core of this type of method lies in designing an efficient teacher-student network architecture and a targeted distillation loss function. For the cargo dataset, the existing anomaly detection algorithm based on knowledge distillation does not adequately mine texture information and structural information, which restricts the further improvement of anomaly detection results.

[0006] In order to overcome this shortcoming, choosing a suitable teacher-student network and designing a targeted distillation loss are the key to improving anomaly detection performance. RD[5] uses WideResnet50[6] as the teacher and student network. Although this design can extract deep semantic information, its feature channels are large and the feature map size is small, resulting in the loss of texture information, which has a certain impact on the detection effect of texture anomalies. Destseg[4] uses resnet18[7] as the teacher-student network. Although it has an advantage in computational overhead, the extracted features are not rich enough. The most common distillation loss is cosine similarity. However, this loss can only constrain the direction of the vector at each spatial position in isolation, without considering the information around the pixel, which will lead to insufficient local consistency of feature expression and fail to fully capture the spatial contextual relationship, thus affecting the detection effect of complex anomaly patterns. Therefore, introducing structural similarity loss will help enhance local consistency and supplement spatial position information.

[0007] However, due to the variety and complex texture of supermarket product packaging, existing methods still have certain limitations in the robustness of anomaly recognition and positioning accuracy. In order to solve the above problems and further improve the detection accuracy and positioning ability, it is urgent to design a more targeted and robust anomaly detection method. Summary of the invention

[0008] The purpose of the present invention is to provide a method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining, and decompose the anomaly detection task into three stages: anomaly synthesis, feature extraction, and knowledge distillation. First, by introducing a third-party data set, local texture anomalies are generated on normal samples. Normal samples and pseudo-abnormal samples synergistically optimize the network, thereby alleviating the problem of consistent expression of the teacher-student network in abnormal areas and improving the sensitivity of the model to anomalies. Secondly, a multi-branch feature extractor is constructed as a teacher and student network, and through cross-upsampling and downsampling, feature details of different scales are integrated to integrate high-level semantic information and low-level texture information, thereby achieving more accurate feature expression and stronger feature discrimination ability. Finally, the prospect mining loss is used as a distillation loss to focus the model on prospect target mining to increase the difference between abnormal and normal prospects in teacher-student features. The structural loss part highlights the local structural relationship and improves the image-level detection performance. The network significantly enhances the sensitivity and feature discrimination ability of the model to anomalies, and shows excellent performance in both image-level detection and pixel-level positioning tasks, and is particularly suitable for high-precision anomaly detection tasks in complex scenes.

[0009] To achieve the above purpose, the technical solution of the present invention is: a method for detecting anomalies in the outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining, which decomposes the anomaly detection task into three stages: anomaly synthesis, feature extraction, and knowledge distillation, wherein:

[0010] In the abnormal synthesis stage, abnormal samples are generated through the abnormal synthesis strategy, and normal samples and abnormal samples are sent to the teacher network and the student network respectively;

[0011] In the feature extraction stage, a multi-branch feature extractor is constructed as the teacher network and the student network to perform multi-scale and multi-level feature extraction;

[0012] In the knowledge distillation stage, the teacher network’s deep understanding of normal samples is transferred to the student network, helping the student network to build accurate feature space decision boundaries, thereby achieving anomaly detection of supermarket goods outer packaging.

[0013] In one embodiment of the present invention, the abnormal synthesis strategy adopts the abnormal synthesis method of DRAEM.

[0014] In one embodiment of the present invention, the abnormal synthesis stage is specifically implemented as follows:

[0015] By randomly sampling Perlin noise and setting a threshold to generate anomaly mask M, the spatial continuity and morphological diversity of the abnormal area are ensured;

[0016] Compare the image A selected from the third-party dataset with the normal sample image I sampled from the training dataset n , the abnormal mask M and the normal mask (1-M) are combined to perform a superposition operation to generate a synthetic image containing the abnormal pattern. The synthesis formula is as follows:

[0017] I a =β(M·A)+(1-β)(M·I n )+(1-M)·I n

[0018] The control factor β is used to adjust the intensity ratio of abnormal images to normal images in the abnormal area to flexibly control the significance and fusion of abnormal patterns.

[0019] In one embodiment of the present invention, the multi-branch feature extractor includes a convolution adjustment module, a feature extraction module and a feature fusion module.

[0020] In one embodiment of the present invention, the multi-branch feature extractor is specifically implemented as follows:

[0021] The convolution adjustment module adjusts the size and channels of the input image or feature to adapt to the diversity of the input data and provide standardized feature input for subsequent processing;

[0022] The feature extraction module uses a stacking method to perform deep feature extraction based on the Basic Block structure in ResNet. It captures the complex texture, edge and semantic information of the input data through multi-layer convolution stacking and residual connection mechanism.

[0023] The feature fusion module implements upsampling and downsampling through convolution operations, so that multi-resolution features between different branches can be fused with each other;

[0024] Through the multi-branch structure design, the multi-branch feature extractor is able to comprehensively analyze the input data at multiple scales and levels.

[0025] In one embodiment of the present invention, a foreground mining loss is proposed in the knowledge distillation stage to achieve mining of foreground anomalies and supplement structural information at the same time.

[0026] In one embodiment of the present invention, the knowledge distillation stage is specifically implemented as follows:

[0027] The loss function is designed based on the cosine loss. The cosine similarity loss is defined as:

[0028]

[0029] in, is the k-th layer feature output by the teacher network, is the k-th layer feature output by the student network, (h,w) is the pixel coordinate of the feature map, M k (h,w) is the k-th two-dimensional anomaly map, H and W correspond to the height and width of the anomaly map respectively;

[0030] The anomaly exists on the foreground target, so we consider introducing the operation of stopping the gradient in the background area to make the model focus on mining the foreground anomaly and introduce the foreground cosine loss, which is defined as follows:

[0031]

[0032] where F(·) is the flatten operation, sg(·) is the stop gradient operation, μ and σ are M k The expectation and variance of (h,w); α is a hyperparameter to control the decay rate;

[0033] Considering that the cosine loss can only constrain the direction of the vector at each spatial position in isolation, without considering the information around the pixel, the loss of structural similarity is introduced to supplement the contextual structure information to enhance the model's perception of edge and texture anomalies. It is defined as follows:

[0034]

[0035] Among them, x and y are the feature maps to be compared, μ x and μ y They correspond to the mean of the image, which is used to evaluate the brightness, σ x and σ y They correspond to the variance of the image, which is used to evaluate the contrast, σ xy is the covariance of the two images, used to evaluate the structural information, c1 and c2 are two constants used to avoid the situation where the denominator is zero;

[0036] The structural similarity loss is defined as follows:

[0037]

[0038] The prospect mining loss is defined as:

[0039] L total =λ1*L fore_cos +λ2*L ssim

[0040] Among them, λ1 and λ2 are two hyperparameters used to adjust the weights of the two loss terms.

[0041] In one embodiment of the present invention, the training objective of the method is to minimize the prospect mining loss.

[0042] The present invention also provides a supermarket goods outer packaging anomaly detection system based on multi-branch feature extraction and prospect mining, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0043] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides an anomaly detection method based on multi-branch feature extraction and foreground mining. The method decomposes the anomaly detection task into three stages: anomaly synthesis, feature extraction, and knowledge distillation. By introducing third-party data to generate pseudo-anomaly samples, combined with normal samples for collaborative optimization, the sensitivity of the model to anomalies is improved. The multi-branch feature extraction module integrates multi-scale feature details with high-level semantic information to achieve accurate feature expression. The foreground mining loss focuses on the anomaly mining of foreground targets, and combines the structural loss to strengthen the local structural relationship, thereby significantly improving the image-level detection and pixel-level positioning performance, and is suitable for high-precision anomaly detection tasks in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is the network model architecture of the method of the present invention.

[0046] Figure 2 This is a module diagram of the multi-scale feature extractor of the present invention.

[0047] Figure 3 This is a heat map of the comparative experiment. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0049] The present invention should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0051] The present invention provides a method for detecting anomalies in the outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining, which decomposes the anomaly detection task into three stages: anomaly synthesis, feature extraction, and knowledge distillation.

[0052] In the abnormal synthesis stage, abnormal samples are generated through the abnormal synthesis strategy, and normal samples and abnormal samples are sent to the teacher network and the student network respectively;

[0053] In the feature extraction stage, a multi-branch feature extractor is constructed as the teacher network and the student network to perform multi-scale and multi-level feature extraction;

[0054] In the knowledge distillation stage, the teacher network’s deep understanding of normal samples is transferred to the student network, helping the student network to build accurate feature space decision boundaries, thereby achieving anomaly detection of supermarket goods outer packaging.

[0055] The following is the specific implementation process of the present invention.

[0056] like Figure 1As shown, in order to solve the incompatibility of existing anomaly detection algorithms with cargo data sets, this embodiment proposes a new anomaly detection method, an anomaly detection method based on multi-branch feature extraction and foreground mining. Among them, abnormal samples are generated through an anomaly synthesis strategy, and normal samples and abnormal samples are sent to the teacher and student networks respectively to enhance the network's perception of abnormal patterns. The multi-branch feature extraction network constitutes a teacher-student network, which can fully integrate texture information and semantic information of different scales, thereby improving the accuracy of anomaly detection. The foreground mining loss focuses on the feature extraction of the target area, and by supplementing the local structural information, it further refines the anomaly perception ability of the target area, thereby improving the detection effect of fine-grained anomalies.

[0057] 1. Model composition

[0058] like Figure 1 As shown in the figure, the model consists of three parts: anomaly synthesis, feature extraction, and knowledge distillation. First, the anomaly synthesis module generates training data containing rich anomaly information, thereby improving the student network's ability to perceive anomalies; then, in the feature extraction stage, a multi-branch structure is used to perform multi-scale and multi-level feature extraction to obtain rich texture and semantic information; finally, through the knowledge distillation module, the teacher network's deep understanding of normal samples is transferred to the student network, helping the student network to build an accurate feature space decision boundary, thereby improving the performance of anomaly detection and localization.

[0059] 2. Abnormal synthesis

[0060] In this method, the anomaly synthesis method of DRAEM[8] is used for data enhancement to improve the method's ability to perceive abnormal patterns. The specific steps are as follows:

[0061] First, the anomaly mask M is generated by randomly sampling Perlin noise and setting a threshold. This step ensures the spatial continuity and morphological diversity of the anomaly region. Second, the image A selected from a third-party dataset (such as the DTD[9] dataset) is compared with the normal sample image I sampled from the training dataset. n , combine the abnormal mask M and the normal mask (1-M) to perform a superposition operation to generate a synthetic image containing the abnormal pattern. The synthesis formula is as follows:

[0062] I a =β(M·A)+(1-β)(M·I n )+(1-M)·I n

[0063] The control factor β is used to adjust the intensity ratio of abnormal images to normal images in the abnormal area, so as to flexibly control the significance and fusion degree of the abnormal pattern.

[0064] 3. Multi-scale feature extractor

[0065] like Figure 2 As shown in the figure, the multi-branch feature extractor constituting the teacher-student network is used to extract multi-level and multi-scale feature information from the input data. The feature extractor includes functional modules of convolution adjustment, feature extraction and feature fusion. First, the convolution adjustment module adjusts the size and channel of the input image or feature to adapt to the diversity of the input data and provide standardized feature input for subsequent processing. Subsequently, the feature extraction module uses a stacking method to perform deep feature extraction based on the Basic Block structure in ResNet[7]. Through multi-layer convolution superposition and residual connection mechanism, the complex texture, edge and semantic information of the input data is captured. Finally, the feature fusion module implements upsampling and downsampling through convolution operations, so that the multi-resolution features between different branches can be fused with each other. Through the multi-branch structure design, the feature extractor can comprehensively analyze the input data at multiple scales and levels, improve the richness and robustness of feature expression, and provide better feature support for downstream tasks.

[0066] 4. Knowledge Distillation

[0067] Taking into account the shortcomings of traditional distillation that uses similarity to isolately constrain the similarity of all feature points, in order to focus on foreground target mining and consider local contextual information, the present invention proposes a foreground mining loss to achieve the mining of foreground anomalies, while supplementing structural information to improve the image-level detection effect.

[0068] First, the present invention designs a loss function based on cosine loss. Cosine similarity loss is defined as:

[0069]

[0070] in, is the k-th layer feature output by the teacher network, is the k-th layer feature output by the student network, and (h,w) is the pixel coordinate of the feature map. k (h,w) is the k-th two-dimensional anomaly map, where H and W correspond to the height and width of the anomaly map, respectively.

[0071] Anomalies exist on foreground objects, so the present invention considers introducing a gradient stop operation in the background area to make the model focus on mining foreground anomalies. The present invention introduces the foreground cosine loss, which is defined as follows:

[0072]

[0073] where F(·) is the flatten operation, sg(·) is the stop gradient operation, μ and σ are M k The expectation and variance of (h,w); α is a hyperparameter to control the decay rate.

[0074] Secondly, considering that the cosine loss can only constrain the direction of the vector at each spatial position in isolation, without considering the information around the pixel, the present invention introduces the loss of structural similarity to supplement the contextual structural information to enhance the model's perception of edge and texture anomalies, which is defined as follows:

[0075]

[0076] Among them, x and y are the feature maps to be compared, μ x and μ y They correspond to the mean of the image, which is used to evaluate the brightness, σ x and σ y They correspond to the variance of the image, which is used to evaluate the contrast, σ xy is the covariance of the two images, used to evaluate the structural information, c1 and c2 are two constants used to avoid the situation where the denominator is zero;

[0077] The structural similarity loss is defined as follows:

[0078]

[0079] Finally, the foreground mining loss is defined as:

[0080] L total =λ1*L fore_cos +λ2*L ssim

[0081] Among them, λ1=0.9, λ2=0.1, which are used to adjust the weights of the two loss terms.

[0082] 5. Training objectives

[0083] The training goal is to minimize the foreground mining loss, so that the model can focus more on mining abnormal features in the foreground target area and improve the detection accuracy by using local context information. total ,In the process of transferring knowledge from the teacher network to the student network, the integrity and consistency of the structural information are highlighted, thereby effectively improving the model's detection ability and positioning accuracy for fine-grained anomalies.

[0084] 6. Experiment

[0085] In order to evaluate the detection and localization performance of the proposed method on the goods dataset, the present invention conducted experiments on the GoodAD

[10] dataset (a total of 6124 goods images collected in supermarket scenes) and compared them with several recently published algorithms. The present invention compared the current excellent algorithms such as RD[5], RD++

[11] , IFgNet

[12] , DAF

[13] , CDO

[14] . The proposed method uses image-levelAUROC and pixel-levelAUROC as the main evaluation indicators.

[0086] 6.1 Parameter selection

[0087] The present invention follows the paradigm of anomaly detection and uniformly adjusts the images to 256×256 pixels. The official dataset has already distinguished between training and test datasets, so the present invention does not need to divide the dataset additionally. The method of the present invention uses PyTorch 2.0 on a machine equipped with a 10-core Intel Xeon CPU, 128GB of memory, and an NVIDIA RTX3090 GPU. Secondly, the model is optimized using the Adam optimizer, where the momentum control is set to (0.9, 0.999) and the learning rate is 0.002. The model was trained for 10,000 batch iterations on all categories with a batch size of 32.

[0088] 6.2 Quantitative comparison

[0089] like Figure 3 As shown, the present invention visualizes the heat maps of several comparison algorithms to intuitively reflect the effect of pixel-level positioning.

[0090] 6.3 Qualitative comparison

[0091] This paper comprehensively evaluates the anomaly detection and anomaly localization performance on the GoodsAD dataset, and compares the proposed method with existing methods (including RD[5], RD++

[11] , IFgNet

[12] , DAF

[13] , CDO

[14] , and Destseg

[15] ). The evaluation index is AUROC (%), and the results are shown in Tables 1 and 2.

[0092] Table 1 Comparison of anomaly detection results (AUROC%) on the GoodsAD dataset

[0093]

[0094] Table 2 Comparison of anomaly localization results (AUROC%) on the GoodsAD dataset

[0095]

[0096]

[0097] In the image-level detection task, as shown in Table 1, the proposed method performs well in all categories, with an average AUROC of 90.28%, which is significantly better than other methods. In the cigarette_box category, the method of the present invention achieves 100% detection accuracy. The main reason for this excellent performance is that the "open" state of the cigarette box, as an abnormal situation, shows significant structural abnormality characteristics, which enables the method of the present invention to capture this type of abnormality more accurately. In other categories, the method of the present invention also performs outstandingly, which fully proves that the introduction of structural information plays an important role in improving detection performance. This shows that the integration of structural information not only enhances the model's ability to capture fine-grained anomalies, but also improves the overall detection accuracy and robustness.

[0098] In the pixel-level localization task, as shown in Table 2, the average AUROC of the proposed method is 95.47%, which is higher than most existing methods and achieves suboptimal performance. The introduction of structural similarity leads to the deterioration of the anomaly map generated by cosine similarity, mainly due to the difference in the measurement characteristics of the two. Structural similarity focuses on capturing local structural information and enhances consistency by smoothing local differences in feature maps, which may mask the sensitivity of cosine similarity to subtle feature differences. In addition, the statistical characteristics of structural similarity interfere with feature distribution, redistribute feature weights, and weaken the significance of abnormal areas. Especially in areas with significant structural anomalies, the emphasis on local consistency may cause small-scale abnormal features to be obscured, reducing the distinguishing ability of cosine similarity. This shows that it is necessary to further optimize the combination of the two to improve the effect of anomaly maps.

[0099] In summary, the experimental results show that the proposed method has achieved significant advantages in both anomaly detection and anomaly localization tasks of the GoodsAD

[10] dataset. It not only improves the detection accuracy, but also takes into account the robustness and generalization of positioning, fully verifying its application potential and effectiveness in real scenarios.

[0100] The present invention also provides a supermarket goods outer packaging anomaly detection system based on multi-branch feature extraction and prospect mining, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0101] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0102] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0103] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0106] References:

[0107] [1]Salehi,Mohammadreza,et al."Multiresolution knowledge distillationfor anomaly detection."Proceedings ofthe IEEE / CVF conference on computervision andpattern recognition.2021.

[0108] [2]Jiang,Yuxin,Yunkang Cao,and Weiming Shen."A masked reverseknowledge distillation method incorporating global and local information forimage anomaly detection."Knowledge-Based Systems 280(2023):110982.

[0109] [3]Cao,Yunkang,et al."Collaborative discrepancy optimization forreliable image anomaly localization."IEEE Transactions on IndustrialInformatics 19.11(2023):10674-10683.

[0110] [4]Zhang,Xuan,et al."Destseg:Segmentation guided denoising student-teacher for anomaly detection."Proceedings of the IEEE / CVF Conference onComputer Vision and Pattern Recognition.2023.

[0111] [5]Deng,Hanqiu,and Xingyu Li."Anomaly detection via reversedistillation from one-class embedding."Proceedings of the IEEE / CVF conferenceon computer vision and pattern recognition.2022.

[0112] [6]Zagoruyko,Sergey."Wide residual networks."arXiv preprint arXiv:1605.07146(2016).

[0113] [7]He,Kaiming,et al."Deep residual learning for image recognition."Proceedings ofthe IEEE conference on computer vision and patternrecognition.2016.

[0114] [8]Zavrtanik,Vitjan,Matej Kristan,and Danijel "Draem-adiscriminatively trained reconstruction embedding for surface anomalydetection."Proceedings ofthe IEEE / CVF international conference on computervision.2021.

[0115] [9]Cimpoi,Mircea,et al."Describing textures in the wild."Proceedingsofthe IEEE conference on computer vision and pattern recognition.2014.

[0116]

[10] Zhang,Jian,et al."PKU-GoodsAD:A supermarket goods dataset forunsupervised anomaly detection and segmentation."IEEE Robotics andAutomationLetters(2024).

[0117]

[11] Tien,Tran Dinh,et al."Revisiting reverse distillation for anomalydetection."Proceedings of the IEEE / CVF conference on computer visionandpattern recognition.2023.

[0118]

[12] Chen,Xiaolu,et al."Implicit Foreground-Guided Network for AnomalyDetection and Localization."ICASSP 2024-2024IEEE International Conference onAcoustics,Speech and Signal Processing(ICASSP).IEEE,2024.

[0119]

[13] Cai,Yuxuan,et al."A discrepancy aware framework for robustanomaly detection."IEEE Transactions on Industrial Informatics(2023).

[0120]

[14] Cao,Yunkang,et al."Collaborative discrepancy optimization forreliable image anomaly localization."IEEE Transactions on IndustrialInformatics 19.11(2023):10674-10683.

[0121]

[15] Zhang, Xuan, et al. "Destseg: Segmentation guided denoising student-teacher for anomaly detection." Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2023.

[0122] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for detecting anomalies in outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining, characterized in that: The anomaly detection task is decomposed into three stages: anomaly synthesis, feature extraction, and knowledge distillation. In the abnormal synthesis stage, abnormal samples are generated through the abnormal synthesis strategy, and normal samples and abnormal samples are sent to the teacher network and the student network respectively; In the feature extraction stage, a multi-branch feature extractor is constructed as the teacher network and the student network to perform multi-scale and multi-level feature extraction; In the knowledge distillation stage, the teacher network’s deep understanding of normal samples is transferred to the student network, helping the student network to build accurate feature space decision boundaries, thereby achieving anomaly detection of supermarket goods outer packaging.

2. The method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining according to claim 1, characterized in that: The abnormal synthesis strategy adopts the abnormal synthesis method of DRAEM.

3. A method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining according to claim 1 or 2, characterized in that: The specific implementation of the exception synthesis stage is as follows: By randomly sampling Perlin noise and setting a threshold to generate anomaly mask M, the spatial continuity and morphological diversity of the abnormal area are ensured; Compare the image A selected from the third-party dataset with the normal sample image I sampled from the training dataset n , the abnormal mask M and the normal mask (1-M) are combined to perform a superposition operation to generate a synthetic image containing the abnormal pattern. The synthesis formula is as follows: I a =β(M·A)+(1-β)(M·I n )+(1-M)·I n The control factor β is used to adjust the intensity ratio of abnormal images to normal images in the abnormal area to flexibly control the significance and fusion of abnormal patterns.

4. The method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining according to claim 1, characterized in that: The multi-branch feature extractor includes a convolution adjustment module, a feature extraction module and a feature fusion module.

5. The method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining according to claim 4 is characterized in that: The multi-branch feature extractor is specifically implemented as follows: The convolution adjustment module adjusts the size and channels of the input image or feature to adapt to the diversity of the input data and provide standardized feature input for subsequent processing; The feature extraction module uses a stacking method to perform deep feature extraction based on the Basic Block structure in ResNet. It captures the complex texture, edge and semantic information of the input data through multi-layer convolution stacking and residual connection mechanism. The feature fusion module implements upsampling and downsampling through convolution operations, so that multi-resolution features between different branches can be fused with each other; Through the multi-branch structure design, the multi-branch feature extractor is able to comprehensively analyze the input data at multiple scales and levels.

6. The method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining according to claim 1, characterized in that: In the knowledge distillation stage, a foreground mining loss is proposed to mine foreground anomalies while supplementing structural information.

7. The method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining according to claim 6, characterized in that: The specific implementation of the knowledge distillation stage is as follows: The loss function is designed based on the cosine loss. The cosine similarity loss is defined as: in, is the k-th layer feature output by the teacher network, is the k-th layer feature output by the student network, (h,w) is the pixel coordinate of the feature map, M k (h,w) is the k-th two-dimensional anomaly map, H and W correspond to the height and width of the anomaly map respectively; The anomaly exists on the foreground target, so we consider introducing the operation of stopping the gradient in the background area to make the model focus on mining the foreground anomaly and introduce the foreground cosine loss, which is defined as follows: where F(·) is the flatten operation, sg(·) is the stop gradient operation, μ and σ are M k The expectation and variance of (h,w); α is a hyperparameter to control the decay rate; Considering that the cosine loss can only constrain the direction of the vector at each spatial position in isolation, without considering the information around the pixel, the loss of structural similarity is introduced to supplement the contextual structure information to enhance the model's perception of edge and texture anomalies. It is defined as follows: Among them, x and y are the feature maps to be compared, μ x and μ y They correspond to the mean of the image, which is used to evaluate the brightness, σ x and σ y They correspond to the variance of the image, which is used to evaluate the contrast, σ xy is the covariance of the two images, used to evaluate the structural information, c1 and c2 are two constants used to avoid the situation where the denominator is zero; The structural similarity loss is defined as follows: The prospect mining loss is defined as: L total =λ1*L fore_cos +λ2*L ssim Among them, λ1 and λ2 are two hyperparameters used to adjust the weights of the two loss terms.

8. The method for detecting anomalies of outer packaging of supermarket goods based on multi-branch feature extraction and prospect mining according to claim 7, characterized in that: The training objective of the method is to minimize the foreground mining loss.

9. A supermarket goods outer packaging anomaly detection system based on multi-branch feature extraction and prospect mining, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as claimed in any one of claims 1 to 8 can be implemented.

10. A computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, and when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 8 can be implemented.

Images