Anomaly detection method and system of normal sample feature reference
By using a dual-branch feature extraction network and bidirectional attention feature fusion, the challenges of target-background separation and anomaly difference capture in unsupervised anomaly detection are solved, thereby improving the accuracy of target localization and anomaly detection in complex scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2026-03-12
- Publication Date
- 2026-06-05
Smart Images
Figure CN122156589A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing pattern recognition technology, and in particular to an anomaly detection method and system for a normal sample feature reference. Background Technology
[0002] Anomaly detection technology aims to promptly identify abnormal targets and states that deviate from normal patterns from massive amounts of data in complex scenarios. With the rapid development of intelligent manufacturing, industrial quality inspection, and smart operation and maintenance, applications such as production line visual inspection, equipment health monitoring, and online quality control have placed higher demands on the accuracy, stability, and real-time performance of anomaly detection.
[0003] Early anomaly detection methods largely followed the supervised learning paradigm. While possessing strong engineering applicability in specific tasks, their performance heavily relied on large-scale, high-quality defect annotation data. Limited by the scarcity of defect data, acquisition costs, and privacy and confidentiality constraints of industrial data, supervised solutions were often difficult to implement in real-world scenarios. In recent years, research focus has gradually shifted to unsupervised anomaly detection: these methods typically utilize only a large number of anomaly-free samples for training to learn the normal distribution and discriminative representation of the data, and then detect and locate anomalous regions during the inference phase by assessing their deviation from the "normal pattern." Compared to supervised methods, unsupervised strategies significantly reduce reliance on anomaly annotations and have demonstrated competitive performance in various real-world scenarios.
[0004] However, existing unsupervised algorithms face two major challenges: First, insufficient generalization due to data dependence, requiring models to cover a large number of training samples, and performance easily degrades when encountering new categories and scenarios; second, the dual uncertainty of "target-anomaly," namely the uncertainty of target position, quantity, and pose, and the diversity and unpredictability of anomaly type, location, and scale; existing models are mostly designed for single-object images with pixel-level annotations, making it difficult to cope with scenarios where cross-target and anomaly variations coexist. Specific shortcomings include: first, in complex multi-target scenarios, target uncertainty makes it difficult to separate targets from the background, making them susceptible to noise interference; second, the diversity of anomalies prevents models from learning complete anomaly representations, and a single normal pattern is insufficient to capture anomaly differences; third, the separation of target localization and anomaly detection, lacking effective normal references, allows localization errors to accumulate and affect detection performance. Summary of the Invention
[0005] The purpose of this invention is to provide an anomaly detection method and system based on normal sample feature reference. By integrating the normal sample feature reference mechanism into the anomaly detection learning framework, the anomaly detection performance under conditions of scarce annotations and changing scenarios can be improved to overcome the shortcomings of the prior art.
[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides an anomaly detection method for a normal sample feature reference, characterized by comprising the following steps: S1, acquire the query image to be detected and at least one supporting image, and preprocess the query image and at least one supporting image, wherein the supporting image is an image containing only the normal state of the target; S2, input the preprocessed query image and the at least one support image into a pre-trained dual-branch feature extraction network to obtain query features and reference features corresponding to the support image; S3, input the reference features into the reference feature aggregation module to obtain reference space hybrid features and reference prototype features; S4, input the query features and the reference prototype features into the bidirectional attention feature fusion module for shared interactive fusion to obtain fused features; S5, input the fused features into the pre-trained target detector to obtain target regression parameters and target confidence scores, process them to obtain the target region location, and crop the pre-processed query image based on the target region location to obtain the target image; S6, input the target image into the dual-branch feature extraction network based on normal sample reference to obtain the target query features; S7, input the target query features and reference prototype features into the bidirectional attention feature fusion module for shared interactive fusion to obtain the target fusion features; S8, The target fusion feature and the reference space hybrid feature are jointly constructed to generate anomaly detection input features; S9, the anomaly detection input features are input into the pre-trained anomaly detector to obtain defect regression parameters and defect confidence scores, and the location of the anomaly region is obtained after processing.
[0007] Furthermore, the specific process of S3 is as follows: Global pooling is performed on the query features and the reference features to obtain query descriptors and support descriptors, and reference weights are calculated based on the similarity between the query descriptors and each support descriptor. The reference features are weighted and summed using the reference weights to generate a reference space hybrid feature and a reference prototype feature.
[0008] Furthermore, the specific process of S4 is as follows: The bidirectional attention feature fusion module includes a reference-to-query branch and a query-to-reference branch; The query features are mapped to query embeddings by referencing the query branch, and the reference prototype features are mapped to key embeddings and value embeddings. An attention relationship is established between the space of the query features and the token space of the reference prototype features to obtain reference-enhanced query features. By querying the reference branch, the reference prototype features are mapped to query embeddings, and the query features are mapped to key embeddings and value embeddings. An attention relationship between the reference prototype features and the query prototype features is established in the token space to obtain query-enhanced reference features, and the interaction results are mapped back to the query feature space. The reference-enhanced query features that are referenced to the query branch output and the results of the query-reference branch output mapped back to the query space are fused to obtain the fused features.
[0009] Furthermore, the specific process of S5 is as follows: The fused features are used as input feature maps to input a prior box-based target detector. The target detector constructs a prior box set according to the spatial size of the input feature map and predicts the target regression parameters and target confidence scores contained in each prior box. Thresholding is applied to the target confidence score to filter out low-scoring prediction boxes, and non-maximum suppression is applied to the remaining prediction boxes to remove redundancy, resulting in the final retained prediction boxes and the corresponding target regression parameters. The target regression parameters corresponding to the final retained prediction box are decoded to obtain the target region location; The preprocessed query image is cropped based on the location of the target region to obtain the target image.
[0010] Furthermore, the specific process of S8 is as follows: Calculate the difference features and absolute difference features between the target fusion features and the reference space mixing features; Calculate the inner product similarity map of the target fusion feature and the reference space hybrid feature after channel dimension normalization; The differential features, absolute differential features, and inner product similarity map are concatenated along the channel dimension to obtain the initial features; The initial features are sequentially subjected to convolutional transformation and nonlinear activation processing to obtain anomaly detection input features.
[0011] Furthermore, the dual-branch feature extraction network includes a query branch and a support branch. The query branch and the support branch have the same network structure and share parameters, and ResNet50 is used as the backbone network.
[0012] Furthermore, the number of supporting images is K, where K is an integer greater than or equal to 2; the dual-branch feature extraction network, reference feature aggregation module, bidirectional attention feature fusion module, object detector, and anomaly detector are obtained through pre-training.
[0013] Furthermore, the pre-joint training includes: Obtain a training dataset, which includes training query images and training support images. Preprocess the training query images and training support images to obtain training samples. The preprocessed training query image and training support image are input into a dual-branch feature extraction network to obtain training query features and training reference features corresponding to the training support image. The training reference features are input into the reference feature aggregation module to obtain training reference space hybrid features and training reference prototype features; The training query features and training reference prototype features are input into the bidirectional attention feature fusion module for shared interactive fusion to obtain training fusion features; the training fusion features are jointly constructed with the training reference space hybrid features to generate training anomaly detection input features; The trained fusion features are input into the target detection head to obtain a first prediction result; the trained anomaly detection input features are input into the anomaly detection head to obtain a second prediction result; loss is calculated based on the first prediction result, the second prediction result and the corresponding real label information, and backpropagation is performed based on the loss value to optimize the network parameters of the dual-branch feature extraction network, the bidirectional attention feature fusion module, the target detection head and the anomaly detection head, so as to obtain the pre-trained dual-branch feature extraction network, the target detector and the anomaly detector.
[0014] Furthermore, when the reference feature aggregation module generates reference space hybrid features and reference prototype features, it uses adaptive pooling to perform pooling processing on the reference features.
[0015] Secondly, the present invention provides an anomaly detection system based on normal sample reference, for implementing an anomaly detection method based on normal sample feature reference, comprising: The image acquisition and preprocessing module is used to acquire a query image to be detected and at least one supporting image, and to preprocess the query image and at least one supporting image, wherein the supporting image is an image containing only the normal state of the target; A dual-branch feature extraction network is used to take the preprocessed query image and the at least one support image as input, and output query features and reference features corresponding to the support image; The reference feature aggregation module is used to input the reference features and output reference space hybrid features and reference prototype features; The bidirectional attention feature fusion module is used to input the query feature and the reference prototype feature, perform shared interactive fusion, and output the fused feature; it is also used to input the target query feature and the reference prototype feature, perform shared interactive fusion, and output the target fused feature. The target detector is used to input the fused features, output target regression parameters and target confidence scores, process them to obtain the target region location, and crop the preprocessed query image based on the target region location to obtain the target image; The joint construction module is used to jointly construct the target fusion features and the reference space hybrid features to generate anomaly detection input features; An anomaly detector is used to take the anomaly detection input features as input, output defect regression parameters and defect confidence scores, and obtain the location of the anomaly region after processing.
[0016] Compared with the prior art, the present invention has the following beneficial technical effects: This invention provides an anomaly detection method based on normal sample feature reference. By constructing a target localization mechanism based on normal sample reference, a dual-branch feature extraction network is used to extract features from the query image and support images respectively. Stable reference space hybrid features and reference prototype features are generated by reference feature aggregation. Then, the query features and reference prototype features are shared and interactively fused through bidirectional attention feature fusion, so that the query features can fully perceive the semantic information of normal samples, effectively suppress background interference during target detection, and achieve accurate target localization in complex scenes. This invention performs feature extraction again on the located target image, obtaining target fusion features through bidirectional attention feature fusion. These target fusion features are then jointly constructed with reference space hybrid features to characterize the differences between the target region and the normal reference from multiple dimensions. This effectively captures the deviation of various anomalies from normal samples, achieving precise localization of anomaly regions and improving the detection capability for diverse and unknown anomaly types. This invention employs a cascaded detection strategy of first locating the target and then detecting anomalies. The target image is cropped from the query image and subjected to separate feature extraction and anomaly detection. This ensures that anomaly detection focuses only on the target region, preventing background interference and localization errors from propagating to the anomaly detection stage. Simultaneously, unified normal reference features are introduced throughout the target localization and anomaly detection process, ensuring that normal reference information permeates the entire detection workflow, significantly improving the stability and robustness of anomaly detection. Attached Figure Description Figure 1 This is a flowchart illustrating an anomaly detection method based on normal sample feature reference in an embodiment of the present invention.
[0017] Figure 2 This is a schematic diagram of the bidirectional cross-attention feature fusion module proposed in an embodiment of the present invention. Detailed Implementation
[0018] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.
[0019] The method of this invention includes a training phase and a detection phase. First, the dual-branch feature extraction network, the object detector, and the anomaly detector are pre-trained jointly. Then, the trained dual-branch feature extraction network, object detector, and anomaly detector are used to perform anomaly detection.
[0020] The pre-joint training process includes: Step 1: Obtain the training dataset, including training query images and training support images. Training query images are images containing background information and multiple targets / defects, while training support images are images containing only the normal state of the targets. For each training query image, select K normal samples from the training support images to form a K-shot support set, which serves as a reference for anomaly detection of that training query image. Specifically, the same preprocessing operation is performed on the training query image and the training support image: the original aspect ratio of the image is maintained, the short side is scaled to the target length, and then zero-value padding is performed on the lower right corner of the image. Finally, the size of the output image is uniformly 640×640, which is the preprocessed training query image and training support image. The purpose of this preprocessing operation is to ensure that the images input to the network have the same size, which is convenient for batch processing. At the same time, by maintaining the aspect ratio and zero-value padding, information loss caused by image deformation is avoided.
[0021] In this embodiment, the number of training support images is K, where K is an integer greater than or equal to 2; Step 2: Input the preprocessed training query image and K training support images into the dual-branch feature extraction network to obtain the training query features and the training reference features corresponding to the training support images.
[0022] The dual-branch feature extraction network consists of a query image feature extraction module branch and a support image feature extraction module branch, represented as follows:
[0023] satisfy:
[0024]
[0025] in, To query images, To support images, where Represented as the first Zhang supports the image; This indicates that feature extraction is performed on the query image. Parameter subnetwork; This indicates feature extraction from the supporting image. Parameter subnetwork; Indicates the dimension of the output features of the queried image; This indicates that the image output reference feature dimension is supported; Indicates the number of feature channels. =256; The purpose of this step is to use a Siamese network structure with shared parameters to map the query image and the supporting images to the same feature space.
[0026] Step 3: Input the training reference features into the reference feature aggregation module to obtain the training reference space hybrid features and the training reference prototype features. When generating the reference space hybrid features and the reference prototype features, the reference feature aggregation module uses adaptive pooling to perform pooling processing on the reference features.
[0027] The specific processing procedure of the reference feature aggregation module is as follows: The K reference features supporting image output The reference space hybrid features are obtained through the reference feature aggregation method. and reference prototype features ; The training reference features obtained in step 2 and training query features Perform global average pooling to obtain support descriptors. and query descriptors , is represented as:
[0028] use Normalization processing supports descriptors and query descriptors The calculation formula is:
[0029] Calculate the normalized query descriptor With each supporting descriptor The cosine similarity between them is used to obtain the reference weights corresponding to each reference feature by normalizing them using the Softmax function;
[0030] Using the generated reference weights The training reference features are weighted and summed to obtain the mixed features of the training reference space. At the same time, for training reference features After adaptive pooling, the same reference weights are used. We perform a weighted summation to obtain the training reference prototype features. , is represented as:
[0031]
[0032] in, , , respectively representing the query descriptor and the first One supporting descriptor Indicates global average pooling. This indicates that a custom pooling method is used, with a pooling size of [value missing]. , =14, The feature dimension representing the mixed features of the reference space. This step represents the reference prototype features. The purpose of this step is to adaptively weight and aggregate normal sample information from multiple support images to generate training reference prototype features that can stably represent the normal state. And the reference space hybrid features that preserve the spatial structure By using query features The weighting calculation based on the similarity to supporting features gives higher weights to supporting images that are more relevant to the current query image, thus enhancing the adaptability of the reference features.
[0033] Step 4, see Figure 2 The training query features Reference space mixing features and training reference prototype features The input bidirectional attention feature fusion module performs shared interactive fusion to complete the training reference features. and training query features The shared interaction and fusion are used to obtain training fusion features; Construct a bidirectional attention feature fusion module, which includes a reference-to-query branch and a query-to-reference branch: Referencing the query branch: Train query features The query embedding is obtained through a 1×1 convolutional projection block. , will train reference prototype features Key embeddings are obtained through 1×1 convolutional projection blocks. Sum embedding :
[0034]
[0035]
[0036] in, , , They respectively represent parameters , , of The convolutional projection block maps the dimension C to d, where d is the projection embedding dimension. In this embodiment, d=128.
[0037] Referencing query branches in training query features Spatial and reference prototype features Establish attention relationships between token spaces and calculate attention weight matrices. :
[0038] in, For query vector, This is the transpose of the key vector. This is the scaling factor; attention weight matrix With value embedding By weighting, we obtain the attention-weighted sequence. The feature map is then reshaped to a two-dimensional feature map, and finally upsampled to restore the spatial size of the original query features, resulting in the reference-enhanced query features. .
[0039]
[0040]
[0041] Indicates a parameter Convolutional projection blocks, convolutional projection blocks reduce the dimension from Mapped to ; Reference branch found: Train query features The query embedding is obtained through a 1×1 convolutional projection block. , will train reference prototype features Key embeddings are obtained through 1×1 convolutional projection blocks. Sum embedding :
[0042]
[0043]
[0044] in, , , , respectively representing parameters , , of Convolutional projection blocks, convolutional projection blocks reduce the dimension from Mapped to ; , , These are respectively represented as operations to flatten a two-dimensional feature map sequence, operations to restore a sequence to a two-dimensional feature map, and operations to upsample a low-resolution feature map to a high-resolution feature map; Establish training reference prototype features within the token space. With training query features The attention relationship is calculated, and the attention weight matrix is calculated. ;
[0045] attention weight matrix With value embedding By weighting, we obtain the attention-weighted sequence. The feature map is then reshaped to a two-dimensional feature map, and finally upsampled to restore the spatial size of the original query features. This results in the query-enhanced reference features mapped back to the query space. .
[0046]
[0047]
[0048] Indicates having parameters of Convolutional projection blocks, convolutional projection blocks reduce the dimension from Mapped to ; The reference-enhanced query features that reference the output of the query branch and the results mapped back to the query space from the output of the reference branch are fused to obtain the final trained fused features. ;
[0049] Step 5: Combine the final training features Features blended with training reference space Perform joint construction to generate training anomaly detection input features; In a more specific embodiment provided by the present invention, a joint construction is performed to generate the training anomaly detection input feature Z: Computational training fusion features Difference characteristics Features blended with training reference space Difference characteristics :
[0050]
[0051] Training fusion features Mixed features with training reference space Perform channel-level normalization. Normalization function. Defined as:
[0052] in, This represents normalization along the feature dimension. To prevent division by zero, C is the number of feature channels.
[0053] Training fusion features based on normalized processing Mixed features with training reference space Calculate the inner product similarity graph This is used to characterize the consistency of the semantic direction between the two.
[0054] The differential features, absolute difference features, and inner product similarity map are concatenated along the channel dimension to obtain the initial features. ;
[0055] in, This indicates that the splicing operation is performed at the channel level; The initial features are processed sequentially through two 1×1 convolution transformations and the nonlinear activation function ReLU to obtain the final training anomaly detection input features Z, which are used for the detection and localization of anomaly regions.
[0056] in, and and represent having parameters and of Convolutional transformation blocks; This represents the nonlinear activation function (ReLU).
[0057] Step 6: Input the training fusion features into the target detection head to obtain a first prediction result; input the training anomaly detection input features into the anomaly detection head to obtain a second prediction result; calculate the loss based on the first prediction result, the second prediction result and the corresponding real label information, and perform backpropagation based on the loss value to optimize the network parameters of the dual-branch feature extraction network, the bidirectional attention feature fusion module, the target detection head and the anomaly detection head, so as to obtain the pre-trained dual-branch feature extraction network, the target detector and the anomaly detector.
[0058] In a more specific embodiment provided by the present invention, the training fusion features are... Input the target detection head to obtain the first prediction result; input the training anomaly detection feature Z to obtain the second prediction result. Calculate the loss based on the first and second prediction results and the corresponding ground truth labels, and perform backpropagation based on the loss value to optimize the network parameters of the dual-branch feature extraction network, the bidirectional attention feature fusion module, the target detection head, and the anomaly detection head.
[0059] Fuse training features An anomaly detection head based on prior bounding boxes is used to predict classification and location regression parameters; the anomaly detection head first determines the feature map size. Generate a set of prior boxes And generate K prior boxes at each location; based on these, output a classification confidence map and a location offset map through classification and regression branches, as follows:
[0060] in, This represents the set of prior bounding boxes generated in the grid coordinate system. , These represent the expanded classification prediction and the location regression parameter prediction, respectively. Indicates having parameters of Convolutional blocks, , They respectively represent parameters , Depth convolutional blocks, This indicates a dimension reconstruction operation; During training, based on the obtained prior boxes With the actual target annotation box The process involves constructing a supervision signal through matching, calculating classification and regression losses, and jointly optimizing the detector and its upstream feature extraction and fusion modules via backpropagation. Specifically, this includes calculating the Intersection over Union (IoU) between each prior bounding box and the ground truth bounding box, and selecting a set of positive samples based on a threshold. A forced matching strategy is employed to ensure that each ground truth bounding box matches at least one prior bounding box. The prior bounding boxes of positive samples and their matching ground truth bounding boxes are parameterized to obtain the regression target. , is represented as:
[0061] For the positive sample set, position regression is performed. Then, a non-positive sample set with the same number of candidate boxes as the positive sample set is selected. With positive sample set The classification loss is calculated jointly; the loss function is expressed as:
[0062] in, To regress the loss, For classifying losses, Indicates the predicted regression parameters. This represents the encoded regression parameters corresponding to the positive sample candidate boxes. Represents the SmoothL1 loss function. Represents the cross-entropy loss function. Indicates the number of categories. This represents the truth label for the l-th category. This represents the corresponding predicted value, which serves as the probability of predicting the l-th category.
[0063] The above steps complete the pre-training of the dual-branch feature extraction network, object detector, and anomaly detector, resulting in the pre-trained dual-branch feature extraction network, object detector, and anomaly detector. Testing phase: See Figure 1 This invention provides an anomaly detection method based on the feature reference of a normal sample, comprising the following steps: S1, acquire the query image to be detected and at least one supporting image, and preprocess the query image and at least one supporting image, wherein the supporting image is an image containing only the normal state of the target; Perform the same preprocessing operations on the query image and support images as during the training phase: maintain the original aspect ratio of the image, scale the short side to the target length, then fill the lower right corner of the image with zero values, and output a uniform size of 640×640.
[0064] S2, the preprocessed query image and the at least one support image are input into a pre-trained dual-branch feature extraction network to obtain query features and reference features corresponding to the support images; this process is the same as step 2 in the training phase; S3, input the reference features into the reference feature aggregation module to obtain reference space hybrid features and reference prototype features; this process is the same as step 3 in the training phase. S4, input the query features and the reference prototype features of the reference space hybrid features into the bidirectional attention feature fusion module for shared interactive fusion to obtain fused features; this process is the same as step 4 in the training phase. S5, input the fused features into the pre-trained target detector to obtain target regression parameters and target confidence scores, process them to obtain the target region location, and crop the pre-processed query image based on the target region location to obtain the target image; In a more specific embodiment provided by the present invention, the specific process of S5 is as follows: The fused features are used as input feature maps to input a prior box-based target detector. The target detector constructs a prior box set according to the spatial size of the input feature map and predicts the target regression parameters and target confidence scores contained in each prior box. Thresholding is applied to the target confidence score to filter out low-scoring prediction boxes, and non-maximum suppression is applied to the remaining prediction boxes to remove redundancy, resulting in the final retained prediction boxes and the corresponding target regression parameters. The target regression parameters corresponding to the final retained prediction box are decoded to obtain the target region location; The preprocessed query image is cropped based on the location of the target region to obtain the target image.
[0065] S6, input the target image into the dual-branch feature extraction network based on normal sample reference to obtain the target query features; S7, input the target query features and reference prototype features into the bidirectional attention feature fusion module for shared interactive fusion to obtain target fusion features; process them using the same fusion method as in step S4 to obtain target fusion features.
[0066] S8, The target fusion feature and the reference space hybrid feature are jointly constructed to generate anomaly detection input features; this process is the same as step 5 in the training phase. S9 inputs the anomaly detection features into the pre-trained anomaly detector to obtain defect regression parameters and defect confidence scores. Thresholding is applied to the defect confidence scores to filter out low-scoring predicted boxes, and non-maximum suppression is then applied to the remaining predicted boxes to remove redundancy, resulting in the final retained predicted boxes and their corresponding defect regression parameters. The defect regression parameters corresponding to the final retained predicted boxes are decoded to obtain the location of the anomaly region in the original image coordinate system.
[0067] Example 1 The effectiveness of the proposed anomaly detection method based on normal sample feature reference is verified on the XDU-GAD dataset. In this embodiment, to verify the effectiveness of the proposed anomaly detection method, performance evaluation is performed on the XDU-GAD dataset.
[0068] The technical effects of the present invention will be further explained below with reference to simulation experiments: 1. Simulation conditions: All simulation experiments were conducted on the operating system Ubuntu 16.04.5, the hardware environment was an Nvidia GeForce GTX 3090 GPU, and the PyTorch deep learning framework was used. 2. Simulation content and result analysis: The results of experiments conducted on the XDU-GAD dataset using the present invention and anomaly detection methods were objectively evaluated using recognized evaluation metrics. The evaluation simulation results are shown in Table 1. Table 1 shows the experimental results on the XDU-GAD dataset.
[0069] Table 2 Experimental results of different feature fusion methods on the XDU-GAD dataset.
[0070] in: Baseline, Baseline('+'), Baseline+ho, and Baseline+ho+ha represent the base network, respectively. The base network and feature fusion process are additive operations. The base network is combined with the dual-branch attention feature fusion module of the object detection stage to form the final model of this invention.
[0071] Higher values for mAP, mIOU, AP50, AP75, and AP95 are all desirable. Table 1 shows that the model proposed in this invention outperforms state-of-the-art models on most metrics. Table 2 shows that the dual-branch cross-attention feature fusion module designed in this invention improves the fine-grained feature extraction capability for anomaly detection, effectively enhancing the overall performance of the model. This fully demonstrates the effectiveness and superiority of the method proposed in this invention.
[0072] In another embodiment of the present invention, an anomaly detection system based on normal sample reference is provided to implement the aforementioned anomaly detection method based on normal sample feature reference, comprising: The image acquisition and preprocessing module is used to acquire a query image to be detected and at least one supporting image, and to preprocess the query image and at least one supporting image, wherein the supporting image is an image containing only the normal state of the target; A dual-branch feature extraction network is used to take the preprocessed query image and the at least one support image as input, and output query features and reference features corresponding to the support image; The reference feature aggregation module is used to input the reference features and output reference space hybrid features and reference prototype features; The bidirectional attention feature fusion module is used to input the query feature and the reference prototype feature, perform shared interactive fusion, and output the fused feature; it is also used to input the target query feature and the reference prototype feature, perform shared interactive fusion, and output the target fused feature. The target detector is used to input the fused features, output target regression parameters and target confidence scores, process them to obtain the target region location, and crop the preprocessed query image based on the target region location to obtain the target image; The joint construction module is used to jointly construct the target fusion features and the reference space hybrid features to generate anomaly detection input features; An anomaly detector is used to take the anomaly detection input features as input, output defect regression parameters and defect confidence scores, and obtain the location of the anomaly region after processing.
[0073] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.
Claims
1. An anomaly detection method using a normal sample feature reference, characterized in that, Includes the following steps: S1, acquire the query image to be detected and at least one supporting image, and preprocess the query image and at least one supporting image, wherein the supporting image is an image containing only the normal state of the target; S2, input the preprocessed query image and the at least one support image into a pre-trained dual-branch feature extraction network to obtain query features and reference features corresponding to the support image; S3, input the reference features into the reference feature aggregation module to obtain reference space hybrid features and reference prototype features; S4, input the query features and the reference prototype features of the reference space hybrid features into the bidirectional attention feature fusion module for shared interactive fusion to obtain the fused features; S5, input the fused features into the pre-trained target detector to obtain target regression parameters and target confidence scores, process them to obtain the target region location, and crop the pre-processed query image based on the target region location to obtain the target image; S6, input the target image into the dual-branch feature extraction network based on normal sample reference to obtain the target query features; S7, input the target query features and reference prototype features into the bidirectional attention feature fusion module for shared interactive fusion to obtain the target fusion features; S8, The target fusion feature and the reference space hybrid feature are jointly constructed to generate anomaly detection input features; S9, the anomaly detection input features are input into the pre-trained anomaly detector to obtain defect regression parameters and defect confidence scores, and the location of the anomaly region is obtained after processing.
2. The anomaly detection method for a normal sample feature reference according to claim 1, characterized in that, The specific process of S3 is as follows: Global pooling is performed on the query features and the reference features to obtain query descriptors and support descriptors, and reference weights are calculated based on the similarity between the query descriptors and each support descriptor. The reference features are weighted and summed using the reference weights to generate a reference space hybrid feature and a reference prototype feature.
3. The anomaly detection method for a normal sample feature reference according to claim 1, characterized in that, The specific process of S4 is as follows: The bidirectional attention feature fusion module includes a reference-to-query branch and a query-to-reference branch; The query features are mapped to query embeddings by referencing the query branch, and the reference prototype features are mapped to key embeddings and value embeddings. An attention relationship is established between the space of the query features and the token space of the reference prototype features to obtain reference-enhanced query features. By querying the reference branch, the reference prototype features are mapped to query embeddings, and the query features are mapped to key embeddings and value embeddings. An attention relationship between the reference prototype features and query features is established in the token space to obtain query-enhanced reference features, and the interaction results are mapped back to the query feature space. The reference-enhanced query features that are referenced to the query branch output and the results of the query-reference branch output mapped back to the query space are fused to obtain the fused features.
4. The anomaly detection method for a normal sample feature reference according to claim 1, characterized in that, The specific process of S5 is as follows: The fused features are used as input feature maps to input a prior box-based target detector. The target detector constructs a prior box set according to the spatial size of the input feature map and predicts the target regression parameters and target confidence scores contained in each prior box. Thresholding is applied to the target confidence score to filter out low-scoring prediction boxes, and non-maximum suppression is applied to the remaining prediction boxes to remove redundancy, resulting in the final retained prediction boxes and the corresponding target regression parameters. The target regression parameters corresponding to the final retained prediction box are decoded to obtain the target region location; The preprocessed query image is cropped based on the location of the target region to obtain the target image.
5. The anomaly detection method for a normal sample feature reference according to claim 1, characterized in that, The specific process of S8 is as follows: Calculate the difference features and absolute difference features between the target fusion features and the reference space mixing features; Calculate the inner product similarity map of the target fusion feature and the reference space hybrid feature after channel dimension normalization; The differential features, absolute differential features, and inner product similarity map are concatenated along the channel dimension to obtain the initial features; The initial features are sequentially subjected to convolutional transformation and nonlinear activation processing to obtain anomaly detection input features.
6. The anomaly detection method for a normal sample feature reference according to claim 1, characterized in that, The dual-branch feature extraction network includes a query branch and a support branch. The query branch and the support branch have the same network structure and share parameters, and ResNet50 is used as the backbone network.
7. The anomaly detection method for a normal sample feature reference according to claim 1, characterized in that, The number of supporting images is K, where K is an integer greater than or equal to 2; the dual-branch feature extraction network, object detector, and anomaly detector are obtained through pre-training.
8. The anomaly detection method for a normal sample feature reference according to claim 7, characterized in that, The pre-joint training process includes: Obtain a training dataset, which includes training query images and training support images. Preprocess the training query images and training support images to obtain training samples. The preprocessed training query image and training support image are input into a dual-branch feature extraction network to obtain training query features and training reference features corresponding to the training support image. The training reference features are input into the reference feature aggregation module to obtain training reference space hybrid features and training reference prototype features; The training query features and training reference prototype features are input into the bidirectional attention feature fusion module for shared interactive fusion to obtain training fusion features; the training fusion features are jointly constructed with the training reference space hybrid features to generate training anomaly detection input features; The trained fusion features are input into the target detection head to obtain a first prediction result; the trained anomaly detection input features are input into the anomaly detection head to obtain a second prediction result; loss is calculated based on the first prediction result, the second prediction result and the corresponding real label information, and backpropagation is performed based on the loss value to optimize the network parameters of the dual-branch feature extraction network, the bidirectional attention feature fusion module, the target detection head and the anomaly detection head, so as to obtain the pre-trained dual-branch feature extraction network, the target detector and the anomaly detector.
9. The anomaly detection method for a normal sample feature reference according to claim 1, characterized in that, When the reference feature aggregation module generates reference space hybrid features and reference prototype features, it uses adaptive pooling to perform pooling processing on the reference features.
10. An anomaly detection system based on normal sample references, characterized in that, An anomaly detection method for implementing a normal sample feature reference as described in any one of claims 1-9 includes: The image acquisition and preprocessing module is used to acquire a query image to be detected and at least one supporting image, and to preprocess the query image and at least one supporting image, wherein the supporting image is an image containing only the normal state of the target; A dual-branch feature extraction network is used to take the preprocessed query image and the at least one support image as input, and output query features and reference features corresponding to the support image; The reference feature aggregation module is used to input the reference features and output reference space hybrid features and reference prototype features; The bidirectional attention feature fusion module is used to input the query feature and the reference prototype feature, perform shared interactive fusion, and output the fused feature; it is also used to input the target query feature and the reference prototype feature, perform shared interactive fusion, and output the target fused feature. The target detector is used to input the fused features, output target regression parameters and target confidence scores, process them to obtain the target region location, and crop the preprocessed query image based on the target region location to obtain the target image; The joint construction module is used to jointly construct the target fusion features and the reference space hybrid features to generate anomaly detection input features; An anomaly detector is used to take the anomaly detection input features as input, output defect regression parameters and defect confidence scores, and obtain the location of the anomaly region after processing.