Feature channel selection-based anomaly detection few-sample fine tuning method

Abstract

The invention discloses an anomaly detection few-sample fine tuning method based on feature channel selection, and the method comprises the steps: screening out a feature channel with discrimination capability for anomaly detection through calculating the similarity and variance of positive and abnormal samples on the feature channel based on a multi-modal model such as CLIP, and discarding an interference channel; therefore, the accuracy of anomaly detection and positioning is improved. According to the method, only a small number of normal and abnormal samples are finely adjusted, the model structure is not changed, additional training parameters are not needed, and the method is suitable for an abnormal detection task with scarce samples and high labeling cost in an industrial scene. Experiments show that the method is excellent in performance on a fabric anomaly detection data set and has high practicability and generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of industrial image anomaly detection, specifically relating to a method, apparatus, and medium for fine-tuning with a small number of samples by selecting image feature channels. Background Technology

[0002] In industrial production, anomaly detection and localization of product images are crucial for ensuring product quality. Traditional image anomaly detection typically requires a large number of labeled samples for training, but obtaining a large number of high-quality labeled samples in actual industrial production is often costly and time-consuming. In recent years, multimodal large models have demonstrated strong generalization and zero-shot learning capabilities, allowing application to downstream tasks with only a small number of samples for fine-tuning, significantly reducing reliance on large amounts of labeled data. While existing methods have improved anomaly detection and localization capabilities to some extent, the detection accuracy still cannot fully meet the stringent requirements of industrial production. Furthermore, the high training cost and time requirements of large models pose a significant obstacle for resource-constrained factories and enterprises. Summary of the Invention

[0003] The purpose of this invention is to improve and enhance the performance of existing research and technologies by proposing a few-sample fine-tuning method for anomaly detection based on feature channel selection.

[0004] The objective of this invention is achieved through the following technical solution: A few-shot fine-tuning method for anomaly detection based on feature channel selection includes the following steps: Step 1: Select the CLIP model with ViT as the visual feature extractor. The self-attention calculation mechanism of ViT is changed from the original QK to VV self-attention calculation. Step 2: The CLIP model determines whether the image to be inspected is abnormal according to the CLIP model's own classification method. For the localization of abnormal regions, the block features output by the intermediate layer are compared with the text features. Step 3: Extract graph features from a small number of normal and abnormal image samples to obtain normal graph feature sets and abnormal graph feature sets. Extract block features from the intermediate layer involved in abnormal region localization to obtain normal block feature sets and abnormal block feature sets. Step 4: Along the feature channel dimension, calculate the average similarity within the set, the variance within the set, and the average similarity between sets for the normal image feature set and the abnormal image feature set, as well as the average similarity within the set, the variance within the set, and the average similarity between sets for the normal block feature set and the abnormal block feature set. Select feature channels based on the average similarity within the set and the variance within the set to obtain the set of channels to be discarded. Step 5: The channels that need to be discarded are not included in the similarity calculation between image features and text features. Calculate the probability that the image belongs to a positive anomaly, calculate the probability that the image patch belongs to a positive anomaly region, and finally, interpolate the image patch anomaly map to the size of the original image through bilinear interpolation to obtain the probability that each pixel belongs to a positive anomaly region.

[0005] Further, step 2 includes: calculating the cosine similarity between the image features and normal text features and abnormal text features respectively to determine whether the image is abnormal. If the similarity with normal text features is high, the image to be tested is a normal image; otherwise, it is an abnormal image. For the localization of abnormal regions, the block features output by the intermediate layer are compared with the text features. If the similarity with normal text features is high, the image block belongs to the normal region; otherwise, it is an abnormal region.

[0006] Furthermore, the graph feature extraction in step 3 specifically involves: given an unknown image Visual encoders for images After encoding, the final output is a feature vector representing the entire image. , dimension Different ViT extractors have different feature dimensions. Given the number of normal image samples for few-sample fine-tuning. and the number of abnormal image samples ,get Normalized normal graph feature set and anomaly graph feature set In this context, i indicates that the feature is a graphical feature, n represents normal, and a represents abnormal. Indicates the first Graphical features of a normal sample , Indicates the first Graph features of anomaly samples , express The shape is , express The shape is .

[0007] Furthermore, the block feature extraction in step 3 specifically involves: the visual encoder processing the image... The output of a certain intermediate layer during encoding is , dimension p indicates that the feature is a block feature. The number of blocks in each layer depends on the input image pixels and the block size that ViT divides the image into when processing it, selecting the intermediate layers involved in anomaly region localization. ,in This represents the k-th intermediate layer, for a single... Extract normal block feature set from all normal samples ,in, Let p represent the i-th normal image, p indicate that the feature is a block feature, and n represent normal. Let N represent the Nth normal image. For all abnormal samples, the pixel positions of the abnormal regions are obtained based on the true label mask of the abnormal samples. Then, the index of the corresponding block is found, and the feature set of the abnormal blocks is extracted. ,in, Let 'i' be the i-th abnormal image, and 'a' represent the abnormality. This represents the Nth abnormal image. This represents the number of blocks corresponding to the abnormal regions in all abnormal images.

[0008] Furthermore, step 4 includes the following feature channel selection strategy: select k channels such that the weighted sum of the average cosine similarity of samples of the same class across these k channels and the variance of the feature values ​​within the same channel is lower than a given value. The average cosine similarity of samples from different classes across these k channels is lower than a given value. ; In practice, the following operations are performed on graph features and block features respectively: the mean cosine similarity and variance of the feature channel directions are calculated for the feature combinations of normal samples, abnormal samples and positive abnormal samples respectively, and three feature channel sets are selected accordingly. Then, the union of these three feature channel sets is taken as the final channel selection.

[0009] Furthermore, in step 4, the method for calculating the mean cosine similarity of graph features is as follows: For a normal graph feature set... Anomaly graph feature set Feature set of positive anomaly graph Calculate the average similarity along the feature channel direction: , , , In the formula, , and Each refers to , and Average similarity along the k-th feature channel and These refer to the number of normal image samples and the number of abnormal images, respectively. The graph features of the m-th sample along the k-th channel , , and Each refers to , and The average similarity set of all feature channels Embedding dimension for each feature; The method for calculating the variance of graph features is as follows: For a normal set of graph features... and anomaly graph feature set Calculate the variance along the characteristic channel direction: , , In the formula, and Each refers to and The variance along the k-th feature channel, and These refer to the number of normal image samples and the number of abnormal images, respectively. The graph features of the m-th sample along the k-th channel , This refers to the average value of all sample features along the k-th channel. and Each refers to and The set of variances along all characteristic channels. Embedding dimension for each feature.

[0010] Furthermore, in step 4, selecting three feature channel sets for the graph features specifically involves: from From each feature dimension, q channels need to be selected to be discarded. For any channel k, the normal image feature score is defined. and anomaly graph feature scores : , , Define the set of normal scores as the weighting factor. and abnormal score set Sort the channels from smallest to largest, and select the first q channels to obtain the set of normal channels. and abnormal channel set This ensures that any score within the set is higher than a given value. Average similarity for combinations of positive abnormal features Sort the channels from largest to smallest, and select the first q channels to obtain the mixed channel set. This ensures that any score within the set is higher than a given value. ; This is the final set of graph feature channels to be discarded. if If the quantity is too small or zero, select or .

[0011] Furthermore, in step 4, the method for calculating the average cosine similarity of block features is as follows: For a normal block feature set... , anomaly block feature set and positive anomaly block feature set Calculate the average similarity along the feature channel direction: , , , In the formula, , and Each refers to , and Average similarity along the k-th feature channel The block feature of the m-th sample along the k-th channel , , and Each refers to , and The average similarity set of all feature channels Embedding dimension for each feature; The method for calculating the variance of block features is as follows: For a normal graph feature set... and anomaly graph feature set Calculate the variance along the characteristic channel direction: , , In the formula, and Each refers to and The variance along the k-th feature channel, The block feature of the m-th sample along the k-th channel , This refers to the average value of all sample block features along the k-th channel. and Each refers to and The set of variances along all characteristic channels. Embedding dimension for each feature.

[0012] Furthermore, in step 4, selecting three feature channel sets for the block features specifically involves: from From each feature dimension, q channels need to be selected to be discarded. For any channel k, the normal block feature score is defined. and anomaly block feature score : , , Define the set of normal scores as the weighting factor. and abnormal score set Sort the channels from smallest to largest, and select the first q channels to obtain the set of normal channels. and abnormal channel set This ensures that any score within the set is higher than a given value. Average similarity for combinations of positive abnormal features Sort the channels from largest to smallest, and select the first q channels to obtain the mixed channel set. This ensures that any score within the set is higher than a given value. ; This is the final set of discarded block signature channels; if If the quantity is too small or zero, select or .

[0013] Further, step 5 specifically involves: in the actual matrix multiplication, assigning zeros to the corresponding channels of the graph features / block features; for image classification of positive anomalies, assigning zeros to the graph features belonging to... After assigning zeros to the channels, cosine similarity is calculated with the two text features respectively. The two similarities are then processed by softmax to obtain the probability that the image belongs to a positive anomaly. For anomaly region localization, a certain intermediate layer... Among all block features belonging to After assigning zeros to the channels, cosine similarity is calculated with the two text features respectively. The two similarities are then processed by softmax to obtain the probability that the image patch belongs to a positive anomalous region; one layer Corresponding to an anomaly diagram , For a certain image patch The probability of belonging to an abnormal region; for all The final anomaly diagram is as follows , Then, by extending the input image size through bilinear interpolation, the probability of each pixel in the original image belonging to an abnormal region is obtained.

[0014] The beneficial effects of this invention are mainly reflected in the following three aspects:

[0015] (1) High detection accuracy: Through the feature channel selection mechanism, interference features that are irrelevant to the anomaly judgment are effectively filtered out, enhancing the model's ability to distinguish abnormal features, so that high-precision anomaly detection and localization can still be achieved with a small number of samples.

[0016] (2) Low sample dependence: Only a very small number of normal and abnormal samples are needed to complete the fine-tuning, which significantly reduces the dependence on large-scale labeled data and is suitable for practical applications in industrial scenarios where sample acquisition is difficult and labeling costs are high.

[0017] (3) High computational efficiency: No need to change the model structure or fine-tune the parameters. Performance improvement is achieved only by masking the feature channels, avoiding additional training overhead. The inference speed is almost unaffected and it is easy to deploy and apply in actual production environments. Attached Figure Description

[0018] Figure 1 This is the overall flowchart of the present invention.

[0019] Figure 2 This is a flowchart of the feature channel selection process in step 3 of the present invention. Detailed Implementation

[0020] The specific implementation method and working principle of the present invention will be described in detail below with reference to the accompanying drawings:

[0021] Please see Figure 1 A few-sample fine-tuning method for anomaly detection based on feature channel selection includes the following steps:

[0022] Step 1: Select the CLIP model that uses ViT as the visual feature extractor. You can choose a model whose parameters have been trained on a publicly available anomaly detection dataset. The self-attention calculation mechanism of ViT has been changed from the original QK to VV self-attention calculation.

[0023] Step 2: The CLIP model determines whether the image to be inspected is abnormal according to the CLIP model's own classification method. For the localization of abnormal regions, it compares the block features output from the intermediate layer with the text features. Specifically:

[0024] The image features (image-level features) are compared with normal text features and abnormal text features respectively to calculate cosine similarity to determine whether the image is abnormal. If the similarity with normal text features is high, the image to be tested is a normal image, otherwise it is an abnormal image. For the localization of abnormal regions, the block features output by the intermediate layer are compared with the text features. If the similarity with normal text features is high, the image block is a normal region, otherwise it is an abnormal region.

[0025] Step 3: Perform image-level feature extraction, i.e., graph feature extraction, on a small number of normal and abnormal image samples to obtain a normal graph feature set. and abnormal sample image feature set Specifically:

[0026] Given an unknown image q, the visual encoder processes the image... After encoding, the final output is a feature vector representing the entire image. , dimension Different ViT extractors have different feature dimensions. Given the number of normal image samples for few-sample fine-tuning. and the number of abnormal image samples ,get Normalized (L2Normalization) normal graph feature set and anomaly graph feature set Where i indicates that the feature is a graphical feature, n represents normal, and a represents abnormal. Indicates the first Image features of a normal sample , Indicates the first Image features of anomaly samples , express The shape is , express The shape is .

[0027] For all intermediate layers involved in anomaly region localization , where l k This indicates that the k-th intermediate layer performs patch feature extraction to obtain the normal sample patch feature set. and abnormal sample block feature set Specifically:

[0028] The output of an intermediate layer in the visual encoder that encodes image i is , dimension Where p indicates that the feature is a block feature. The number of patches for each layer depends on the input image pixels and the patch size that ViT divides the image into during processing. This determines the intermediate layers selected for anomaly region localization. For a single Extract normal block feature set from all normal samples ,in, This represents the i-th normal image, where n represents normal. This represents the Nth normal image. For all abnormal samples, since each patch represents the intermediate layer features of a certain pixel set in the original image, and these patches correspond one-to-one with the pixel regions in the original image, the pixel positions of the abnormal regions are obtained based on the true label mask of the abnormal samples. Then, the index of the corresponding patch is found, and the abnormal patch feature set is extracted. ,in, Let N represent the i-th abnormal image, where a represents the abnormality. a This represents the Nth abnormal image. This represents the number of blocks corresponding to the abnormal regions in all abnormal images.

[0029] Step 4: Along the feature channel dimension, process the normal image feature set. Anomaly graph feature set Calculate the average similarity and variance for the feature set of positive anomalies. Calculate the average similarity, perform feature channel selection on the three feature sets, and take the union of the results to obtain the set of channels to be discarded. Specifically:

[0030] The feature channel selection strategy is as follows: Select k channels such that the weighted sum of the average cosine similarity of samples of the same class across these k channels and the variance of the feature values ​​within the same channel is lower than a given value. The average cosine similarity of samples from different classes across these k channels is lower than a given value. .

[0031] In practice, the following operations are performed on graph features and block features respectively: calculate the average similarity and variance of the feature channel directions for the feature combinations of normal samples, abnormal samples and positive abnormal samples, and select three feature channel sets accordingly. Then, the union of these three feature channel sets is used as the final channel selection.

[0032] Specifically, for the normal graph feature set Anomaly graph feature set Feature set of positive anomaly graph First, calculate the average similarity of samples along the feature channel direction:

[0033]

[0034]

[0035] In the formula, , and Each refers to , and Average similarity along the k-th feature channel and These refer to the number of normal image samples and the number of abnormal images, respectively. The graph features of the m-th sample along the k-th channel , , and Each refers to , and The average similarity set of all feature channels Embedding dimension for each feature.

[0036] Then calculate the variance of the sample along the feature channel direction:

[0037]

[0038] In the formula, and Each refers to and The variance along the k-th feature channel, and These refer to the number of normal image samples and the number of abnormal images, respectively. The graph features of the m-th sample along the k-th channel , This refers to the average value of all sample features along the k-th channel. and Each refers to and The set of variances along all characteristic channels. Embedding dimension for each feature.

[0039] In order to From the ... and anomaly graph feature scores :

[0040]

[0041] in, As weighting factors. Define the set of normal scores. and abnormal score set Sort the channels from smallest to largest, and select the first q channels to obtain the set of normal channels. and abnormal channel set This ensures that any score within the set is higher than a given value. Average similarity for combinations of positive abnormal features Sort the channels from largest to smallest, and select the first q channels to obtain the mixed channel set. This ensures that any score within the set is higher than a given value. ; This is the final set of graph feature channels to be discarded.

[0042] if If the quantity is too small or zero, select or .

[0043] For the normal block feature set , anomaly block feature set and positive anomaly block feature set Calculate the average similarity along the feature channel direction:

[0044]

[0045]

[0046]

[0047] In the formula, , and Each refers to , and Average similarity along the k-th feature channel The block feature of the m-th sample along the k-th channel , , and Each refers to , and The average similarity set of all feature channels Embedding dimension for each feature.

[0048] For the normal graph feature set and anomaly graph feature set Calculate the variance along the characteristic channel direction:

[0049]

[0050] In the formula, and Each refers to and The variance along the k-th feature channel, The block feature of the m-th sample along the k-th channel , This refers to the average value of all sample block features along the k-th channel. and Each refers to and The set of variances along all characteristic channels. Embedding dimension for each feature.

[0051] In order to From each feature dimension, q channels need to be selected to be discarded. For any channel k, the normal block feature score is defined. and anomaly block feature score :

[0052]

[0053] As weighting factors. Define the set of normal scores. and abnormal score set Sort the channels from smallest to largest, and select the first q channels to obtain the set of normal channels. and abnormal channel set This ensures that any score within the set is higher than a given value. Average similarity for combinations of positive abnormal features Sort the channels from largest to smallest, and select the first q channels to obtain the mixed channel set. This ensures that any score within the set is higher than a given value. ; This is the final set of discarded block signature channels;

[0054] if If the quantity is too small or zero, select or .

[0055] Step 5: The channels selected for rejection are not included in the similarity calculation between image features and text features. The probability of an image belonging to a positive anomalous region is calculated, and the probability of an image patch belonging to a positive anomalous region is calculated. Finally, the image patch anomalous map is bilinearly interpolated to the size of the original image to obtain the probability of each pixel belonging to a positive anomalous region. Specifically:

[0056] In actual matrix multiplication, the corresponding channels of graph features / block features are assigned zeros. As described in step (2), for image classification of positive anomalies, the graph features belonging to... After assigning zeros to the channels, cosine similarity is calculated with the two text features respectively. The two similarities are then processed by softmax to obtain the probability that the image belongs to a positive anomaly. For anomaly region localization, a certain intermediate layer... Among all block features belonging to After assigning zeros to the channels, cosine similarity is calculated with the two text features respectively. The two similarities are then processed by softmax to obtain the probability that the image patch belongs to a positive anomalous region; one layer Corresponding to an anomaly diagram , For a certain image patch The probability of belonging to an abnormal region; for all The final anomaly diagram is as follows , Then, by extending the input image size through bilinear interpolation, the probability of each pixel in the original image belonging to an abnormal region is obtained.

[0057] Example:

[0058] This embodiment processes fabric samples obtained from a factory. The fabric is illuminated with a flashlight and photographed with a mobile phone. Abnormal samples are annotated at the pixel level to obtain a dataset.

[0059] (1) The CLIP model with ViT-L / 14@334px as the visual feature extractor was selected. ViT-L has 24 layers, and the self-attention calculation mechanism of the last 20 layers was changed from the original QK to VV self-attention calculation. Learnable parameters were added before the normal and abnormal text features, and these parameters were pre-trained using the public fabric dataset ZJU-Leaper. The image input size was 518.

[0060] (2) CLIP determines whether the image to be inspected is abnormal according to CLIP's own classification method. For the localization of abnormal regions, it compares the block features output by the intermediate layers with the text features. Layers 6, 12, 18, and 24 are selected as the intermediate layers for abnormal region localization. The overall process is shown in the attached figure. Figure 1 As shown.

[0061] (3) Randomly select samples from the dataset, choosing 5 normal images and 5 abnormal images. Perform image-level feature extraction, and obtain the feature set of normal sample images from the final output of the image encoder. and abnormal sample image feature set .

[0062] (4) As attached Figure 2 As shown, along the feature channel dimension, the feature set of normal sample images... Abnormal sample image feature set Calculate the average similarity and variance for the feature set of positive outliers. Calculate the average similarity. Calculate the score. and And reorder them from smallest to largest. Sort the channels from largest to smallest, then take the first 153 channels and find their union. Since the number of channels in the union is relatively small, we ultimately select... and The union of the two sets yields the set of discarded channels. .

[0063] (5) Obtain the pixel positions of the abnormal regions based on the real label mask corresponding to the abnormal samples. Since the image input size is... The block size of the image processed by the model is Therefore, the number of blocks is For all pixel coordinates, divide by 14 and round down, then multiply the row index by 37 and add the column index to obtain the corresponding patch index (the patch index is one-dimensional). Block feature extraction is performed on layers 6, 12, 18, and 24 to obtain the normal sample block feature set. and abnormal sample block feature set .

[0064] (6) The selection strategy and actual operation are the same as in step (4), performing feature channel selection for image patch-level features. For all Calculations yielded The final index of the discarded channels is shown in Table 1.

[0065] Table 1

[0066] Channels selected for discard are assigned a value of 0. All samples are tested, and the cosine similarity between image and text features is calculated, followed by softmax to determine whether the image is normal or abnormal. Similarly, the cosine similarity between block features and text features is calculated, followed by softmax to determine the probability that an image block belongs to a positive abnormal region. Finally, the image block anomaly map is bilinearly interpolated to the size of the original image to obtain the probability of each pixel belonging to a positive abnormal region. The anomaly region probability matrix is ​​normalized and a heatmap is plotted, as shown in the attached image. Figure 1 As shown in the rightmost image, the closer an area is to red, the higher the probability that it is an abnormal area; the closer it is to blue, the more normal the area is.

[0067] Although this example is based on the CLIP demonstration, this invention is applicable to all models that use ViT as the backbone for image feature extraction. This invention does not involve changes to the model structure or parameter fine-tuning, therefore it does not increase training costs or runtime, making it easy to implement and highly applicable.

[0068] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0069] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for fine-tuning anomaly detection with few samples based on feature channel selection, characterized in that, Includes the following steps: Step 1: Select the CLIP model with ViT as the visual feature extractor. The self-attention calculation mechanism of ViT is changed from the original QK to VV self-attention calculation. Step 2: The CLIP model determines whether the image to be inspected is abnormal according to the CLIP model's own classification method. For the localization of abnormal regions, the block features output by the intermediate layer are compared with the text features. Step 3: Extract graph features from a small number of normal and abnormal image samples to obtain normal graph feature sets and abnormal graph feature sets. Extract block features from the intermediate layer involved in abnormal region localization to obtain normal block feature sets and abnormal block feature sets. Step 4: Along the feature channel dimension, calculate the average similarity within the set, the variance within the set, and the average similarity between sets for the normal image feature set and the abnormal image feature set, as well as the average similarity within the set, the variance within the set, and the average similarity between sets for the normal block feature set and the abnormal block feature set. Select feature channels based on the average similarity within the set and the variance within the set to obtain the set of channels to be discarded. Step 5: The channels that need to be discarded are not included in the similarity calculation between image features and text features. Calculate the probability that the image belongs to a positive anomaly, calculate the probability that the image patch belongs to a positive anomaly region, and finally, interpolate the image patch anomaly map to the size of the original image through bilinear interpolation to obtain the probability that each pixel belongs to a positive anomaly region.

2. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 1, characterized in that, Step 2 includes: calculating the cosine similarity between the image features and normal text features and abnormal text features respectively to determine whether the image is abnormal. If the similarity with normal text features is high, the image to be tested is a normal image; otherwise, it is an abnormal image. For the localization of abnormal regions, the block features output by the intermediate layer are compared with the text features. If the similarity with normal text features is high, the image block is a normal region; otherwise, it is an abnormal region.

3. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 1, characterized in that, The graph feature extraction in step 3 specifically involves: given an unknown image Visual encoders for images After encoding, the final output is a feature vector representing the entire image. , dimension Different ViT extractors have different feature dimensions. ; Given the number of normal image samples for few-sample fine-tuning and the number of abnormal image samples ,get Normalized normal graph feature set and anomaly graph feature set In this context, i indicates that the feature is a graphical feature, n represents normal, and a represents abnormal. Indicates the first Graphical features of a normal sample , Indicates the first Graph features of anomaly samples , express The shape is , express The shape is .

4. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 1, characterized in that, The block feature extraction in step 3 specifically involves: the visual encoder processing the image... The output of a certain intermediate layer during encoding is , dimension p indicates that the feature is a block feature. The number of blocks in each layer depends on the input image pixels and the block size that ViT divides the image into when processing it, selecting the intermediate layers involved in anomaly region localization. ,in This represents the k-th intermediate layer, for a single... Extract normal block feature set from all normal samples ,in, Let p represent the i-th normal image, p indicate that the feature is a block feature, and n represent normal. Let N represent the Nth normal image. For all abnormal samples, the pixel positions of the abnormal regions are obtained based on the true label mask of the abnormal samples. Then, the index of the corresponding block is found, and the feature set of the abnormal blocks is extracted. ,in, Let 'i' be the i-th abnormal image, and 'a' represent the abnormality. This represents the Nth abnormal image. This represents the number of blocks corresponding to the abnormal regions in all abnormal images.

5. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 1, characterized in that, Step 4 includes the following feature channel selection strategy: select k channels such that the weighted sum of the average cosine similarity of samples of the same class on these k channels and the variance of the feature values ​​within the same channel is lower than a given value. The average cosine similarity of samples from different classes across these k channels is lower than a given value. ; In practice, the following operations are performed on graph features and block features respectively: the mean cosine similarity and variance of the feature channel directions are calculated for the feature combinations of normal samples, abnormal samples and positive abnormal samples respectively, and three feature channel sets are selected accordingly. Then, the union of these three feature channel sets is taken as the final channel selection.

6. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 5, characterized in that, In step 4, the method for calculating the mean cosine similarity of graph features is as follows: For a normal graph feature set... Anomaly graph feature set Feature set of positive anomaly graph Calculate the average similarity along the feature channel direction: ， ， ， In the formula, , and Each refers to , and Average similarity along the k-th feature channel and These refer to the number of normal image samples and the number of abnormal images, respectively. The graph features of the m-th sample along the k-th channel , , and Each refers to , and The average similarity set of all feature channels Embedding dimension for each feature; The method for calculating the variance of graph features is as follows: For a normal set of graph features... and anomaly graph feature set Calculate the variance along the characteristic channel direction: ， ， In the formula, and Each refers to and The variance along the k-th feature channel, and These refer to the number of normal image samples and the number of abnormal images, respectively. The graph features of the m-th sample along the k-th channel , This refers to the average value of all sample features along the k-th channel. and Each refers to and The set of variances along all characteristic channels. Embedding dimension for each feature.

7. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 5, characterized in that, In step 4, the selection of three feature channel sets for the graph features is specifically as follows: from From each feature dimension, q channels need to be selected to be discarded. For any channel k, the normal image feature score is defined. and anomaly graph feature scores : ， ， Define the set of normal scores as the weighting factor. and abnormal score set Sort the channels from smallest to largest, and select the first q channels to obtain the set of normal channels. and abnormal channel set This ensures that any score within the set is higher than a given value. Average similarity for combinations of positive abnormal features Sort the channels from largest to smallest, and select the first q channels to obtain the mixed channel set. This ensures that any score within the set is higher than a given value. ; This is the final set of graph feature channels to be discarded. if If the quantity is too small or zero, select or .

8. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 5, characterized in that, In step 4, the method for calculating the mean cosine similarity of block features is as follows: For a normal block feature set... , anomaly block feature set and positive anomaly block feature set Calculate the average similarity along the feature channel direction: ， ， ， In the formula, , and Each refers to , and Average similarity along the k-th feature channel The block feature of the m-th sample along the k-th channel , , and Each refers to , and The average similarity set of all feature channels Embedding dimension for each feature; The method for calculating the variance of block features is as follows: For a normal graph feature set... and anomaly graph feature set Calculate the variance along the characteristic channel direction: ， ， In the formula, and Each refers to and The variance along the k-th feature channel, The block feature of the m-th sample along the k-th channel , This refers to the average value of all sample block features along the k-th channel. and Each refers to and The set of variances along all characteristic channels. Embedding dimension for each feature.

9. A method for fine-tuning anomaly detection with few samples based on feature channel selection according to claim 5, characterized in that, In step 4, the selection of three feature channel sets for the block features specifically involves: from From each feature dimension, q channels need to be selected to be discarded. For any channel k, the normal block feature score is defined. and anomaly block feature score : ， ， Define the set of normal scores as the weighting factor. and abnormal score set Sort the channels from smallest to largest, and select the first q channels to obtain the set of normal channels. and abnormal channel set This ensures that any score within the set is higher than a given value. Average similarity for combinations of positive abnormal features Sort the channels from largest to smallest, and select the first q channels to obtain the mixed channel set. This ensures that any score within the set is higher than a given value. ; This is the final set of discarded block signature channels; if If the quantity is too small or zero, select or .

10. The anomaly detection few-sample fine-tuning method based on feature channel selection according to claim 1, characterized in that, Step 5 specifically involves: in the actual matrix multiplication, the corresponding channels of the graph features / block features are assigned zeros. For image classification of positive anomalies, the graph features belonging to... After assigning zero to the channel, cosine similarity is calculated with the two text features respectively. The two similarities are then processed by softmax to obtain the probability that the image belongs to a positive anomaly. For locating abnormal regions, a certain intermediate layer Among all block features belonging to After assigning zeros to the channels, cosine similarity is calculated with the two text features respectively. The two similarities are then processed by softmax to obtain the probability that the image patch belongs to a positive anomalous region; one layer Corresponding to an anomaly diagram , For a certain image patch The probability of belonging to an abnormal region; for all The final anomaly diagram is as follows , Then, by extending the input image size through bilinear interpolation, the probability of each pixel in the original image belonging to an abnormal region is obtained.

Images