Memory bank-based small sample industrial anomaly detection method and device

Abstract

The application discloses a small sample industrial anomaly detection method based on a memory bank, comprising the following steps: performing feature extraction on a plurality of support images, constructing a semantic memory bank, and connecting the semantic memory bank with a situational memory bank to form a total memory bank; wherein the semantic memory bank comprises semantic features of the plurality of support images; performing online registration on situational features of a current query image and the semantic features in the semantic memory bank based on an online CT theory to obtain a best matching flow; and performing situational memory calibration in combination with the situational features of the current query image to update the situational memory bank and then update the total memory bank; performing registration on the situational features of the current query image and the features in the updated total memory bank based on the online CT theory, and obtaining an anomaly score map of the current query image according to a registration result, and realizing anomaly detection of the current query image based on the anomaly score map. The method uses CT as a measurement for online detection, realizes accurate feature matching, and improves detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial detection technology, and particularly relates to a small sample industrial anomaly detection method and device based on a memory bank. BACKGROUND

[0002] Industrial anomaly detection is a key process monitoring technology in industrial manufacturing, aiming to identify and diagnose abnormal situations on the production line to ensure the stability of the production process and the quality of the products. Existing industrial anomaly detection is mainly divided into supervised and unsupervised types. The fragmentation of industrial anomalies, such as fine scratches and scale-like obvious damage, brings difficulties to supervised industrial anomaly detection. Therefore, unsupervised industrial anomaly detection has been developed in recent years, which only uses normal industrial images of each product to train the model, and then uses the trained model to detect abnormal industrial images and accurately locate the corresponding abnormal area.

[0003] The research of unsupervised industrial anomaly detection can be divided into two categories, namely, reconstruction-based models and memory-based models. Reconstruction-based models use deep generative models (DGM) to learn continuous or discrete feature representations of normal images through a tedious training process. Memory-based models usually use a memory bank to extend DGM to record normal features of normal training images. During testing, feature matching between the test image and the memory is usually performed. However, both types of models are heavily dependent on a large number of normal training images and cannot quickly generalize across products under low data volume conditions.

[0004] Inspired by human detection anomalies, few-shot learning (FSL) is introduced into industrial anomaly detection (IAD) to learn a general model that can be shared among multiple products, and can also be extended to new products that only provide limited normal training images, such as providing only 1 or 2 samples per product. This new paradigm is called few-shot industrial anomaly detection (FS-IAD). The purpose of FS-IAD is to identify abnormal cases using only a small number of support samples in the target category. FS-IAD methods can be generally divided into two categories: inductive FS-IAD (IFS-IAD) models and transductive FS-IAD (TFS-IAD) models. Compared with IFS-IAD methods that only use a small amount of statistical information of support images (training images) to perform anomaly detection, TFS-IAD models use statistical data from support images and query images (test images) for anomaly detection, which improves the generality of anomaly detection.

[0005] However, existing TFS-IAD methods rely on heuristics and linear metrics for anomaly detection, which can compromise the integrity of image structure and lose local features, thereby affecting the detection results. In addition, in actual industrial scenarios, the samples to be detected are usually detected one by one or in groups from the pipeline, i.e., online detection, and existing TFS-IAD methods cannot meet the online detection requirements. SUMMARY

[0006] To solve the above problems in the prior art, the present application provides a few-shot industrial anomaly detection method and device based on a memory bank. The technical problems to be solved by the present application are solved by the following technical solutions:

[0007] In a first aspect, the present application provides a few-shot industrial anomaly detection method based on a memory bank, comprising:

[0008] performing feature extraction on a plurality of support images, constructing a semantic memory bank, and connecting the semantic memory bank and a situational memory bank in series to form a total memory bank; wherein the semantic memory bank includes semantic features of the plurality of support images; the situational memory bank is initially empty;

[0009] The scene feature of the current query image is registered with the semantic feature in the semantic memory bank based on the online CT (Conditional Transport) theory to obtain a best matching flow; the scene memory is calibrated based on the best matching flow and the scene feature of the current query image, the scene memory bank is updated, and the total memory bank is further updated;

[0010] The scene feature of the current query image is registered with the feature in the updated total memory bank based on the online CT theory, and an abnormal score map of the current query image is obtained according to the registration result, and the abnormal detection of the current query image is realized based on the abnormal score map.

[0011] In a second aspect, the present application provides a small sample industrial anomaly detection device based on a memory bank, which is used to realize the method provided in the first aspect of the present application, and comprises:

[0012] The semantic memory module is used to extract features from a plurality of support images, construct a semantic memory bank, and connect the semantic memory bank and the scene memory bank in series to form a total memory bank; wherein the semantic memory bank comprises semantic features of the plurality of support images; the scene memory bank is initially empty;

[0013] The scene memory calibration module is used to register the scene feature of the current query image with the semantic feature in the semantic memory bank based on the online CT theory to obtain a best matching flow; the scene memory is calibrated based on the best matching flow and the scene feature of the current query image, the scene memory bank is updated, and the total memory bank is further updated.

[0014] The anomaly detection module is used to register the scene feature of the current query image with the feature in the updated total memory bank based on the online CT theory, and an abnormal score map of the current query image is obtained according to the registration result, and the abnormal detection of the current query image is realized based on the abnormal score map.

[0015] The present application has the following advantages:

[0016] 1. The small sample industrial anomaly detection method based on a memory bank provided by the present application firstly constructs a total memory bank comprising a semantic memory bank and a scene memory bank; then uses a nonlinear CT function as an index for calculating the structural distance between the memory bank and the query image to update the memory bank online; finally, based on the updated memory bank, the CT function is used again to realize the anomaly detection of the query image. This method uses a nonlinear and theoretically guaranteed CT function as a measurement tool between the memory bank and the query image, retains the local features of the image, realizes accurate feature matching; at the same time, in the anomaly detection process, the query image is used for memory calibration, which enhances the generalization ability of the memory bank, so that the method can quickly generalize across products under low data volume, while improving the detection accuracy, which can meet the online detection needs of small sample industrial anomaly detection;

[0017] 2、The application extracts foreground features and background features supporting the image respectively when constructing the semantic memory bank, solves the unreasonable attention allocation problem of foreground and background in small sample industrial anomaly detection, and reasonably balances the calculation efficiency and detection accuracy;

[0018] 3、The application further compresses the scenario memory bank when it is full, relieves the storage and calculation pressure of the memory bank, and further improves the efficiency and accuracy of anomaly detection.

[0019] The application will be further described in detail below in combination with the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of the small sample industrial anomaly detection method based on the memory bank provided by the embodiment of the application;

[0021] Figure 2 is an architecture diagram of the small sample industrial anomaly detection method based on the memory bank provided by the embodiment of the application;

[0022] Figure 3 is a process diagram of constructing the semantic memory bank provided by the embodiment of the application;

[0023] Figure 4 is a process diagram of scenario memory calibration provided by the embodiment of the application;

[0024] Figure 5 is a process diagram of online anomaly detection provided by the embodiment of the application;

[0025] Figure 6 is an experimental effect diagram of anomaly detection of different industrial products by using the detection method of the application;

[0026] Figure 7 is a structure block diagram of a small sample industrial anomaly detection device based on the memory bank provided by the embodiment of the application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the application will be described clearly and completely below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0028] The first aspect of the present application provides a small sample industrial anomaly detection method based on a memory bank, which constructs a TFS-IAD framework based on a memory bank using online CT theory, specifically including three parts, one is to build an initial total memory bank according to support images, two is to use nonlinear CT distance as a measure of memory bank and query image to calibrate situational memory, update the total memory bank online, three is to use nonlinear CT distance to register between query image features and updated memory bank, to realize anomaly detection and positioning of the current query image.

[0029] Specifically, please refer to Figure 1 and Figure 2 , Figure 1 is a flowchart of the small sample industrial anomaly detection method based on a memory bank provided by the embodiment of the present application, Figure 2 is an architecture diagram of the small sample industrial anomaly detection method based on a memory bank provided by the embodiment of the present application. The method mainly includes the following steps:

[0030] Step 1: feature extraction is performed on a plurality of support images, a semantic memory bank is constructed, and the semantic memory bank and a situational memory bank are connected in series to form a total memory bank; wherein the semantic memory bank includes semantic features of the plurality of support images; the situational memory bank is initially empty.

[0031] Specifically, in step 1, the construction process of the semantic memory bank can be seen from Figure 3 , mainly including:

[0032] 1) using a visual segmentation model to segment the foreground and background of the support image, and extracting the foreground feature and the background feature of the support image respectively;

[0033] 2) concatenating the foreground feature and the background feature into a semantic feature to form a semantic memory bank.

[0034] Optionally, as an implementation manner, the visual segmentation model is pre-trained, which can include a WRN (Wide Residual Network) network and a SAM (Segment Anything Model) model; then using the visual segmentation model to segment the foreground and background of the support image, and extracting the foreground feature and the background feature of the support image respectively, specifically including:

[0035] 1a) using the WRN network to extract a multi-scale feature map of the support image;

[0036] 1b) using the SAM model to calculate the foreground mask map and the background mask map of the support image;

[0037] 1c) performing down-sampling operation on the foreground mask map and the background mask map respectively;

[0038] 1d) using the down-sampled foreground mask map, background mask map and multi-scale feature maps to obtain foreground features and background features of the support image.

[0039] Specifically, on the basis of existing research, first use theta WRN The parameterized WRN model extracts the multi-scale feature map of the support image x 1:k , denoted as f 1:k = g θWRN (x 1:k ), and then use the SAM model with parameters theta SAM to calculate the mask map corresponding to the support image, denoted as Where ds(.) represents the down-sampling operation. Wherein, There are, k is the number of images, h is the length, w is the width, and c is the number of channels.

[0040] Then, using the feature map and the mask map obtain the foreground features and background features of the support image. The formula is as follows:

[0041]

[0042] In the formula, N f and N b are the number of feature vectors of the foreground and background, which are usually large, and fla(.) is a flattening operation. Therefore, the size of the foreground and background features needs to be compressed according to different compression sampling rates alpha and beta to reduce redundancy. Finally, the semantic memory can be concatenated as

[0043] The present application uses a foreground perception strategy when constructing the semantic memory bank, extracts the foreground features and background features of the support image respectively, solves the problem of unreasonable allocation of foreground and background attention in small sample industrial anomaly detection, and reasonably balances the calculation efficiency and detection accuracy.

[0044] It should be noted that at the initial moment, the episodic memory bank is an empty set, and the episodic memory bank and the constructed semantic memory bank are concatenated together to form the total memory bank. The episodic memory bank will be updated in real time with the query image of online detection, so the total memory bank is also updated.

[0045] Step 2: Based on the online CT theory, the episodic features of the current query image and the semantic features in the semantic memory bank are registered online to obtain the best matching flow; based on the best matching flow and the episodic features of the current query image, the episodic memory is calibrated, the episodic memory bank is updated, and then the total memory bank is updated.

[0046] CT is a measure of the distance between two sets of weighted objects or distributions through a non-linear mapping function. Given a distribution and two sets of discrete objects y 1:m and z 1:n , where a e A m and b e A n represent the probability simplices of spaces and respectively. The definition of CT function is as follows:

[0047]

[0048] where C φ (y→z) and C φ (z→y) represent forward and backward CT distances respectively, and a is an optimization parameter, which is a trade-off coefficient to control the strength of forward and backward distances.

[0049] Taking forward distance as an example:

[0050]

[0051] where c(·,·) is the Euclidean distance, π n (z j |y i ; φ) represents the forward transition probability matrix satisfying , i.e. the transition probability from y i to z j , and d φ (·,·) represents a non-linear metric function. The forward CT can be interpreted as the expected cost of transmitting a random source point to one of the m random anchor points.

[0052] By minimizing the forward distance formula, an unbiased estimate of the two distributions can be obtained, and the correlation between the two distributions can be reasonably measured. Therefore, the invention selects a non-linear CT distance as a measure for scenario memory calibration and anomaly detection.

[0053] Specifically, please refer to Figure 4 , Figure 4 is a process diagram of scenario memory calibration provided by an embodiment of the invention. For scenario memory calibration, the embodiment designs a memory enhancement side strategy. The purpose of memory enhancement is to use the features of the current tth query image to collect the statistical data of the historical query images of the old scenario memory to form an enhanced scenario memory.

[0054] In order to use the current query image The statistical data obtains new episodic memories, first need to solve the semantic memory characteristics M s with the scene characteristics of the query image The optimal matching flow between them, as shown in (a) of FIG. Figure 4 M s and are respectively regarded as two groups of data sampled from two distributions, and the CT function is used to represent the distance between the two.

[0055] Specifically, in step 2, based on the online CT theory, the scene characteristics of the current query image and the semantic characteristics in the semantic memory library are registered to obtain the best matching flow, including:

[0056] 21) Feature extraction is performed on the current query image to obtain the scene characteristics.

[0057] Optionally, the embodiment can use the same WRN network as extracting the semantic memory characteristics to obtain the scene characteristics of the current query image , denoted as , which indicates that a query image can extract m feature vectors.

[0058] 22) Calculate the CT function between the semantic characteristics in the semantic memory library and the scene characteristics; and optimize the CT function to obtain the first optimal forward transition probability matrix as the best matching flow.

[0059] Wherein, the CT function is expressed as:

[0060]

[0061] In the formula, L φ,ρ denotes the CT function, also known as the CT distance, φ denotes the optimization parameter, ρ denotes the weight coefficient controlling the strength of the forward distance and the backward distance, n denotes the number of semantic characteristics in the semantic memory library; m denotes the number of features of the current query image, M i s denotes the i-th semantic characteristic in the semantic memory library, denotes the i-th feature scene of the current query image, denotes the forward transition probability matrix, denotes the backward transition probability matrix, and c(·,·) denotes the Euclidean distance between two features.

[0062] Wherein, the expressions of the forward and backward transition probability matrices are:

[0063]

[0064]

[0065] Then, the semantic memory and the current query features are obtained by minimizing the formula (2) through the Adam gradient descent algorithm, and the optimal forward transition probability matrix is obtained, that is, the best matching flow.

[0066] Finally, the scene memory is calibrated based on the best matching flow and the scene features of the current query image, and the scene memory library is updated, as shown in the (b) of the formula (2). Figure 4

[0067] a) argmax operation is performed on each column of the first optimal forward transition probability matrix, and K scene features most related to the semantic features in the semantic memory library are selected to obtain memory enhanced features.

[0068] After the optimal forward transition probability matrix is obtained argmax operation is performed on each column, and K query features most related to the normal features are selected to obtain memory enhanced ΔM t e , denoted as:

[0069]

[0070] Here

[0071] b) The memory enhanced features are added to the scene memory library, and the scene memory library is updated.

[0072] The scene memory is updated by using the memory enhancement The enhanced memory library is denoted as, Here L t denotes the size of the scene memory library when the tth query sample comes.

[0073] In another embodiment of the application, a memory compression strategy is also designed for scene memory calibration. When the scene memory library is full, the scene features in the scene memory library are compressed, as shown in the (c) of the formula (2). Figure 4

[0074] Specifically, in practice, the memory library is not infinite, and as the number of online query images increases, the scene memory library may be full. Therefore, the application designs a memory compression mechanism for effective data management of the scene memory, and the target is to obtain a subset of the scene memory so that the subset can obtain the most accurate and fastest approximate solution to the scene memory problem, and the calculation on the scene memory can closely adjust the subset.

[0075] ​​In the embodiment, the greedy core-set approximation solution selection algorithm and the Johnson-Lindenstrauss theorem are used to compress the scene features in the scene memory bank, to obtain compressed scene features.

[0076] The greedy core-set approximation solution selection algorithm is used in the embodiment, which can guarantee the coverage rate equivalent to the original scene memory, solves the inherent NP-hard problem, and obtains the core-set approximation solution. In addition, the Johnson-Lindenstrauss theorem can also reduce the dimension of the scene memory and improve the efficiency of memory compression. After obtaining the compressed scene memory, the combination of the new semantic memory and the scene memory can be described as Where λ represents the data compression rate.

[0077] The application also compresses the scene memory when the scene memory bank is full, relieves the storage and computing pressure of the memory bank, and makes the anomaly detection accurate and efficient.

[0078] The scene memory calibration method designed in the embodiment can complement the corresponding semantic module, and can better capture the statistical data of the current and historical query images.

[0079] Step 3: Based on the online CT theory, the scene features of the current query image are matched with the features in the updated total memory bank, and the anomaly score map of the current query image is obtained according to the matching result, and the anomaly detection of the current query image is realized based on the anomaly score map.

[0080] Please refer to Figure 5 , Figure 5 is a process diagram of online anomaly detection provided by the embodiment of the application, and specifically includes:

[0081] 31) The CT function between the scene features of the current query image and the features in the updated total memory bank is optimized to obtain a second optimal forward transition probability matrix.

[0082] The detailed process can refer to the process in step 2.

[0083] 32) Based on the second optimal forward transition probability matrix, the anomaly score map of the current online query image is obtained.

[0084] Specifically, after optimizing the CT function, the anomaly score map of the current query image can be obtained, denoted as s j , and the expression is:

[0085]

[0086] 33) Based on the abnormal score map, find the pixels with scores greater than the preset threshold, determine as abnormal points, and locate, thereby realizing industrial anomaly detection.

[0087] Specifically, for image-level anomaly detection, the maximum distance score s * = max j＝[1,m] s j in all pixels is used to represent; when this maximum distance score exceeds a certain threshold, the pixel point is determined as an abnormal point, and the abnormal point detection and positioning are realized by locating the pixel point.

[0088] Optionally, for pixel-level positioning, first, bilinear interpolation is used to enlarge the score map to match the original input resolution, and then a Gaussian kernel with a size of 4 is used for smoothing operation on the score map.

[0089] It should be noted that for a query image, there can be multiple abnormal points, therefore, for the pixels with scores greater than the preset threshold, they can be determined as abnormal points and positioned, thereby realizing industrial anomaly detection.

[0090] The present application uses a foreground perception strategy to segment foreground and background features, and combines the subsequent online CT theory to build a TFS-IAD framework based on FOCT (Foreground-aware Online Conditional Transport), and realizes a small sample industrial anomaly detection method based on a memory bank. The method first constructs a total memory bank including a semantic memory bank and a situational memory bank;Then, a nonlinear CT function is used as an index for calculating the structural distance between the memory bank and the query image, and the memory bank is updated online;Finally, based on the updated memory bank, the CT function is used again to realize anomaly detection of the query image. The method uses a nonlinear and theoretically guaranteed CT function as a measurement tool between the memory bank and the query image, retains the local features of the image, and realizes accurate feature matching;At the same time, in the anomaly detection process, the query image is used for memory calibration, which enhances the generalization ability of the memory bank, so that the method can quickly generalize across products under low data volume, while improving the detection accuracy, which can meet the online detection needs of small sample industrial anomaly detection.

[0091] The beneficial effects of the present application can also be verified by simulation experiments.

[0092] Please refer to Figure 6 , Figure 6It is the experimental effect diagram of different industrial products for abnormal detection by using the detection method of the application. Among them, OCT (Online Conditional Transport) means the result of detection without using the foreground perception strategy, and only using the online conditional transmission strategy, and FOCT is the result of detection using the foreground perception online conditional transmission strategy of the application. In the simulation process, the model updates the CT parameter φ using the MLP with ReLU activation. Taking a two-layer MLP with a size of [d1, d2, d3] as an example, the calculation complexity of forward and backward calculation before each iteration is O(NMd2(d1+d3)), wherein N and M are the feature numbers of the two sets respectively. The size of the MLP or the size of the two sets can be further controlled to realize the trade-off between accuracy and real-time efficiency.

[0093] From Figure 6 It can be seen that FOCT allocates attention (the red box represents the selected feature vector corresponding to the position on the image) to the foreground features, reduces the invalid enhancement of the memory bank caused by redundant background (the background features are similar, and the redundant background information is not conducive to enhancing the feature diversity of the memory bank), and OCT pays more attention to the redundant background information. Compared with OCT without the foreground perception mechanism, FOCT shows more reasonable attention allocation when processing foreground and background.

[0094] The second aspect of the application provides a small sample industrial anomaly detection device based on a memory bank. Please refer to Figure 7 , Figure 7 is a structural block diagram of a small sample industrial anomaly detection device based on a memory bank provided by the embodiment of the application, and the device comprises:

[0095] The semantic memory module is used for feature extraction on a plurality of support images, construction of a semantic memory bank, and connection of the semantic memory bank and a episodic memory bank to form a total memory bank. The semantic memory bank comprises semantic features of the plurality of support images. The episodic memory bank is initially empty.

[0096] The episodic memory calibration module is used for registration of the episodic features of the current query image and the semantic features in the semantic memory bank based on the online CT theory to obtain a best matching flow. The episodic memory calibration module is used for episodic memory calibration based on the best matching flow and the episodic features of the current query image, and the episodic memory bank is updated, and then the total memory bank is updated.

[0097] The anomaly detection module is used for registration of the episodic features of the current query image and the features in the updated total memory bank based on the online CT theory, and obtains an anomaly score map of the current query image according to the registration result, and realizes anomaly detection of the current query image based on the anomaly score map.

[0098] Optionally, as an implementation manner, in the semantic memory module, the embodiment adopts a foreground awareness strategy to separate the foreground and background features of the support samples to construct the memory library, also known as the foreground awareness semantic memory. In the episodic memory calibration module, the embodiment adopts the online CT theory and the memory enhancement and memory compression strategies, combines the foreground awareness strategy of the semantic memory module, forms the FOCT theory framework, and finally, in the anomaly detection module, the online CT theory is used again to realize anomaly detection of the query image.

[0099] The FOCT theory framework proposed in the embodiment can adaptively obtain the prior distribution of the query feature by optimizing the CT function, which conforms to the actual scene that the prior distribution of the query feature is usually unknown in advance. Another practical feature of the device is that it reasonably balances the calculation efficiency and the detection accuracy, especially for the sparse foreground product commonly encountered in actual applications.

[0100] The online memory calibration strategy proposed in the embodiment fully improves the data utilization rate, increases the diversity of the memory library by updating the episodic memory through the query image, further ensures the stronger robustness of the model, and relieves the storage and calculation pressure of the model by updating and compressing the online episodic memory.

[0101] In the description of the specification, the terms "first", "second" are only used for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0102] In addition, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0103] Although the application has been described in connection with specific embodiments thereof, it will be understood that it is capable of further modifications and this application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended to cover and embrace all adaptations or modifications of the application. The terminology used herein for the purpose of describing particular embodiments is not intended to be limiting of the application. This application is intended to cover all such modifications and alternatives within the scope of the application including combinations of aspects of the application with other items not described.

[0104] The above description is further detailed in connection with specific preferred embodiments of the application, and it is not to be construed that the specific implementation of the application is limited to these descriptions. For those skilled in the art, without departing from the concept of the application, a number of simple deductions or substitutions can be made, and all of them should be considered as falling within the protection scope of the application.

Claims

1. A memory bank-based small sample industrial anomaly detection method, characterized in that, The method comprises the following steps: segmenting the foreground and background of a support image by using a visual segmentation model, and extracting the foreground feature and the background feature of the support image respectively; concatenating the foreground feature and the background feature into a semantic feature to form a semantic memory bank, and concatenating the semantic memory bank with a episodic memory bank to form a total memory bank; the semantic memory bank comprises semantic features of a plurality of support images; the episodic memory bank is initially empty; extracting a feature of a current query image to obtain an episodic feature; establishing a CT function between the semantic features in the semantic memory bank and the episodic feature, and optimizing the CT function to obtain a first optimal forward transition probability matrix as a best matching flow; performing episodic memory calibration based on the best matching flow and the episodic feature of the current query image to update the episodic memory bank, and further update the total memory bank; optimizing a CT function between the episodic feature of the current query image and the features in the updated total memory bank to obtain a second optimal forward transition probability matrix; based on the second optimal forward transition probability matrix, obtaining an anomaly score map of the current online query image; based on the anomaly score map, finding pixels with a score greater than a preset threshold and determining the pixels as abnormal points, and positioning the abnormal points to realize anomaly detection of the current query image.

2. The memory bank-based small sample industrial anomaly detection method according to claim 1, characterized in that, The visual segmentation model comprises a WRN network and a SAM model; segmenting the foreground and background of a support image by using a visual segmentation model, and extracting the foreground feature and the background feature of the support image respectively, specifically comprising: extracting a multi-scale feature map of the support image by using the WRN network; calculating a foreground mask map and a background mask map of the support image by using the SAM model; performing down-sampling operation on the foreground mask map and the background mask map respectively; obtaining the foreground feature and the background feature of the support image by using the down-sampled foreground mask map, the background mask map and the multi-scale feature map. 3.The memory bank based small sample industrial anomaly detection method of claim 1, wherein, The CT function is expressed as: ； In the formula, Represents the CT function. Indicates the optimization parameters. This represents the trade-off coefficient controlling the intensity of forward and backward distances. This represents the number of semantic features in the semantic memory. This indicates the number of features in the currently queried image. Represents the first in the semantic memory. A semantic feature, This indicates the first image in the current query. A scenario feature, This represents the forward transition probability matrix. This represents the backward transition probability matrix. This represents the Euclidean distance between two features.

4. The memory bank-based small sample industrial anomaly detection method according to claim 3, characterized in that, updating the episodic memory bank by performing episodic memory calibration based on the best matching flow and the episodic feature of the current query image, specifically comprising: performing argmax operation on each column of the first optimal forward transition probability matrix to select K episodic features most related to the semantic features in the semantic memory bank to obtain memory enhanced features; adding the memory enhanced features to the episodic memory bank to update the episodic memory bank.

5. The memory bank-based small sample industrial anomaly detection method according to claim 4, characterized in that, When the episodic memory bank is full, memory compression is performed on the episodic features in the episodic memory bank.

6. The memory bank-based small sample industrial anomaly detection method according to claim 5, characterized in that, The memory compression of the episodic features in the episodic memory bank specifically comprises: adopting a greedy core set approximate solution selection algorithm and Johnson-Lindenstrauss theorem to compress the episodic features in the episodic memory bank to obtain compressed episodic features.

7. A memory bank based small sample industrial anomaly detection apparatus for implementing the method of any one of claims 1-6, characterized in that, The method comprises the following steps: a semantic memory module is configured to segment the foreground and background of a support image by using a visual segmentation model, and extract the foreground feature and the background feature of the support image respectively; The foreground feature and the background feature are concatenated as a semantic feature to form a semantic memory bank, and the semantic memory bank is concatenated with a episodic memory bank to form a total memory bank; the semantic memory bank includes semantic features of a plurality of support images; the episodic memory bank is initially empty; An episodic memory calibration module is configured to perform feature extraction on a current query image to obtain an episodic feature; establish a CT function between the semantic features in the semantic memory bank and the episodic feature, and optimize the CT function to obtain a first optimal forward transition probability matrix as a best matching flow; perform episodic memory calibration based on the best matching flow and the episodic feature of the current query image to update the episodic memory bank, and further update the total memory bank; An anomaly detection module is configured to optimize a CT function between the episodic feature of the current query image and the features in the updated total memory bank to obtain a second optimal forward transition probability matrix; obtain an anomaly score map of the current online query image based on the second optimal forward transition probability matrix; find out pixels with a score greater than a preset threshold based on the anomaly score map, determine the pixels as anomaly points, and position the anomaly points to achieve anomaly detection of the current query image.

Images