Small sample industrial anomaly detection method and device based on memory bank
By constructing semantic and episodic memory banks, using online CT theory to perform feature registration and memory bank updates, the problems of low data volume generalization and online detection in the existing technology are solved, and efficient and accurate small sample industrial anomaly detection is achieved.
Patent Information
- Application Number
- CN202411115675.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The existing unsupervised industrial anomaly detection methods cannot quickly generalize across products under low data volumes, and rely on heuristics and linear measurements to damage the integrity of the image structure and cannot meet the online detection needs.
Using a small sample industrial anomaly detection method based on memory database, by constructing semantic memory and epic memory, using online CT theory to perform feature registration and memory update, combined with nonlinear CT functions for abnormal detection, retain local features and enhance the general ability to transform.
With low data volume, it achieves rapid cross-product generalization, improves detection accuracy and efficiency, meets the online detection requirements of small sample industrial anomaly detection, and optimizes computing efficiency through foreground background feature separation and memory bank compression.
Smart Images

Figure CN119006423A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of industrial detection, and in particular relates to a small sample industrial anomaly detection method and device based on a memory library. Background Art
[0002] Industrial anomaly detection is a key process monitoring technology in industrial manufacturing, which aims to identify and diagnose abnormal situations on the production line to ensure the stability of the production process and the quality of the product. Existing industrial anomaly detection is mainly divided into two categories: supervised and unsupervised. The fragmentation of industrial anomalies, such as subtle scars and obvious scaly damage, makes supervised industrial anomaly detection difficult. Therefore, unsupervised industrial anomaly detection has been developed recently. It only uses normal industrial images of each product to train the model, and then uses the trained model to detect abnormal industrial images and accurately locate the corresponding abnormal areas.
[0003] Research on unsupervised industrial anomaly detection can be divided into two categories, namely reconstruction-based models and memory-based models. Reconstruction-based models use deep generative models (DGM) to learn continuous or discrete feature representations of normal images through a tedious training process. Memory-based models usually use a memory library to extend the DGM to record the normal features of normal training images. During testing, feature matching is usually performed between the test image and the memory. However, both types of models rely heavily on a large number of normal training images and cannot generalize quickly across products with low data volumes.
[0004] Inspired by human anomaly detection, few-shot learning (FSL) has been introduced into industrial anomaly detection (IAD) to learn a common model that can be shared among multiple products. It can also be extended to new products that only provide limited normal training images, such as only 1 or 2 samples for each product. This new paradigm is called few-shot industrial anomaly detection (FS-IAD). The purpose of FS-IAD is to identify anomalies using only a small number of support samples in the target class. FS-IAD methods can generally be divided into two categories: inductive few-shot industrial anomaly detection (IFS-IAD) models and transductive few-shot industrial anomaly detection (TFS-IAD) models. Compared with the IFS-IAD method, which only uses the statistical information of a small number of support images (training images) to perform anomaly detection, the TFS-IAD model uses the statistical data from the support images and the query images (test images) for anomaly detection, which improves the versatility of anomaly detection.
[0005] However, the existing TFS-IAD method relies on heuristics and linear metrics for anomaly detection, which damages the integrity of the image structure and loses local features, thus affecting the detection results. In addition, in actual industrial scenarios, the samples to be detected are usually detected one by one or in groups from the pipeline, that is, online detection, and the existing TFS-IAD method cannot meet the needs of online detection. Summary of the invention
[0006] In order to solve the above problems existing in the prior art, the present invention provides a small sample industrial anomaly detection method and device based on a memory library. The technical problem to be solved by the present invention is achieved through the following technical solutions:
[0007] In a first aspect, the present invention proposes a small sample industrial anomaly detection method based on a memory library, comprising:
[0008] Extract features from several supporting images, construct a semantic memory library, and connect the semantic memory library with the episodic memory library in series to form a total memory library; wherein the semantic memory library includes the semantic features of several supporting images; the episodic memory library is initially empty;
[0009] Based on the online CT (Conditional Transport) theory, the scene features of the current query image are registered with the semantic features in the semantic memory library to obtain the best matching stream; based on the best matching stream and the scene features of the current query image, the scene memory is calibrated to update the scene memory library, and then the total memory library is updated;
[0010] Based on the online CT theory, the scene features of the current query image are aligned with the features in the updated total memory library, and the anomaly score map of the current query image is obtained according to the alignment result. The anomaly detection of the current query image is realized based on the anomaly score map.
[0011] In a second aspect, the present invention proposes a small sample industrial anomaly detection device based on a memory library, which is used to implement the method provided in the first aspect of the present invention, comprising:
[0012] The semantic memory module is used to extract features from a number of supporting images, construct a semantic memory library, and connect the semantic memory library with the episodic memory library in series to form a total memory library; wherein the semantic memory library includes the semantic features of a number of supporting images; and the episodic memory library is initially empty;
[0013] The scenario memory calibration module is used to align the scenario features of the current query image with the semantic features in the semantic memory library based on the online CT theory to obtain the best matching stream; perform scenario memory calibration based on the best matching stream and the scenario features of the current query image, update the scenario memory library, and then update the total memory library;
[0014] The anomaly detection module is used to align the scene features of the current query image with the features in the updated total memory library based on the online CT theory, and obtain the anomaly score map of the current query image according to the alignment result, and realize anomaly detection of the current query image based on the anomaly score map.
[0015] Beneficial effects of the present invention:
[0016] 1. The present invention provides a small sample industrial anomaly detection method based on a memory library. First, a total memory library including a semantic memory library and a situational memory library is constructed; then a nonlinear CT function is used as an indicator to calculate the structural distance between the memory library and the query image, and the memory library is updated online; finally, based on the updated memory library, the CT function is used again to realize the anomaly detection of the query image. This method uses a nonlinear and theoretically guaranteed CT function as a measurement tool between the memory library and the query image, retains the local features of the image, and realizes accurate feature matching; at the same time, in the anomaly detection process, the query image is used for memory calibration, which enhances the generalization ability of the memory library, so that the method can be quickly generalized across products under low data volume conditions, while improving the detection accuracy, and can meet the online detection needs of small sample industrial anomaly detection;
[0017] 2. When constructing the semantic memory library, the present invention extracts the foreground features and background features of the supporting image respectively, solves the problem of unreasonable foreground and background attention allocation in small sample industrial anomaly detection, and reasonably balances the computational efficiency and detection accuracy;
[0018] 3. When the situational memory library is fully loaded, the present invention also performs compression processing on it, which alleviates the storage and calculation pressure of the memory library and further improves the efficiency and accuracy of anomaly detection.
[0019] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a flow chart of a small sample industrial anomaly detection method based on a memory bank provided by an embodiment of the present invention;
[0021] Figure 2 It is an architecture diagram of a small sample industrial anomaly detection method based on a memory bank provided by an embodiment of the present invention;
[0022] Figure 3 is a schematic diagram of a process of constructing a semantic memory library provided by an embodiment of the present invention;
[0023] Figure 4 is a schematic diagram of the process of episodic memory calibration provided by an embodiment of the present invention;
[0024] Figure 5 is a schematic diagram of the process of online anomaly detection provided by an embodiment of the present invention;
[0025] Figure 6 It is an experimental effect diagram of abnormality detection of different industrial products using the detection method of the present invention;
[0026] Figure 7 It is a structural block diagram of a small sample industrial anomaly detection device based on a memory bank provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] The first aspect of the present invention provides a small sample industrial anomaly detection method based on a memory library. The method uses online CT theory to construct a TFS-IAD framework based on a memory library. The framework specifically includes three parts: first, an initial total memory library is built according to a supporting image; second, nonlinear CT distance is used as a metric for the memory library and the query image to perform situational memory calibration and update the total memory library online; third, nonlinear CT distance is used to perform registration between query image features and the updated memory library to achieve anomaly detection and positioning of the current query image.
[0029] For details, see Figure 1 and Figure 2 , Figure 1 is a flow chart of a small sample industrial anomaly detection method based on a memory bank provided by an embodiment of the present invention. Figure 2 : is an architecture diagram of a small sample industrial anomaly detection method based on a memory bank provided by an embodiment of the present invention. The method mainly includes the following steps:
[0030] Step 1: Extract features from several supporting images, construct a semantic memory library, and connect the semantic memory library with the episodic memory library in series to form a total memory library; wherein the semantic memory library includes the semantic features of several supporting images; the episodic memory library is initially empty.
[0031] Specifically, in step 1, the process of building the semantic memory library can be seen in Figure 3 As shown, it mainly includes:
[0032] 1) Use the visual segmentation model to segment the foreground and background of the support image, and extract the foreground features and background features of the support image respectively;
[0033] 2) Concatenate foreground features and background features into semantic features to form a semantic memory library.
[0034] Optionally, as an implementation method, the visual segmentation model is pre-trained in advance, and may include a WRN (Wide Residual Network) network and a SAM (Segment Anything Model) model; the visual segmentation model is used to segment the foreground and background of the support image, and the foreground features and background features of the support image are extracted respectively, specifically including:
[0035] 1a) Using the WRN network to extract multi-scale feature maps of the support image;
[0036] 1b) Calculate the foreground mask map and background mask map of the support image using the SAM model;
[0037] 1c) performing downsampling operations on the foreground mask image and the background mask image respectively;
[0038] 1d) Using the downsampled foreground mask map, background mask map and multi-scale feature map, the foreground features and background features of the support image are obtained.
[0039] Specifically, based on existing research, we first use θ WRN Parameterized WRN model extracts support image x 1:k The multi-scale feature map is denoted as f 1:k =g θWRN (x 1:k ), and then use the parameter θ SAM The SAM model is used to calculate the corresponding mask map of the support image, denoted as Where ds(.) represents the downsampling operation. common, k is the number of images, h is the length, w is the width, and c is the number of channels.
[0040] Then, using the feature map and mask map Get the foreground features of the support image and background characteristics The formula is:
[0041]
[0042] Where N f and N b is the number of feature vectors of foreground and background, which are usually large, and fla(.) is a flattening operation. Therefore, it is necessary to compress the size of foreground and background features according to different compression sampling rates α and β to reduce redundancy. Finally, the semantic memory can be represented in series as
[0043] When constructing the semantic memory library, the present invention uses a foreground perception strategy to extract the foreground features and background features of the supporting image respectively, solves the problem of unreasonable foreground and background attention allocation in small-sample industrial anomaly detection, and reasonably balances computational efficiency and detection accuracy.
[0044] It should be noted that at the initial moment, the scene memory library is an empty set, and the scene memory library is connected in series with the constructed semantic memory library to form a total memory library. The subsequent scene memory library will be updated in real time with the query image detected online, so that the total memory library will also be updated together.
[0045] Step 2: Based on the online CT theory, the scene features of the current query image are online aligned with the semantic features in the semantic memory library to obtain the best matching stream; based on the best matching stream and the scene features of the current query image, the scene memory is calibrated, the scene memory library is updated, and then the total memory library is updated.
[0046] CT measures the distance between two sets of weighted objects or distributions through a nonlinear mapping function. and Two sets of discrete objects y are sampled 1:m and z 1:n , where a∈Δ m and b∈Δ n Respectively represent space and The probability simplex of . The CT function is defined as follows:
[0047]
[0048] Among them, C φ (y→z) and C φ (z→y) represents the forward and backward CT distances respectively, which are optimization parameters and are the trade-off coefficients controlling the strength of the forward distance and the backward distance.
[0049] Take the forward distance as an example:
[0050]
[0051] Where c(·,·) is the Euclidean distance, π n (z j |y i ; φ) means satisfying The forward transition probability matrix of i to z j The transition probability, d φ (·,·) represents a nonlinear metric function. The forward CT can be interpreted as the expected cost of transmitting a random source point to one of the m random anchor points.
[0052] By minimizing the forward distance formula, the two distributions can be estimated unbiasedly, and the correlation between the two distributions can be reasonably measured. Therefore, the present invention selects to use the nonlinear CT distance as a metric for contextual memory calibration and anomaly detection.
[0053] For details, see Figure 4 , Figure 4 is a schematic diagram of the process of scene memory calibration provided by an embodiment of the present invention. For scene memory calibration, this embodiment designs a memory enhancement strategy. The purpose of memory enhancement is to use the current t-th query image to collect old episodic memories Historical query images statistical data to form enhanced episodic memory.
[0054] In order to use the current query image To obtain new episodic memory from statistical data, we first need to solve the semantic memory feature M s The contextual features of the query image The optimal matching flow between Figure 4 As shown in Figure (a). s and They are regarded as two sets of data sampled from two distributions, and the CT function is used to represent the distance between them.
[0055] Specifically, in step 2, the scene features of the current query image are aligned with the semantic features in the semantic memory based on the online CT theory to obtain the best matching flow, including:
[0056] 21) For the current query graph Perform feature extraction to obtain scenario features.
[0057] Optionally, this embodiment can use the same WRN network as used to extract semantic memory features to obtain the current query image The situational features are recorded as It means that m feature vectors can be extracted from a query image.
[0058] 22) Calculate the CT function between the semantic features and the situational features in the semantic memory library; and optimize the CT function to obtain the first optimal forward transfer probability matrix as the best matching flow.
[0059] Among them, the CT function is expressed as:
[0060]
[0061] Where, L φ,ρ represents the CT function, also known as the CT distance, φ represents the optimization parameter, ρ represents the trade-off coefficient controlling the strength of the forward distance and the backward distance, n represents the number of semantic features in the semantic memory library; m represents the number of features of the current query image, M i s represents the i-th semantic feature in the semantic memory bank, represents the i-th feature scene of the current query image, represents the forward transition probability matrix, represents the backward transition probability matrix, and c(·,·) represents the Euclidean distance between two features.
[0062] Among them, the expressions of the forward and backward transition probability matrices are:
[0063]
[0064]
[0065] Then, by minimizing φ in formula (2) through a gradient descent algorithm such as Adam, the optimal forward transfer probability matrix between the semantic memory and the current query features can be obtained, that is, the best matching flow.
[0066] Finally, the context memory is calibrated based on the best matching stream and the context features of the current query image, and the context memory library is updated, such as Figure 4 As shown in Figure (b) of the figure, it specifically includes:
[0067] a) Performing an argmax operation on each column of the first optimal forward transfer probability matrix, and selecting K context features that are most relevant to the semantic features in the semantic memory library to obtain memory enhancement features.
[0068] After obtaining the optimal forward transfer probability matrix After that, we perform argmax operation on each column and select the K query features that are most relevant to the normal features to get the memory enhancement ΔM t e , recorded as:
[0069]
[0070] here
[0071] b) adding the memory enhancement feature to the episodic memory library and updating the episodic memory library.
[0072] Updating episodic memory using memory enhancement The enhanced memory bank is represented as, Here L t Refers to the size of the context memory bank when the tth query sample comes.
[0073] In another embodiment of the present invention, a memory compression strategy is also designed for the situational memory calibration. When the situational memory library is full, the situational features in the situational memory library are compressed. Figure 4 As shown in Figure (c).
[0074] Specifically, in practice, the memory is not infinite, and as the number of online query images increases, the situation where the scene memory is full may occur. Therefore, the present invention designs a memory compression mechanism for effective data management of the scene memory, with the goal of obtaining a scene memory Subset of This subset is enabled to obtain the most accurate and fastest approximate solution to the episodic memory problem, and the subset can be closely tuned using computations on the episodic memory.
[0075] In this embodiment, for the scenario feature set in the scenario memory library, a greedy core set approximate solution selection algorithm and the Johnson-Lindenstrauss theorem are used to compress the scenario features in the scenario memory library to obtain compressed scenario features.
[0076] This embodiment adopts a greedy core set approximate solution selection algorithm, which can ensure that the coverage rate is equivalent to that of the original episodic memory, solves the inherent NP-hard problem, and obtains. In addition, the Johnson-Lindenstrauss theorem can also reduce the dimension of episodic memory and improve the efficiency of memory compression. After obtaining the compressed episodic memory, the combination of the new semantic memory and episodic memory can be described as Where λ represents the data compression ratio.
[0077] When the situational memory bank is fully loaded, the present invention also performs compression processing on it, thereby alleviating the storage and calculation pressure of the memory bank, making anomaly detection accurate and efficient.
[0078] The context memory calibration method designed in this embodiment can complement its corresponding semantic module and can better capture the statistical data of current and historical query images.
[0079] Step 3: Based on the online CT theory, the scene features of the current query image are aligned with the features in the updated total memory library, and the anomaly score map of the current query image is obtained according to the alignment results. The anomaly detection of the current query image is realized based on the anomaly score map.
[0080] See also Figure 5 , Figure 5 Schematic diagram of the process of online anomaly detection provided by an embodiment of the present invention, which specifically includes:
[0081] 31) Optimize the CT function between the scene features of the current query image and the features in the updated total memory library to obtain a second optimal forward transfer probability matrix.
[0082] For detailed process, please refer to the process in step 2.
[0083] 32) Based on the second optimal forward transfer probability matrix, an abnormality score map of the current online query image is obtained.
[0084] Specifically, after optimizing the CT function, the abnormal score map of the current query image can be obtained, denoted as s j , whose expression is:
[0085]
[0086] 33) Based on the anomaly score map, find out the pixels whose scores are greater than the preset threshold, determine them as abnormal points, and locate them, thereby realizing industrial anomaly detection.
[0087] Specifically, for image-level anomaly detection, all pixels are used The maximum distance between * =max j=[1,m] s j To represent; when the maximum distance score exceeds a certain threshold, the pixel point can be determined as an outlier point, and outlier point detection and positioning can be achieved by locating the pixel point.
[0088] Optionally, for pixel-level localization, the score map is first upscaled to match the original input resolution using bilinear interpolation, and then smoothed with a Gaussian kernel of size 4.
[0089] It should be noted that for a certain query image, there may be multiple outliers. Therefore, pixels with scores greater than a preset threshold can be determined as outliers and located, thereby realizing industrial anomaly detection.
[0090] The present invention uses a foreground-aware strategy to segment foreground and background features, and combines the following online CT theory to build a TFS-IAD framework based on FOCT (Foreground-aware Online Conditional Transport), and implements a small sample industrial anomaly detection method based on a memory library. The method first constructs a total memory library including a semantic memory library and a situational memory library; then uses a nonlinear CT function as an indicator to calculate the structural distance between the memory library and the query image, and updates the memory library online; finally, based on the updated memory library, the CT function is used again to realize the anomaly detection of the query image. The method uses a nonlinear and theoretically guaranteed CT function as a measurement tool between the memory library and the query image, retains the local features of the image, and realizes accurate feature matching; at the same time, in the anomaly detection process, the query image is used for memory calibration, which enhances the generalization ability of the memory library, so that the method can be quickly generalized across products under low data volume conditions, while improving the detection accuracy, and can meet the online detection needs of small sample industrial anomaly detection.
[0091] The beneficial effects of the present invention can also be verified through simulation experiments.
[0092] See also Figure 6 , Figure 6It is an experimental effect diagram of abnormality detection of different industrial products using the detection method of the present invention. Among them, OCT stands for Online Conditional Transport, which means the result of detection without adopting the foreground perception strategy and only using the online conditional transmission strategy. FOCT is the result of detection using the foreground perception online conditional transmission strategy of the present invention. During the simulation process, the model uses the ReLU activated MLP to update the CT parameter φ. Taking the two-layer MLP of size [d1, d2, d3] as an example, the forward and backward computational complexity of each iteration is O(NMd2(d1+d3)), where N and M are the number of features of the two sets, respectively. The size of the MLP or the size of the two sets can be further controlled to achieve a trade-off between accuracy and real-time efficiency.
[0093] from Figure 6 As can be seen from the figure, FOCT allocates attention (the red box indicates the position of the selected feature vector on the image) to the foreground features, reducing the ineffective enhancement of the memory bank caused by redundant background (background features are similar, and redundant background information is not conducive to enhancing the feature diversity of the memory bank), while OCT pays attention to more redundant background information. Compared with OCT without a foreground perception mechanism, FOCT shows a more reasonable attention allocation when processing foreground and background.
[0094] The second aspect of the present invention proposes a small sample industrial anomaly detection device based on a memory library. Figure 7 , Figure 7 : is a structural block diagram of a small sample industrial anomaly detection device based on a memory bank provided by an embodiment of the present invention, the device comprising:
[0095] The semantic memory module is used to extract features from a number of supporting images, construct a semantic memory library, and connect the semantic memory library with the episodic memory library in series to form a total memory library; wherein the semantic memory library includes the semantic features of a number of supporting images; and the episodic memory library is initially empty;
[0096] The scenario memory calibration module is used to align the scenario features of the current query image with the semantic features in the semantic memory library based on the online CT theory to obtain the best matching stream; perform scenario memory calibration based on the best matching stream and the scenario features of the current query image, update the scenario memory library, and then update the total memory library;
[0097] The anomaly detection module is used to align the scene features of the current query image with the features in the updated total memory library based on the online CT theory, and obtain the anomaly score map of the current query image according to the alignment result, and realize anomaly detection of the current query image based on the anomaly score map.
[0098] Optionally, as an implementation method, in the semantic memory module, this embodiment adopts a foreground perception strategy to separate the foreground and background features of the supporting samples to build a memory library, also known as foreground-aware semantic memory. In the episodic memory calibration module, this embodiment adopts the online CT theory and two strategies of memory enhancement and memory compression, combined with the foreground perception strategy of the semantic memory module, to form a FOCT theoretical framework. Finally, in the anomaly detection module, the online CT theory is used again to implement anomaly detection of the query image.
[0099] The FOCT theoretical framework proposed in the present invention can adaptively obtain the prior distribution of query features by optimizing the CT function, which is in line with the actual scenario that the prior distribution of query features is usually unknown in advance. Another practical feature of the device is that it reasonably balances computational efficiency and detection accuracy, especially for sparse foreground products often encountered in practical applications.
[0100] The online memory calibration strategy proposed in the present invention fully improves data utilization. Since the context memory is updated by querying images, the diversity of the memory library is increased, which further ensures the stronger robustness of the model. In addition, the storage and computing pressure of the model is alleviated by updating and compressing the online context memory.
[0101] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0102] In addition, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.
[0103] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in a claim. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.
[0104] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.
Claims
1. A small sample industrial anomaly detection method based on memory library, characterized in that: include: Extract features from a number of supporting images, construct a semantic memory library, and connect the semantic memory library with the situational memory library in series to form a total memory library; wherein the semantic memory library includes the semantic features of a number of supporting images; and the situational memory library is initially empty; Based on the online CT theory, the scene features of the current query image are registered with the semantic features in the semantic memory to obtain the best matching flow; Performing situational memory calibration based on the best matching stream and the situational features of the current query image, updating the situational memory library, and then updating the total memory library; Based on the online CT theory, the scene features of the query image are aligned with the features in the updated total memory library, and an anomaly score map of the current query image is obtained according to the alignment result, and anomaly detection of the current query image is implemented based on the anomaly score map.
2. The small sample industrial anomaly detection method based on memory library according to claim 1 is characterized in that: Feature extraction is performed on several supporting images to build a semantic memory library, including: Segmenting the foreground and background of the support image using a visual segmentation model, and extracting foreground features and background features of the support image respectively; The foreground features and the background features are connected in series to form semantic features to form a semantic memory library.
3. The small sample industrial anomaly detection method based on memory library according to claim 2 is characterized in that: The visual segmentation model includes a WRN network and a SAM model; The visual segmentation model is used to segment the foreground and background of the support image, and the foreground features and background features of the support image are extracted respectively, specifically including: Extracting a multi-scale feature map of the support image using the WRN network; Calculating a foreground mask map and a background mask map of the support image using the SAM model; Performing downsampling operations on the foreground mask image and the background mask image respectively; The foreground features and background features of the support image are obtained by using the downsampled foreground mask map, the background mask map and the multi-scale feature map.
4. The small sample industrial anomaly detection method based on memory library according to claim 1 is characterized in that: Based on the online CT theory, the scene features of the current query image are aligned with the semantic features in the semantic memory library to obtain the best matching flow, which specifically includes: Extract features from the current query image to obtain scene features; A CT function is established between the semantic features in the semantic memory library and the situational features, and the CT function is optimized to obtain a first optimal forward transfer probability matrix as the best matching flow.
5. The small sample industrial anomaly detection method based on memory library according to claim 4 is characterized in that: The CT function is expressed as: Where, L φ,ρ represents the CT function, φ represents the optimization parameter, ρ represents the trade-off coefficient controlling the strength of forward distance and backward distance, and n represents the number of semantic features in the semantic memory bank; m represents the number of features of the current query image, represents the i-th semantic feature in the semantic memory bank, represents the i-th scene feature of the current query image, represents the forward transition probability matrix, represents the backward transition probability matrix, and c(·,·) represents the Euclidean distance between two features.
6. The small sample industrial anomaly detection method based on memory library according to claim 4 is characterized in that: Performing situational memory calibration based on the best matching stream and the situational features of the current query image to update the situational memory library specifically includes: Performing an argmax operation on each column of the first optimal forward transfer probability matrix, selecting K context features that are most relevant to the semantic features in the semantic memory library, and obtaining memory enhancement features; The memory enhancement feature is added to the contextual memory library, and the contextual memory library is updated.
7. The memory-based small sample industrial anomaly detection method according to claim 6, characterized in that: When the situation memory library is fully loaded, the situation features in the situation memory library are compressed.
8. The memory-based small sample industrial anomaly detection method according to claim 7, characterized in that: Memory compression of the scene features in the scene memory library specifically includes: The scenario features in the scenario memory bank are compressed using a greedy core set approximate solution selection algorithm and the Johnson-Lindenstrauss theorem to obtain compressed scenario features.
9. The small sample industrial anomaly detection method based on memory library according to claim 1 is characterized in that:Based on the online CT theory, the scene features of the query image are registered with the features in the updated total memory library, and an abnormality score map of the current query image is obtained according to the registration result. The abnormality detection of the current query image is implemented based on the abnormality score map, which specifically includes: Optimizing the CT function between the scene features of the current query image and the features in the updated total memory library to obtain a second optimal forward transfer probability matrix; Based on the second optimal forward transfer probability matrix, obtaining an abnormality score map of the current online query image; Based on the anomaly score map, pixels with scores greater than a preset threshold are found, determined as anomalies, and the anomalies are located, thereby achieving anomaly detection of the current query image.
10. A small sample industrial anomaly detection device based on a memory library, used to implement the method according to any one of claims 1 to 9, characterized in that: include: A semantic memory module is used to extract features from a number of supporting images, construct a semantic memory library, and connect the semantic memory library with the contextual memory library in series to form a total memory library; wherein the semantic memory library includes the semantic features of a number of supporting images; and the contextual memory library is initially empty; A situation memory calibration module is used to align the situation features of the current query image with the semantic features in the semantic memory library based on the online CT theory to obtain the best matching stream; perform situation memory calibration based on the best matching stream and the situation features of the current query image, update the situation memory library, and then update the total memory library; The anomaly detection module is used to align the scene features of the current query image with the features in the updated total memory library based on the online CT theory, and obtain the anomaly score map of the current query image according to the alignment result, and realize anomaly detection of the current query image based on the anomaly score map.
Citation Information
Patent Citations
Defect prediction method and system for small sample knowledge transfer learning
CN114663401A
Lightweight industrial environment defect detection method based on feature memory library
CN115690058A
Small sample anomaly detection method and device based on feature storage
CN117710772A
Image anomaly detection method and system based on lightweight memory bank
CN117765363A
Self-supervised industrial defect detection method based on prototype memory guidance
CN118196051A
