An adaptive geometric modal alignment method, system, device and medium for small sample industrial anomaly detection

Abstract

The present application relates to the technical field of industrial visual angle detection, in particular to a self-adaptive geometric modal alignment method, system, device and medium for small sample industrial anomaly detection; first, a pre-trained visual language model is used to extract visual embeddings and text embeddings of normal reference images, images to be detected and defect text descriptions respectively; then a symmetric positive definite covariance is constructed; covariance interpolation is performed in the logarithmic Euclidean space to obtain a target covariance correction matrix; the text covariance integrated by combining the prompt word is used to perform geometric correction on the text embedding while keeping the visual embedding unchanged; finally, the cosine similarity between the corrected text embedding and the visual embedding is calculated to obtain an anomaly score, and image-level anomaly detection results and pixel-level anomaly positioning results are output; the influence of covariance estimation distortion and text semantic noise on cross-modal matching is suppressed when the number of samples is much smaller than the feature dimension, and the stability and accuracy of small sample industrial anomaly detection are improved.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and more specifically, to an adaptive geometric modal alignment method, system, device, and medium for small-sample industrial anomaly detection. Background Technology

[0002] Industrial anomaly detection is a crucial aspect of quality control in intelligent manufacturing. Its goal is to identify surface defects, structural anomalies, or logical anomalies in products during the production process. Due to the scarcity of defect samples in industrial settings, high collection costs, and the diverse forms of anomalies, practical applications typically only yield a very small number of normal or reference samples; for example, only a few reference images may be available for each product type.

[0003] In recent years, pre-trained vision-language models such as CLIP have achieved strong open-vocabulary understanding capabilities through large-scale image-text contrastive learning. They can map images and text to a unified embedding space, thereby supporting anomaly detection based on text prompts. For industrial defect detection, the model can use text descriptions such as "scratches," "damage," and "the number of screws should be 15" to assist in identifying appearance defects or logical anomalies.

[0004] However, in small-sample industrial scenarios, there is often a systematic modal gap between image embeddings and text embeddings. Even if the image and text have the same semantics, they are often distributed in different regions of the embedding space after being encoded by the pre-trained model. If this distribution difference is not modeled and matching is performed directly using the original similarity, it can easily lead to systematic biases in cross-modal matching.

[0005] Existing methods attempt to align cross-modal distributions by estimating statistics such as the mean and covariance of the image and text domains. However, under small sample conditions, the number of samples N is often much smaller than the feature dimension d. For example, when N=4 and d=768, the sample covariance matrix exhibits severe estimation errors, manifesting as mean shift, variance distortion, and abnormal amplification of anisotropy. Directly using the distorted sample covariance for alignment can overstretch text features and amplify noise in some directions, while overcompressing true semantic changes in others, leading to overfitting or underfitting.

[0006] Meanwhile, industrial defect descriptions typically originate from manual annotation, rule bases, or domain knowledge bases, leading to issues such as synonymous distortion, semantic ambiguity, and redundant embellishments. For example, "scratches," "grazes," and "abrasions" may describe the same type of defect, but their text embeddings may be scattered; descriptions like "looks abnormal" lack specific semantic boundaries; and irrelevant embellishments further reduce the signal-to-noise ratio of text features. Under small sample conditions, this textual noise further amplifies the instability of cross-modal alignment.

[0007] Therefore, there is an urgent need for a cross-modal geometric correction method that can stably utilize sample information and reference covariance under extremely small sample conditions, in order to suppress covariance estimation distortion and text semantic noise, and improve the accuracy and stability of industrial anomaly detection. Summary of the Invention

[0008] This invention addresses the challenges of reliably correcting modal gaps between the image and text domains under small sample conditions, as well as the distortion of sample covariance estimation and the impact of textual semantic noise on detection performance. It proposes an adaptive geometric modal alignment method, system, device, and medium for small-sample industrial anomaly detection. First, a pre-trained visual language model is used to extract visual and textual embeddings from a small number of normal reference images, the image to be detected, and defective text descriptions. Then, the image domain sample covariance is estimated based on the small-sample visual embeddings, and a stable symmetric positive definite covariance is constructed by shrinking it towards an isotropic symmetric positive definite reference covariance. Subsequently, covariance interpolation is performed in logarithmic Euclidean space to obtain the target covariance correction matrix. Further, a whitening-coloring transformation is constructed using the text covariance obtained from cue word integration to geometrically correct the text embeddings while maintaining the visual embeddings unchanged. Finally, anomaly scores are calculated based on the cosine similarity between the corrected text and visual embeddings, outputting image-level anomaly detection results and pixel-level anomaly localization results. This approach suppresses the impact of covariance estimation distortion and textual semantic noise on cross-modal matching even when the sample size is much smaller than the feature dimension, improving the stability and accuracy of small-sample industrial anomaly detection.

[0009] The specific implementation details of this invention are as follows: An adaptive geometric modal alignment method for small-sample industrial anomaly detection includes the following steps: Step S1: Acquire the normal reference image, the image to be inspected, and the text description of the industrial object to be inspected; Step S2: Based on the normal reference image, the image to be detected, and the text description of the defect category obtained by encoding the pre-trained visual language model, obtain the reference image embedding, the image to be detected embedding, and the text embedding; Step S3: Estimate the mean and covariance of the image domain samples based on the reference image embedding, and construct the isotropic symmetric positive definite reference covariance of the image domain based on the set scale parameters and identity matrix. Step S4: Take the sample covariance and the image domain isotropic symmetric positive definite reference covariance as symmetric positive definite matrices and place them on the covariance manifold to calculate the adaptive coefficients; Step S5: Based on the adaptive coefficients, interpolate the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance on the covariance manifold geodesic line to obtain the target covariance correction matrix. Step S6: Keeping the reference image embedding unchanged, obtain the corrected text embedding based on the image domain target covariance correction matrix and the integrated text covariance of the prompt words; Step S7: Perform cosine similarity matching between the corrected text embedding and the image embedding to be detected, calculate the anomaly score, and determine whether there is anomaly in the image to be detected based on the anomaly score.

[0010] To better realize the present invention, step S2 further includes the following steps: Step S21: Input the normal reference image and the image to be detected into the pre-trained visual encoder to obtain the reference image embedding set and the image to be detected embedding; Step S22: Input the text prompt into the pre-trained text encoder to obtain the original text embedding and construct the text embedding set.

[0011] To better realize the present invention, step S3 further includes the following steps: Step S31: Normalize the sets of image embeddings and text embeddings to be detected; Step S32: Calculate the mean of the image domain samples based on the reference image embedding; Step S33: Calculate the image domain sample covariance based on the reference image embedding and the set regularization coefficients; Step S34: Construct an isotropic symmetric positive definite reference covariance in the image domain based on the image domain sample covariance, the set scale parameter, and the identity matrix.

[0012] To better realize the present invention, step S34 further includes the following steps: Step S341: Based on the image domain sample covariance and the set feature dimensions of the visual embedding, call the matrix trace operation to calculate the image domain reference scale parameters; Step S342: Construct an isotropic symmetric positive definite reference covariance in the image domain based on the image domain reference scale parameter and the unit matrix under the set visual embedding feature dimension.

[0013] To better realize the present invention, step S4 further includes the following steps: Step S41: Perform feature decomposition on the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance to obtain the direction vector and scale vector respectively; Step S42: Calculate the direction difference term and scale difference term based on the direction vector and scale vector; Step S43: Based on the set stability term, orientation difference term, and scale difference term, use the total change of the isotropic symmetric positive definite reference covariance in the image domain as the normalization factor, and call the clip() function to obtain the adaptive coefficients.

[0014] To better realize the present invention, step S6 further includes the following steps: Step S61: Calculate the mean of the integrated text embeddings of the prompt words and the covariance of the integrated text embeddings of the prompt words based on the text embedding set; Step S62: Whiten the original text embedding based on the integrated text covariance of the prompt words, and color it according to the image domain target covariance correction matrix to obtain the corrected text embedding.

[0015] To better realize the present invention, step S7 further includes the following steps: Step S71: Based on the image embedding to be detected and the corrected text embedding, call the cosine similarity to calculate the anomaly score; Step S72: Determine whether there is an anomaly in the image to be detected based on the set threshold and the anomaly score; if the anomaly score is greater than the set threshold, it is determined that there is an anomaly in the image to be detected; if the anomaly score is less than the set threshold, it is determined that there is no anomaly in the image to be detected.

[0016] Based on the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection, in order to better realize this invention, a further adaptive geometric modal alignment system for small-sample industrial anomaly detection is proposed, which is used to execute the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection, including a data input unit, a feature encoding unit, a statistical estimation unit, an adaptive coefficient calculation unit, a manifold interpolation unit, a cross-modal geometric correction unit, and an anomaly score output unit; The data input unit is used to acquire a normal reference image, an image to be detected, and a text description of the industrial object to be detected. The feature encoding unit is used to encode the normal reference image, the image to be detected, and the text description of the defect category obtained by the pre-trained visual language model to obtain the reference image embedding, the image to be detected embedding, and the text embedding. The statistical estimation unit is used to estimate the mean and covariance of image domain samples based on the reference image embedding, and to construct an isotropic symmetric positive definite reference covariance in the image domain based on the set scale parameters and identity matrix. The adaptive coefficient calculation unit is used to take the sample covariance and the image domain isotropic symmetric positive definite reference covariance as symmetric positive definite matrices and place them on the covariance manifold to calculate the adaptive coefficients. The manifold interpolation unit is used to interpolate the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance on the covariance manifold geodesic line according to the adaptive coefficients, so as to obtain the target covariance correction matrix. The cross-modal geometric correction unit is used to keep the reference image embedding unchanged and obtain the corrected text embedding based on the image domain target covariance correction matrix and the integrated text covariance of the prompt words. The anomaly scoring output unit is used to calculate an anomaly score by performing cosine similarity matching between the corrected text embedding and the image embedding to be detected, and to determine whether the image to be detected is abnormal based on the anomaly score.

[0017] Based on the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection, to better realize this invention, an electronic device is further proposed, including a memory and a processor; the memory stores a computer program, and when the computer program is executed on the processor, it implements the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection.

[0018] Based on the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection, to better realize this invention, a computer-readable storage medium is further proposed. This computer-readable storage medium stores computer instructions that, when executed on the aforementioned electronic device, implement the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection.

[0019] The present invention has the following beneficial effects: (1) This invention does not rely directly on the distorted sample covariance, but integrates the sample statistics and reference statistics on the covariance manifold, thereby reducing the impact of small sample estimation error on cross-modal alignment and improving the stability of small sample cross-modal alignment.

[0020] (2) This invention uses adaptive coefficients It automatically determines the reliability of the current sample information, which can avoid underfitting caused by relying entirely on priors, and also avoid overfitting caused by relying entirely on small sample statistics. It has strong scene adaptability and prior reliability.

[0021] (3) This invention uses geodesic interpolation instead of Euclidean linear interpolation to ensure that the interpolation result is always within the symmetric positive definite matrix manifold, thus avoiding the problem of negative eigenvalues ​​or numerical instability and maintaining the geometric validity of the covariance matrix.

[0022] (4) This invention improves the reliability of text prompts in small sample anomaly detection and reduces the impact of text semantic noise by normalizing and aggregating synonym descriptions, attribute descriptions and logical rule descriptions, and combining similarity calculation after geometric correction.

[0023] (5) This invention can be used for detecting appearance defects such as scratches and damage, as well as for detecting logical anomalies such as quantity errors, missing parts, and incorrect positional relationships. It is suitable for detecting both appearance defects and logical anomalies. Attached Figure Description

[0024] Figure 1 The flowchart of the cross-modal adaptive geometric correction method for small-sample industrial anomaly detection provided by the present invention is shown.

[0025] Figure 2 The diagram shows the structure of the adaptive geometric correction module provided by this invention.

[0026] Figure 3 The adaptive coefficients provided for this invention Calculation flowchart.

[0027] Figure 4 This is a schematic diagram of manifold geodesic interpolation and modal gap correction provided by the present invention.

[0028] Figure 5 This is a visual comparison of the detection effects of the method of the present invention and existing anomaly detection methods on industrial defect samples. Detailed Implementation

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments, and therefore should not be regarded as a limitation on the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "set up," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0031] Example 1: This embodiment proposes an adaptive geometric modal alignment method for small-sample industrial anomaly detection, which specifically includes the following steps: Step S1: Acquire the normal reference image, the image to be inspected, and the text description of the industrial object to be inspected; Step S2: Based on the normal reference image, the image to be detected, and the text description of the defect category obtained by encoding the pre-trained visual language model, obtain the reference image embedding, the image to be detected embedding, and the text embedding; Step S2 specifically includes the following steps: Step S21: Input the normal reference image and the image to be detected into the pre-trained visual encoder to obtain the reference image embedding set and the image to be detected embedding; Step S22: Input the text prompt into the pre-trained text encoder to obtain the original text embedding and construct the text embedding set.

[0032] Step S3: Estimate the mean and covariance of the image domain samples based on the reference image embedding, and construct the isotropic symmetric positive definite reference covariance of the image domain based on the set scale parameters and identity matrix. Step S3 specifically includes the following steps: Step S31: Normalize the sets of image embeddings and text embeddings to be detected; Step S32: Calculate the mean of the image domain samples based on the reference image embedding; Step S33: Calculate the image domain sample covariance based on the reference image embedding and the set regularization coefficients; Step S34: Construct an isotropic symmetric positive definite reference covariance in the image domain based on the image domain sample covariance, the set scale parameter, and the identity matrix.

[0033] Step S34 specifically includes the following steps: Step S341: Based on the image domain sample covariance and the set feature dimensions of the visual embedding, call the matrix trace operation to calculate the image domain reference scale parameters; Step S342: Construct an isotropic symmetric positive definite reference covariance in the image domain based on the image domain reference scale parameter and the unit matrix under the set visual embedding feature dimension.

[0034] Step S4: Take the sample covariance and the image domain isotropic symmetric positive definite reference covariance as symmetric positive definite matrices and place them on the covariance manifold to calculate the adaptive coefficients; Step S4 specifically includes the following steps: Step S41: Perform feature decomposition on the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance to obtain the direction vector and scale vector respectively; Step S42: Calculate the direction difference term and scale difference term based on the direction vector and scale vector; Step S43: Based on the set stability term, orientation difference term, and scale difference term, use the total change of the isotropic symmetric positive definite reference covariance in the image domain as the normalization factor, and call the clip() function to obtain the adaptive coefficients.

[0035] Step S5: Based on the adaptive coefficients, interpolate the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance on the covariance manifold geodesic line to obtain the target covariance correction matrix. Step S6: Keeping the reference image embedding unchanged, obtain the corrected text embedding based on the image domain target covariance correction matrix and the integrated text covariance of the prompt words; Step S6 specifically includes the following steps: Step S61: Calculate the mean of the integrated text embeddings of the prompt words and the covariance of the integrated text embeddings of the prompt words based on the text embedding set; Step S62: Whiten the original text embedding based on the integrated text covariance of the prompt words, and color it according to the image domain target covariance correction matrix to obtain the corrected text embedding.

[0036] Step S7: Perform cosine similarity matching between the corrected text embedding and the image embedding to be detected, calculate the anomaly score, and determine whether there is anomaly in the image to be detected based on the anomaly score.

[0037] Step S7 specifically includes the following steps: Step S71: Based on the image embedding to be detected and the corrected text embedding, call the cosine similarity to calculate the anomaly score; Step S72: Determine whether there is an anomaly in the image to be detected based on the set threshold and the anomaly score; if the anomaly score is greater than the set threshold, it is determined that there is an anomaly in the image to be detected; if the anomaly score is less than the set threshold, it is determined that there is no anomaly in the image to be detected.

[0038] Working Principle: This embodiment first uses a pre-trained vision-language model to extract visual and text embeddings from a small number of normal reference images, images to be detected, and defective text descriptions. Then, it estimates the image domain sample covariance based on the small sample visual embeddings and constructs a stable symmetric positive definite covariance by shrinking it towards an isotropic symmetric positive definite reference covariance. The shrinkage strength is adaptively determined by the directional difference, scale difference, and small sample estimation uncertainty between the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance. Subsequently, covariance interpolation is performed in log-Euclidean space to obtain the target covariance correction matrix. Further, combined with the text covariance obtained from cue word integration, a whitening-coloring transformation is constructed to geometrically correct the text embeddings while keeping the visual embeddings unchanged. Finally, the anomaly score is calculated based on the cosine similarity between the corrected text embeddings and the visual embeddings, outputting image-level anomaly detection results and pixel-level anomaly localization results. This method can suppress the influence of covariance estimation distortion and text semantic noise on cross-modal matching when the number of samples is much smaller than the feature dimension, thereby improving the stability and accuracy of small sample industrial anomaly detection.

[0039] Example 2: This embodiment is based on Embodiment 1 above, and will be described in detail with a specific embodiment.

[0040] Step 1: Obtain a small number of normal reference images, images to be detected, and text descriptions related to defect categories or logical anomalies of the industrial object to be inspected, and construct a small-sample cross-modal detection input; Step 2: Encode the normal reference image, the image to be detected, and the text description using a pre-trained vision-language model to obtain the reference image embedding. Image embedding to be detected and text embedding The pre-trained visual-language model includes a visual encoder and a text encoder, wherein the text encoder is an encoding network pre-trained through image-text contrast learning or image-text matching tasks; In step 1 or step 2, semantic normalization and embedding aggregation are performed on multiple text descriptions related to the same defect semantics. The multiple text descriptions include at least one of synonym expressions, category descriptions, attribute descriptions, and logical rule descriptions. The impact of semantic ambiguity and redundant modifications in a single text description on cross-modal alignment is reduced by embedding the aggregated text.

[0041] Step 3: Calculate the image domain sample statistics based on the reference image embedding, including at least the image domain sample mean. Image domain sample covariance And construct an isotropic symmetric positive definite reference covariance in the image domain. ; In step 3 of this embodiment, the sample covariance is calculated from the embedding of N normal reference images, and N is less than the feature dimension d of the embedding. Before calculating the sample covariance, the reference image embedding is centered, normalized and regularized to ensure that the sample covariance is a symmetric positive definite matrix.

[0042] The image domain isotropic symmetric positive definite reference covariance mentioned in step 3 Construct it in the following form: ,in, For the image domain reference scale parameter, It is a d-dimensional identity matrix; the image domain reference scale parameter Determined in the following manner: ,in, Represents the matrix trace operation. denoted as the regularized image domain sample covariance, and d represents the feature dimension of the visual embedding.

[0043] Step 4: Calculate the image domain sample covariance. and the isotropic symmetric positive definite reference covariance in the image domain As a symmetric positive definite matrix, it is placed on the covariance manifold, and based on the directional difference term between the two... and scale difference term Calculate the adaptive coefficients in the image domain ,in ∈[0,1]; The adaptive coefficient This is used to automatically balance sample information and reference covariance information. When the difference between the sample covariance and the isotropic, symmetric, positive definite reference covariance in the image domain is small... A value approaching 0 indicates that the current sample information is insufficient to significantly correct the reference statistic; when the sample covariance changes significantly in direction or scale relative to the isotropic, symmetric, positive definite reference covariance in the image domain... An increase indicates a need to make better use of the scene-specific information in the current small sample.

[0044] The adaptive coefficient mentioned in step 4 Determined jointly by the direction difference term and the scale difference term; the direction difference term The scale difference term is used to characterize the degree of deviation between the principal directions or feature subspaces of the sample covariance and the isotropic symmetric positive definite reference covariance in the image domain. Used to characterize the degree of deviation between the two eigenvalues ​​or variance magnitudes, and the directional difference term and scale difference term are mapped to the [0,1] interval by a normalization function.

[0045] The adaptive coefficients under small sample conditions The estimation error decreases as the number of reference samples N increases, and its upper bound is... The same order ensures that as the number of samples increases, the target covariance correction matrix consistently converges to the alignment matrix obtained based on the true sample distribution.

[0046] Step 5: Based on the image domain adaptive coefficients The image domain sample covariance in log-Euclidean space and the isotropic symmetric positive definite reference covariance in the image domain Interpolation is performed to obtain the target covariance correction matrix in the image domain. ; The adaptive coefficient Determined in the following manner: ,in Indicates the difference in direction. Indicates the scale difference term. The normalized scale or overall variation of the isotropic, symmetric, positive definite reference covariance in the image domain. To avoid stable terms with a denominator of zero, This means that the calculation result will be truncated to the interval [0,1].

[0047] The geodesic interpolation described in step 5 uses a log-Euclidean metric; when using a log-Euclidean metric, the target covariance correction matrix... satisfy ,in This represents the isotropic, symmetric, positive definite reference covariance in the image domain. Indicates the sample covariance. Represents the matrix logarithmic mapping, This represents the matrix exponential mapping.

[0048] The manifold geodesic interpolation is used to avoid the problem of non-positive definite covariance matrix or geometric structure destruction caused by ordinary Euclidean linear interpolation, so that each matrix obtained in the interpolation process remains a valid symmetric positive definite matrix.

[0049] Step 6: Preserve the embedding of the image to be detected and reference image embedding The image domain target covariance correction matrix remains unchanged. Integrating text covariance with cue words Construct a whitening-coloring transform, perform geometric correction on the text embedding, and obtain the corrected text embedding. ; In step 6, the visual embedding remains unchanged, and the correction is based on the image domain target covariance matrix. Integrating text covariance with cue words Construct a whitening-coloring transformation to embed the original text. Perform correction to obtain the corrected text embedding. ;in, Indicates the first Text prompt The original text embedding is obtained by encoding with a pre-trained text encoder, wherein the pre-trained text encoder is a text-side encoding network in a pre-trained vision-language model. Indicates to The corrected text embedding obtained after whitening-coloring transformation, wherein the cue words integrate text covariance This data was obtained from statistics embedded in multiple text prompts.

[0050] The corrected text embedding Obtained in the following manner: in, The mean of the integrated text embedding of the prompt words. Represents the mean of the image domain samples. This indicates the integrated text covariance of the cue words. Represents the covariance of image domain samples and image domain isotropic symmetric positive definite reference covariance The image domain target covariance correction matrix obtained by interpolation in logarithmic Euclidean space.

[0051] Step 7: Calculate the similarity or anomaly score between the image and the text based on the corrected cross-modal embedding, and output the image-level anomaly detection result and pixel-level anomaly localization result based on the anomaly score.

[0052] The anomaly score mentioned in step 7 is obtained by the similarity difference between the corrected image embedding and the normal text description, abnormal text description, or defect category text description; when the image to be detected is divided into image blocks or pixel regions, the anomaly score is calculated for each image block or pixel region to generate a pixel-level anomaly heatmap.

[0053] Working principle: The industrial anomaly detection in this embodiment includes the detection of appearance defects of industrial objects and the detection of logical anomalies; the logical anomalies include at least one of quantity errors, position errors, structural relationship errors, missing parts, and mismatched parts.

[0054] The other parts of this embodiment are the same as those in Embodiment 1 above, so they will not be described again.

[0055] Example 3: This embodiment is based on any one of Embodiments 1-2 above, such as Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5As shown in Table 1, a specific embodiment will be described in detail.

[0056] like Figure 1 As shown, the input construction process for small-sample cross-modal anomaly detection is as follows: For the industrial object to be inspected, a small number of normal reference images are collected as supporting samples, for example, four normal sample images are collected for each category. Simultaneously, the images to be inspected are acquired, and a text description set is constructed. The text description set can include normal descriptions, abnormal descriptions, defect category descriptions, and logical rule descriptions. For example, for assembly products, logical rule texts such as "the number of screws should be 15" and "the component position should be consistent with the template" can be set; for appearance defects, defect category texts such as "scratches exist on the surface" and "damage exists on the surface" can be set. The construction process of this small-sample cross-modal detection input can be referred to... Figure 1 As shown in S1.

[0057] To reduce the impact of text noise, synonyms with the same semantic defects are normalized, or multiple synonymous texts are input into a pre-trained text encoder and then embedded and aggregated. Aggregation methods can include average aggregation, weighted average aggregation, or similarity-based filtering aggregation.

[0058] like Figure 1 , Figure 2 As shown, the cross-modal feature encoding process is as follows: Input the normal reference image and the image to be detected into the pre-trained visual encoder to obtain the reference image embedding set. And the image to be detected is embedded in q. Let the set of text prompts be... ,in Indicates the first M represents the number of text prompts. (The text prompts are then displayed.) Input the pre-trained text encoder to obtain the first... Original text embedding This leads to a set of text embeddings. In pixel-level anomaly localization scenarios, the image to be detected can be further divided into multiple image blocks or local regions, and local image embeddings can be extracted separately. The above cross-modal feature encoding process can be referred to... Figure 1 S2 and Figure 2 As shown.

[0059] To improve the comparability between different embeddings, L2 normalization was performed on image embeddings and text embeddings, and the mean of image domain samples was calculated based on the normal reference image embedding. Image domain sample covariance To avoid matrix singularity issues caused by N being much smaller than d, a regularization term can be added to the image domain sample covariance: in, The regularization coefficient is . It is a d-dimensional identity matrix. The statistical estimation process described above can be referred to... Figure 1 As shown in S3.

[0060] like Figure 2 , Figure 3 As shown, the adaptive coefficients The calculation process is as follows: Constructing an isotropic, symmetric, positive definite reference covariance in the image domain .in, ,in The image domain reference scale parameter is the image domain reference scale parameter. Determined in the following manner: ,in, Represents the matrix trace operation. Let d represent the regularized image domain sample covariance, and let d represent the feature dimension of the visual embedding. It is a d-dimensional identity matrix. The image domain isotropic symmetric positive definite reference covariance is used to provide a stable universal geometric reference, and the image domain sample covariance... Used to provide scene-specific distribution information for current industrial objects. The structure of the common input adaptive geometric correction module can be found in [reference]. Figure 2 As shown.

[0061] Image domain sample covariance and image domain isotropic symmetric positive definite reference covariance Perform feature decomposition separately to obtain the corresponding orientation and scale components:

[0062]

[0063] in, and These represent the directional components of the image domain sample covariance and the reference covariance, respectively. and These represent the corresponding scale components. The directional difference term is calculated based on the degree of deviation between the directional components of the two covariance matrices. Calculate the scale difference term based on the differences between scale components. .

[0064]

[0065]

[0066] Furthermore, the orientation difference term and scale difference term are fused, and the overall change in the isotropic symmetric positive definite reference covariance in the image domain is utilized. Normalization is performed to obtain the image domain adaptive coefficients. The calculation process for directional and scale differences is as follows: Figure 3 As shown.

[0067] in, It can be calculated using the following formula: .in, This is a stabilizing term used to avoid the denominator being zero; This indicates that the calculation result is restricted to the range [0,1]. This coefficient automatically reflects the reliability of the current sample statistic relative to the reference statistic. The result is obtained through this normalization and truncation process. Used to control the fusion ratio of sample information and reference covariance information in subsequent manifold interpolation, such as Figure 3 As shown.

[0068] like Figure 4 As shown, the geodesic interpolation calculation process on the covariance manifold is as follows: The covariance matrix belongs to the set of symmetric positive definite matrices, which has a non-Euclidean geometry. Direct linear interpolation in Euclidean space may violate the positive definiteness of the matrix and lead to numerical instability. Therefore, this embodiment performs geodesic interpolation on the covariance manifold. The relationship between the covariance matrix manifold and the geodesic interpolation can be found in [reference needed]. Figure 4 As shown.

[0069] This section uses log-Euclidean interpolation to calculate the target covariance correction matrix. satisfy: .in For matrix logarithmic mapping, This is a matrix exponential mapping. Because... and All are symmetric positive definite matrices, therefore It remains a symmetric positive definite matrix.

[0070] The cross-modal embedding geometry correction process is as follows: Based on the target covariance correction matrix in the image domain Construct a geometric correction transformation. In this embodiment, the reference image is embedded. While remaining unchanged, the geometric correction transformation is applied to the text embedding to adjust the covariance structure of the text embedding to be consistent with or approximately consistent with the covariance structure of the target in the image domain.

[0071] Based on text embedding set Calculate the mean of the integrated text embedding of prompt words. Integrating text covariance with cue words .in, Indicates text prompt The original text embedding obtained by encoding with a pre-trained text encoder. Corrected text embedding. Indicates embedding of the original text The text embedding obtained after whitening-coloring transformation.

[0072] This section embeds the original text. A whitening-coloring transformation is performed to make the distribution shape of the text embedding consistent with or approximately consistent with the target distribution shape of the reference image embedding. Specifically, the text covariance is first integrated using cue words. Embedding of raw text Whitening is performed, followed by the use of the image domain target covariance correction matrix. Coloring is performed to obtain corrected text embeddings. :

[0073] in, Represents the mean of the image domain samples. This represents the target covariance correction matrix in the image domain.

[0074] After completing the text embedding geometric correction, preserve the embedding of the image to be detected. Remain unchanged, and use corrected text embedding. Cosine similarity matching is performed with the image embedding to be detected, thereby avoiding the introduction of additional transformations on the visual embedding side and maintaining the discriminative structure of the original visual feature space.

[0075] like Figure 5 As shown in Table 1, the process of calculating and outputting the anomaly score is as follows: Calculate the cosine similarity between the image embedding to be detected and the corrected text embedding. For image-level anomaly detection, the anomaly score can be calculated as follows: ,in, This represents the visual embedding of the image x to be detected. This represents the j-th corrected text embedding. When the anomaly score exceeds a preset threshold, the image to be detected is determined to be abnormal. The anomaly scoring and output process can be referenced... Figure 1 S7 and Figure 2 The bottom output structure is shown.

[0076] For pixel-level anomaly localization, the image to be detected can be divided into multiple image blocks, and an anomaly score can be calculated for each block. These scores are then mapped back to the image space to form an anomaly heatmap. After normalization, smoothing, and thresholding of the anomaly heatmap, the defect region is output. The anomaly heatmap and localization performance of different methods on industrial defect samples can be found in [reference needed]. Figure 5 As shown.

[0077] Figure 5The different methods involved include: SubspaceAD method generates anomaly responses by constructing a feature subspace of normal samples and measuring the deviation of the sample to be detected from this subspace; LogSAD method focuses on modeling logical anomalies and structural relationship anomalies in industrial products and outputs corresponding anomaly heatmaps; MRAD-TF method enhances the response of anomaly regions by using feature transformation and fusion of multiple reference samples; RAD method estimates anomaly regions by utilizing the difference relationship between reference samples and the sample to be detected; AA-CLIP method is based on a pre-trained visual language model and enhances the ability to identify open-category anomalies through anomaly perception cues or semantic alignment; the method in this paper corrects the distribution difference between text embedding and image embedding under small sample conditions by adaptive geometric modal alignment to improve the focus of the anomaly heatmap on the real defect region.

[0078] In the experimental implementation, this embodiment can be applied to the MVTec LOCO industrial anomaly detection dataset. The MVTecLOCO dataset targets the detection of logical constraint anomalies and structural relationship anomalies in industrial products, containing images of various industrial objects. Anomalies include not only local appearance defects but also logical anomalies such as quantity errors, location errors, structural relationship errors, missing parts, and mismatched parts. In the experiment, a small-sample detection setting can be constructed with only a few normal reference images provided for each type of industrial object. For example, four normal reference images can be used as support samples, and the detection performance can be evaluated using metrics such as image-level AUC, image-level AP, pixel-level AUC, and pixel-level AP. Experimental results show that this embodiment can improve the stability of cross-modal matching under small-sample conditions and improve the detection effect of image-level and pixel-level anomalies. A comparison of the quantitative metrics between this embodiment and existing methods is shown in Table 1. A visual comparison of the detection effect on specific industrial defect samples is available in [reference needed]. Figure 5 As shown.

[0079] Table 1 Comparison results on MVTec LOCO (4-shot)

[0080] This embodiment can be deployed in a vision inspection system on an industrial production line, and can be used in conjunction with industrial cameras, edge computing devices, or server-side inspection platforms. This method requires only a small number of normal reference samples and text descriptions to achieve anomaly detection, making it suitable for industrial scenarios where sample collection is difficult, anomaly categories are open, and product models change frequently, thus demonstrating good industrial applicability.

[0081] The other parts of this embodiment are the same as any one of the above embodiments 1-2, so they will not be described again.

[0082] Example 4: Based on any one of Embodiments 1-3 above, this embodiment proposes an adaptive geometric modal alignment system for small-sample industrial anomaly detection, based on the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection, to execute the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection. The system includes a data input unit, a feature encoding unit, a statistical estimation unit, an adaptive coefficient calculation unit, a manifold interpolation unit, a cross-modal geometric correction unit, and an anomaly score output unit. The data input unit is used to acquire a normal reference image, an image to be detected, and a text description of the industrial object to be detected. The feature encoding unit is used to encode the normal reference image, the image to be detected, and the text description of the defect category obtained by the pre-trained visual language model to obtain the reference image embedding, the image to be detected embedding, and the text embedding. The statistical estimation unit is used to estimate the mean and covariance of image domain samples based on the reference image embedding, and to construct an isotropic symmetric positive definite reference covariance in the image domain based on the set scale parameters and identity matrix. The adaptive coefficient calculation unit is used to take the sample covariance and the image domain isotropic symmetric positive definite reference covariance as symmetric positive definite matrices and place them on the covariance manifold to calculate the adaptive coefficients. The manifold interpolation unit is used to interpolate the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance on the covariance manifold geodesic line according to the adaptive coefficients, so as to obtain the target covariance correction matrix. The cross-modal geometric correction unit is used to keep the reference image embedding unchanged and obtain the corrected text embedding based on the image domain target covariance correction matrix and the integrated text covariance of the prompt words. The anomaly scoring output unit is used to calculate an anomaly score by performing cosine similarity matching between the corrected text embedding and the image embedding to be detected, and to determine whether the image to be detected is abnormal based on the anomaly score.

[0083] This embodiment also proposes an electronic device, including a memory and a processor; the memory stores a computer program, and when the computer program is executed on the processor, it implements the above-described adaptive geometric modal alignment method for small-sample industrial anomaly detection.

[0084] This embodiment also proposes a computer-readable storage medium storing computer instructions. When the computer instructions are executed on the aforementioned electronic device, they implement the aforementioned adaptive geometric modal alignment method for small-sample industrial anomaly detection.

[0085] The other parts of this embodiment are the same as any one of the embodiments 1-3 above, so they will not be described again.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. An adaptive geometric modal alignment method for small-sample industrial anomaly detection, characterized in that, Specifically, the following steps are included: Step S1: Obtain the normal reference image, the image to be detected, and the text description of the industrial object to be inspected; Step S2: Based on the normal reference image, the image to be detected, and the text description of the defect category obtained by encoding the pre-trained visual language model, obtain the reference image embedding, the image to be detected embedding, and the text embedding; Step S3: Estimate the mean and covariance of the image domain samples based on the reference image embedding, and construct the isotropic symmetric positive definite reference covariance of the image domain based on the set scale parameters and identity matrix. Step S4: Take the sample covariance and the image domain isotropic symmetric positive definite reference covariance as symmetric positive definite matrices and place them on the covariance manifold to calculate the adaptive coefficients; Step S5: Based on the adaptive coefficients, interpolate the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance on the covariance manifold geodesic line to obtain the target covariance correction matrix. Step S6: Keeping the reference image embedding unchanged, obtain the corrected text embedding based on the image domain target covariance correction matrix and the integrated text covariance of the prompt words; Step S7: Perform cosine similarity matching between the corrected text embedding and the image embedding to be detected, calculate the anomaly score, and determine whether there is anomaly in the image to be detected based on the anomaly score; Step S4 specifically includes the following steps: Step S41: Perform feature decomposition on the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance to obtain the image domain sample covariance direction vector, the image domain isotropic symmetric positive definite reference covariance direction vector, the image domain sample covariance scale vector, and the image domain isotropic symmetric positive definite reference covariance scale vector. Step S42: Calculate the direction difference term and scale difference term based on the image domain sample covariance direction vector, the image domain isotropic symmetric positive definite reference covariance direction vector, the image domain sample covariance scale vector, and the image domain isotropic symmetric positive definite reference covariance scale vector. Step S43: Based on the set stability term, orientation difference term, and scale difference term, use the total change of the isotropic symmetric positive definite reference covariance in the image domain as the normalization factor, and call the clip() function to obtain the adaptive coefficients. Step S6 specifically includes the following steps: Step S61: Calculate the mean of the integrated text embeddings of the prompt words and the covariance of the integrated text embeddings of the prompt words based on the text embedding set; Step S62: Whiten the original text embedding based on the integrated text covariance of the prompt words, and color it according to the image domain target covariance correction matrix to obtain the corrected text embedding.

2. The adaptive geometric modal alignment method for small-sample industrial anomaly detection according to claim 1, characterized in that, Step S2 specifically includes the following steps: Step S21: Input the normal reference image and the image to be detected into the pre-trained visual encoder to obtain the reference image embedding set and the image to be detected embedding; Step S22: Input the text prompt into the pre-trained text encoder to obtain the original text embedding and construct the text embedding set.

3. The adaptive geometric modal alignment method for small-sample industrial anomaly detection according to claim 1, characterized in that, Step S3 specifically includes the following steps: Step S31: Normalize the sets of image embeddings and text embeddings to be detected; Step S32: Calculate the mean of the image domain samples based on the reference image embedding; Step S33: Calculate the image domain sample covariance based on the reference image embedding and the set regularization coefficients; Step S34: Construct an isotropic symmetric positive definite reference covariance in the image domain based on the image domain sample covariance, the set scale parameter, and the identity matrix.

4. The adaptive geometric modal alignment method for small-sample industrial anomaly detection according to claim 3, characterized in that, Step S34 specifically includes the following steps: Step S341: Based on the image domain sample covariance and the set feature dimensions of the visual embedding, call the matrix trace operation to calculate the image domain reference scale parameters; Step S342: Construct an isotropic symmetric positive definite reference covariance in the image domain based on the image domain reference scale parameter and the unit matrix under the set visual embedding feature dimension.

5. The adaptive geometric modal alignment method for small-sample industrial anomaly detection according to claim 1, characterized in that, Step S7 specifically includes the following steps: Step S71: Based on the image embedding to be detected and the corrected text embedding, call the cosine similarity to calculate the anomaly score; Step S72: Determine whether there is an anomaly in the image to be detected based on the set threshold and the anomaly score; if the anomaly score is greater than the set threshold, it is determined that there is an anomaly in the image to be detected; if the anomaly score is less than the set threshold, it is determined that there is no anomaly in the image to be detected.

6. An adaptive geometric modal alignment system for small-sample industrial anomaly detection, used to execute the adaptive geometric modal alignment method for small-sample industrial anomaly detection as described in claim 1, characterized in that, It includes a data input unit, a feature encoding unit, a statistical estimation unit, an adaptive coefficient calculation unit, a manifold interpolation unit, a cross-modal geometric correction unit, and an anomaly scoring output unit; The data input unit is used to acquire a normal reference image, an image to be detected, and a text description of the industrial object to be detected. The feature encoding unit is used to encode the normal reference image, the image to be detected, and the text description of the defect category obtained by the pre-trained visual language model to obtain the reference image embedding, the image to be detected embedding, and the text embedding. The statistical estimation unit is used to estimate the mean and covariance of image domain samples based on the reference image embedding, and to construct an isotropic symmetric positive definite reference covariance in the image domain based on the set scale parameters and identity matrix. The adaptive coefficient calculation unit is used to take the sample covariance and the image domain isotropic symmetric positive definite reference covariance as symmetric positive definite matrices and place them on the covariance manifold to calculate the adaptive coefficients. The manifold interpolation unit is used to interpolate the image domain sample covariance and the image domain isotropic symmetric positive definite reference covariance on the covariance manifold geodesic line according to the adaptive coefficients, so as to obtain the target covariance correction matrix. The cross-modal geometric correction unit is used to keep the reference image embedding unchanged and obtain the corrected text embedding based on the image domain target covariance correction matrix and the integrated text covariance of the prompt words. The anomaly scoring output unit is used to calculate an anomaly score by performing cosine similarity matching between the corrected text embedding and the image embedding to be detected, and to determine whether the image to be detected is abnormal based on the anomaly score.

7. An electronic device, characterized in that, It includes a memory and a processor; the memory stores a computer program that, when executed on the processor, implements an adaptive geometric modal alignment method for small-sample industrial anomaly detection as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on the electronic device as described in claim 7, implement an adaptive geometric modal alignment method for small-sample industrial anomaly detection as described in any one of claims 1-5.

Images