Unsupervised multi-mode industrial anomaly detection method based on self-correlation learning

Abstract

The invention belongs to the technical field of industrial artificial intelligence, and particularly relates to an unsupervised multi-mode industrial anomaly detection method based on self-correlation learning. The method comprises the following steps: acquiring a to-be-detected RGB image and depth map data, and generating a multi-modal pseudo sample based on Berlin noise and texture filling so as to enhance the discrimination capability of a model for abnormity; rGB and deep modal features are fused by using comparative learning, and interaction information between modals is extracted; calculating local association and global association of the features by using a learnable Gaussian kernel function and a self-attention mechanism respectively; and finally, the association difference between the local association and the global association is used as a discrimination standard, and the reconstruction error is combined to calculate an anomaly score, so that the tiny anomaly on the surface of the industrial product is accurately detected and positioned. According to the method, the self-correlation learning framework which fuses the multi-modal features and uses the global-local correlation difference as the anomaly discrimination basis is constructed, so that high-precision anomaly positioning is realized under the condition of lack of anomaly sample labeling.

Description

Technical Field

[0001] This invention belongs to the field of industrial artificial intelligence technology, specifically relating to an unsupervised multimodal industrial anomaly detection method based on autocorrelation learning. Background Technology

[0002] With the deepening of global Industry 4.0 and smart manufacturing strategies, improving product quality and production line efficiency has become a core objective in the industrial sector. Industrial anomaly detection, as a crucial link in production quality control, is increasingly important. Traditional quality inspection methods mainly rely on manual visual inspection, which suffers from inherent drawbacks such as low efficiency, high cost, and susceptibility to operator fatigue, making it difficult to meet the demands of high-paced, high-precision large-scale production. In recent years, industrial artificial intelligence technologies based on deep learning and computer vision have provided revolutionary solutions for achieving automated, high-precision, and robust anomaly detection. In particular, unsupervised learning methods, by modeling the feature distribution of normal patterns using only a large number of normal samples for training, have shown enormous application potential and broad market prospects in situations where anomaly sample data is scarce or difficult to collect in industrial settings, becoming an important direction for the development of current industrial intelligent inspection technologies.

[0003] However, existing unsupervised anomaly detection techniques still face significant challenges in practical industrial applications. These include misjudgments due to complex workpiece surface textures, the diversity of anomaly samples in shape and scale, and poor detection performance for minute and subtle defects. More importantly, most existing methods rely solely on single RGB image information for analysis, failing to fully utilize modal data with complementary structural information, such as depth maps, thus limiting robustness and accuracy in complex environments. This invention addresses these technical shortcomings and practical needs by introducing a multimodal feature fusion mechanism and an autocorrelation learning framework. The aim is to overcome the problems of insufficient single-modal information and the difficulty in detecting minute anomalies, achieving more accurate and robust unsupervised detection of defects in industrial products. Summary of the Invention

[0004] The purpose of this invention is to propose an unsupervised multimodal industrial anomaly detection method based on autocorrelation learning, which aims to solve the problems of scarce anomaly samples, diverse anomaly scales, and insufficient fusion of multimodal information that are common in existing industrial anomaly detection methods, thereby improving the detection accuracy, robustness, and deployability of small anomalies in industrial scenarios.

[0005] This invention constructs an autocorrelation learning framework that integrates multimodal features and uses global-local correlation differences as the basis for anomaly discrimination, which can achieve high-precision anomaly localization even in the absence of anomaly sample annotations.

[0006] This invention provides an unsupervised multimodal industrial anomaly detection method based on autocorrelation learning, comprising: acquiring RGB images and depth maps of normal samples to construct pseudo-anomaly samples; performing multimodal feature extraction and fusion; introducing an autocorrelation learning module to calculate global correlation, local correlation, and correlation differences; and finally obtaining pixel-level anomaly regions based on correlation differences, thereby achieving accurate identification of minute defects on the surface of industrial parts. This method can be trained without real anomaly samples, and has stronger generalization ability and practical deployment value. The specific steps are as follows:

[0007] Step 1: Multimodal data acquisition and preprocessing;

[0008] The process involves acquiring RGB images of the industrial product to be inspected and spatially aligned single-channel depth map data. The depth map characterizes the three-dimensional geometric structure of the workpiece surface and complements the appearance and texture information of the RGB image. Data preprocessing includes: first, scaling the RGB image and depth map to a preset input size to meet the size requirements of the feature extraction network; second, normalizing the mean and variance of the RGB image to reduce brightness and color deviations caused by different acquisition environments; and third, linearly scaling the depth map to map its pixel values ​​to the [0, 1] interval to reduce amplitude differences caused by different depth sensors or acquisition conditions. Finally, the processed multimodal image is divided into fixed-size image blocks to provide structured input for subsequent patch-level feature extraction based on Vision Transformer.

[0009] Step 2: Generation of multimodal pseudo-samples;

[0010] The purpose of constructing a pseudo-sample generation module is to solve the problem of the scarcity of abnormal samples in industrial anomaly detection. This module generates RGB pseudo-samples and depth pseudo-samples by introducing artificially simulated anomalies into real samples containing only normal patterns. Specifically, it uses Berlin noise with smooth randomness to generate a binary mask to determine the spatial distribution area of ​​anomalies. This mask can simulate potential defects with multiple scales and irregular shapes.

[0011] For RGB images, the masked area is filled with a preset random texture to make the area exhibit visual features of structural damage or surface inconsistency, while keeping the original information of the non-masked area unchanged, so as to avoid generating overly simple or unrealistic abnormal patterns.

[0012] For depth images, anomalous depth values ​​are simulated by random perturbations that satisfy a standard normal distribution, and injected into the mask region by weighted fusion or direct replacement to realistically represent surface depressions, bulges or other geometric anomalies.

[0013] In this way, the generated pseudo-samples are used together with real samples without abnormalities to participate in model training, thereby enhancing the model's ability to distinguish between normal and abnormal structural patterns.

[0014] Step 3: Multimodal feature fusion;

[0015] A multimodal feature fusion module is constructed to fully extract complementary key information from the RGB and deep modalities. Specifically, firstly, the real or pseudo-samples obtained in step 2 are input into a pre-trained feature extraction network, preferably a Vision Transformer (ViT), to obtain RGB feature sequences and deep feature sequences respectively. Subsequently, a contrastive learning mechanism is introduced to achieve multimodal feature fusion, including aligning RGB and deep feature blocks at the same spatial location in the feature space, making them close to each other in the embedding space, while maintaining a large distance from features at other locations or other samples. This contrastive constraint can maximize the shared semantic and structural information across modalities, ensuring that the fused representation has both modal consistency and complementarity. Finally, the two types of modal features optimized by contrastive learning are concatenated or weighted along the channel dimension to form a richer, more robust, and more expressive multimodal fusion feature.

[0016] Step 4: Autocorrelation learning;

[0017] An autocorrelation learning module is constructed to calculate and analyze the inter-patch correlations within fused features, in order to establish an accurate model of normal patterns; this module is used to simultaneously calculate the local and global correlations of features.

[0018] Local correlation computation: A learnable Gaussian kernel function is introduced to compute the local dependencies between each patch and its neighboring patches; the scale parameter of the Gaussian kernel function is learnable, enabling the model to adaptively focus on short-range feature associations that are sensitive to minor anomalies; local correlation focuses on modeling subtle consistency of texture and local structure.

[0019] Global correlation computation: Introducing a self-attention mechanism to calculate the global dependencies between each patch and all other patches; global correlation computation captures the macroscopic consistency of the overall structure, shape, and long-distance repeating patterns of industrial workpieces;

[0020] The autocorrelation learning module trains the model through contrastive learning, enabling the local and global correlations of normal samples to maintain a specific correlation pattern, thereby achieving an accurate characterization of normal features; that is, forming an accurate model of the distribution of normal features, providing a reliable basis for subsequent anomaly detection.

[0021] Step 5: Determine correlation differences and calculate anomaly scores;

[0022] When evaluating model performance and performing anomaly detection, the difference between local and global correlations is used as the core criterion. That is, anomaly identification and score calculation are completed based on the difference between local and global correlations. During model training, the optimization objective is used to make the local and global correlations of normal samples significantly different: the local texture relationships of normal regions are usually highly consistent, while their overall structure maintains stable long-distance correlations from a global perspective, thus forming a large correlation difference between the two. In contrast, the local structure of anomaly samples is often disrupted, and their correlation relationships within the overall scope are also disturbed, resulting in a significant reduction in the difference between local and global correlations. To quantify these correlation differences, KL divergence or other suitable distance metrics are used to compare the local and global correlation distributions to obtain a patch-level correlation difference map. Furthermore, this correlation difference is fused with the pixel-level reconstruction error obtained from the feature reconstruction module to comprehensively reflect the dual deviation of the anomaly region in both local and global structure. By thresholding the fused score map, the final anomaly score map can be generated, where regions with higher scores are identified as having anomalies, thereby achieving high-precision, pixel-level localization of surface defects in industrial workpieces.

[0023] Furthermore:

[0024] In step 2, a binary mask is generated using Berlin noise to determine the spatial distribution area of ​​anomalies;

[0025] For RGB images RGB pseudo-samples are generated by filling with a preset texture image. The process is represented as follows:

[0026] ;

[0027] in, It's a mask. The reversal, It is the opacity parameter. This indicates element-wise multiplication;

[0028] For single-channel depth maps Anomaly depth values ​​are simulated using random numbers that follow a standard normal distribution to generate pseudo-sample depth maps. The process is represented as follows:

[0029] ;

[0030] in, It is generated by combining random numbers with the average depth of the depth map.

[0031] Step 3, which describes multimodal feature fusion through a contrastive learning mechanism, specifically includes: inputting RGB and deep modal features from the same spatial location into a multilayer perceptron for projection processing, mapping them to query vectors or key vectors; and applying the InfoNCE loss function to constrain the corresponding features of the two modalities to enhance their matching degree in the feature space, thereby making RGB features and deep features at the same location more consistent in semantic representation. We use cosine similarity to calculate the similarity between vectors and use two trainable fully connected layers to map the feature vectors processed by the multilayer perceptron to query vectors or key vectors, i.e. and ( Indicates the sample index. (Represents the patch index). Loss function Represented as:

[0032] ;

[0033] in, For batch size, For non-zero patch count, This is the matrix transpose; finally, the two modal features processed by the multilayer perceptron are connected along the channel dimension to obtain the fused features.

[0034] In step 4, the calculation of local and global associations of features specifically includes: firstly, projecting the input features into a query vector, a key vector, a value vector, and a learnable parameter vector for local associations. ;

[0035] Computing local correlations: Utilizing a learnable Gaussian kernel function to focus on neighborhood information, the first... The patch and the first Local associations between patches are represented as follows:

[0036] ;

[0037] in, and For patch location, For the reason The obtained scale parameters;

[0038] Calculate global associations: Capture long-distance dependencies using a self-attention mechanism, represented as:

[0039] ;

[0040] in, For query vector, For key vectors, For matrix transpose, Key vector Dimensions This represents the operation of converting attention scores (similarity) into a probability distribution.

[0041] In step 5, the anomaly identification and anomaly score calculation are performed based on the difference between local and global associations. Normal patches typically exhibit a large difference between local and global associations, while the difference between abnormal patches is significantly reduced due to local structural damage and overall association disorder. KL divergence or other suitable distance metrics are used to measure the local and global association distributions (KL divergence is a statistic that measures the difference between two probability distributions) to obtain patch-level association difference values, which are then compared with the reconstructed features. and input features The deviations between them are used together to calculate the final outlier score:

[0042] ;

[0043] in, This indicates the local correlation of the patch calculated using KL divergence. and global association The model learns to distinguish between normal and abnormal patterns by increasing the correlation differences between normal samples and suppressing the correlation differences between abnormal samples during training. This enables the model to effectively detect anomalies.

[0044] Technical features:

[0045] This invention proposes an unsupervised multimodal industrial anomaly detection method based on autocorrelation learning. It integrates RGB texture information and deep structural information, and achieves accurate representation of normal patterns through pseudo-sample generation, contrastive learning feature alignment, and local-global autocorrelation modeling. The method employs Berlin noise to construct pseudo-anomalies to compensate for insufficient anomalous samples, utilizes InfoNCE to apply consistent constraints to cross-modal features, and jointly captures the local and global correlation structures of normal samples through a learnable Gaussian kernel and a self-attention mechanism.

[0046] Functional advantages:

[0047] This invention enables training under unsupervised conditions, significantly reducing data annotation costs; multimodal complementarity enhances the dual recognition capability for texture and structural anomalies; pseudo-sample enhancement and autocorrelation learning improve the generalization ability for unknown anomalies; and combining correlation differences and reconstruction errors to generate pixel-level anomaly score maps achieves high-precision defect localization, exhibiting high robustness, wide applicability, and promising prospects for industrial applications. Attached Figure Description

[0048] Figure 1This is a flowchart of the unsupervised multimodal industrial anomaly detection method based on autocorrelation learning of the present invention.

[0049] Figure 2 This is a sample visualization of anomaly detection. Detailed Implementation

[0050] The present invention will be further described below with reference to the embodiments and accompanying drawings, but the scope of the invention is not limited in any way.

[0051] like Figure 1 As shown, the overall process of the unsupervised multimodal industrial anomaly detection method based on autocorrelation learning proposed in this invention can be divided into the following five main steps: acquiring multimodal input data of industrial products, generating multimodal pseudo-samples for training, extracting and fusing multimodal features, constructing an autocorrelation learning module to establish the association structure of normal patterns, and combining association differences and reconstruction errors for anomaly evaluation. By introducing multimodal information and a global-local association mechanism, this invention can effectively alleviate the detection difficulties caused by the scarcity of anomaly samples, diverse anomaly scales, and complex textures in real-world industrial scenarios, thereby improving the localization accuracy and robustness of anomaly regions.

[0052] The following is a specific implementation process of the present invention based on a typical industrial surface inspection scenario:

[0053] 1. Acquire RGB images and depth maps of industrial products and perform preprocessing;

[0054] In this embodiment, the MVTec 3D-AD dataset is used. [1] and MSC-AD dataset [2] As input data, both datasets contain a large number of RGB and depth image samples of industrial products. The MVTec 3D-AD dataset covers 10 typical industrial objects with over 4,000 samples, including various realistic anomalies such as scratches, indentations, cracks, contamination, and edge defects. It also provides depth images strictly registered with the RGB images, enabling the model to learn both texture and spatial structure information simultaneously. The MSC-AD dataset simulates more complex real-world conditions, including multiple lighting conditions, strong reflections, and rough surface textures, further enhancing the diversity and generalization ability of the training samples.

[0055] To ensure the consistency and validity of the input data, this invention first standardizes the RGB and depth modalities. The RGB image is adjusted to a fixed size using bilinear interpolation and normalized based on the mean and variance of ImageNet, thereby reducing the impact of different imaging devices or lighting scenarios on the model. The depth image obtains the Z-axis depth value through 3D point cloud transformation and uses a local neighborhood-based interpolation method to repeatedly fill in missing pixels, maintaining the continuity of the depth distribution. Furthermore, to accommodate the Transformer network's requirement for patched input, the depth map is copied into three channels, and both the RGB and depth images are divided into fixed-size image patches, ensuring a consistent format for subsequent feature extraction.

[0056] In practical applications, depth maps can be affected by reflections, lighting angles, or sensor noise. The depth processing strategy of this invention emphasizes maintaining local structural stability, thereby providing a more reliable geometric information foundation during the multimodal fusion stage. Furthermore, the standardization of RGB images reduces interference from background complexity and color variations, allowing the model to focus more on the texture and structure of the workpiece itself.

[0057] 2. Generate pseudo-samples based on normal industrial samples;

[0058] In real-world industrial scenarios, anomalous samples are extremely scarce, and some anomalies may not even appear naturally during sample collection. Therefore, this invention constructs a pseudo-sample generation module that simulates and builds multi-morphological, multi-scale pseudo-anomalies on normal samples, thereby enhancing the model's ability to learn the differences between "normal patterns" and "abnormal patterns." The pseudo-sample generation module constructs pseudo-anomalies for both the RGB and depth modalities, ensuring that the model can simultaneously learn texture and structural anomalies.

[0059] First, a mask with a smooth variation pattern and random shape distribution is generated using Berlin noise. This mask can naturally simulate the irregular contours of abnormal areas such as cracks, contamination, and damage. Compared with simple binary random noise, this noise generation method better matches the morphological characteristics of natural defects, thus generating more realistic abnormal area structures. In the RGB modality, this invention uses a preset random texture image to blend and fill the mask area, enabling the generated pseudo-anomalies to exhibit complex texture variations, such as roughness, stripes, and spots, allowing the model to learn the visual features of defects in different materials. To maintain image realism, the area outside the mask remains completely unchanged, thus preventing pseudo-anomalies from visually disrupting the overall structure. In the depth modality, this invention generates depth perturbations based on a standard normal distribution and implants them into the mask area through weighted or direct replacement methods to simulate three-dimensional geometric defects such as depressions and protrusions. Depth pseudo-anomalies compensate for the spatial scale variations that RGB pseudo-anomalies cannot express, enabling the model to identify three-dimensional deformation anomalies. Pseudo-samples are ultimately trained together with real normal samples, significantly improving the model's robustness in anomaly recognition.

[0060] 3. Fuse the acquired multimodal features;

[0061] To maximize the complementary advantages between RGB and depth modes, this invention constructs a multimodal feature fusion module, which uses a pre-trained ViT network to extract high-dimensional patch features for both modes. The feature sequences extracted by ViT have stronger global modeling and patch-level representation capabilities, making them suitable for detecting small-scale defects with uneven distribution on complex industrial surfaces.

[0062] By introducing a contrastive learning mechanism, this invention enhances the synergy between features from different modalities. Within the contrastive learning framework, RGB and depth features from the same spatial location are treated as positive samples, mapped to a unified feature space, making them semantically consistent. Simultaneously, features from different locations are treated as negative samples, maintaining their distinction within the feature space. This contrastive constraint, constructed using the InfoNCE loss function, effectively strengthens the correlation between modalities, ensuring that the fused features contain both rich texture details and stable geometric information. Furthermore, the fused features, concatenated along the channel dimension, possess higher expressive power, enabling the model to capture more refined anomalies based on multimodal inputs.

[0063] 4. Self-related learning;

[0064] The autocorrelation learning module is the core innovation of this invention. Its goal is to model the local and overall structures within the fused features, thereby capturing stable correlation patterns within normal samples.

[0065] In local correlation calculations, the dependency relationship between each patch and its neighboring patches is analyzed using a Gaussian kernel function with learnable scale parameters. Local correlation mainly reflects the continuity of surface texture and the consistency of small-scale geometry, thus it is of great significance for detecting minute defects. The Gaussian kernel in this embodiment has adaptive capabilities, which can automatically adjust the aggregation range according to the surface texture complexity of different workpieces, enabling the model to maintain stable performance under multiple scene conditions.

[0066] The global association mechanism employs a multi-head self-attention mechanism to calculate the attention distribution between each patch and all patches. This mechanism can capture macroscopic features such as the overall shape of the workpiece, symmetrical structure, and long-distance texture repetition, helping to identify potential anomalies that disrupt the overall structural consistency. During training, the model forms a stable normal structural pattern by maximizing the local-global association differences of normal samples. Once an abnormal region appears in a real sample, its local or global association will be disrupted, leading to a significant change in the association difference, thereby achieving effective anomaly identification.

[0067] 5. Calculate outlier scores and evaluate model performance;

[0068] After establishing an autocorrelation structural representation, this invention generates a final anomaly score map by combining correlation differences and reconstruction errors, enabling pixel-level localization of anomaly regions. For each patch, the difference between local and global correlations is calculated using statistical distances such as KL divergence. Normal patches, due to their stable local continuity and global structural consistency, typically exhibit larger correlation differences; while anomalous patches, due to disrupted structural relationships, show significantly reduced correlation differences. This phenomenon provides a reliable basis for anomaly detection.

[0069] This invention further employs a feature reconstruction mechanism to calculate reconstruction error, which reflects the reconstruction capability of features deviating from the normal pattern. Normal samples can usually be reconstructed well, while abnormal samples have higher reconstruction errors due to their feature distribution deviating from the normal pattern. Therefore, weighted fusion of correlation differences and reconstruction errors yields a more robust anomaly score map. Finally, the patch-level anomaly score is upsampled to the original image resolution to obtain a pixel-level anomaly heatmap. By setting a threshold, anomaly regions can be accurately segmented, enabling visualized detection of surface anomalies in industrial products. This invention evaluates and visualizes results on the MVTec 3D-AD dataset test set, displaying the anomaly segmentation results in the form of a heatmap, such as... Figure 2 As shown, the present invention can accurately identify abnormal regions in a sample.

[0070] When performing anomaly detection tasks, this invention uses two metrics, I-AUROC and AUPRO, which represent the area under the image-level receiver operating feature curve (evaluating image-level detection performance) and the area under the region overlap curve (evaluating pixel-level segmentation performance), respectively. This invention achieves an I-AUROC score of 81.2 and an AUPRO score of 92.5 on the MVTec 3D-AD dataset, and an I-AUROC score of 49.8 on the MSC-AD dataset, indicating that this invention exhibits excellent accuracy and generalization ability when detecting small, sparse, and irregular surface defects.

[0071] Dataset source:

[0072] [1] Zhao Q, Wang Y, Wang B, et al. MSC-AD: A multiscene unsupervised anomaly detection dataset for small defect detection of casting surface[J]. IEEE Transactions on Industrial Informatics, 2023, 20(4): 6041-6052.

[0073] [2] Bergmann P, Jin

Claims

1. An unsupervised multimodal industrial anomaly detection method based on autocorrelation learning, characterized in that, include: The process involves acquiring RGB images and depth maps of normal samples to construct pseudo-abnormal samples; performing multimodal feature extraction and fusion; introducing an autocorrelation learning module to calculate global correlation, local correlation, and correlation differences; and obtaining pixel-level abnormal regions based on correlation differences to achieve accurate identification of minute defects on the surface of industrial parts. The specific steps are as follows: Step 1: Multimodal data acquisition and preprocessing; Acquire RGB images of the industrial product to be inspected and single-channel depth map data aligned with its space; The depth map is used to characterize the three-dimensional geometric structure information of the workpiece surface, which complements the appearance texture information of the RGB image; Data preprocessing includes: scaling the RGB image and depth map to a preset input size to meet the size requirements of the feature extraction network; performing mean-variance normalization on the RGB image to reduce brightness and color deviations caused by different acquisition environments; linearly scaling the depth map to map its pixel values ​​to the [0, 1] interval to reduce amplitude differences caused by different depth sensors or acquisition conditions; and dividing the processed multimodal image into fixed-size image blocks to provide structured input for subsequent patch-level feature extraction based on Vision Transformer. Step 2: Generation of multimodal pseudo-samples; A pseudo-sample generation module is constructed to address the scarcity of anomalous samples in industrial anomaly detection. This module generates RGB pseudo-samples and depth pseudo-samples by introducing artificially simulated anomalies into real samples containing only normal patterns. Specifically, a binary mask is generated using smooth and random Berlin noise to determine the spatial distribution area of ​​anomalies. This mask can simulate potential defects with multi-scale and irregular shapes. For RGB images, the masked area is filled with a preset random texture to make the area exhibit visual features of structural damage or surface inconsistency, while keeping the original information of the non-masked area unchanged, so as to avoid generating overly simple or unrealistic abnormal patterns. For depth images, anomalous depth values ​​are simulated by random perturbations that satisfy a standard normal distribution, and injected into the mask region by weighted fusion or direct replacement to realistically represent surface depressions, bulges or other geometric anomalies. The generated pseudo-samples are used together with real samples without abnormalities to train the model, so as to enhance the model's ability to distinguish between normal and abnormal structural patterns. Step 3: Multimodal feature fusion; A multimodal feature fusion module is constructed to fully extract complementary key information from the RGB and deep modalities. Specifically, firstly, the real or pseudo-samples obtained in step 2 are input into a pre-trained feature extraction network to obtain RGB feature sequences and deep feature sequences, respectively. Subsequently, a contrastive learning mechanism is introduced to achieve multimodal feature fusion. This includes aligning RGB and deep feature blocks at the same spatial location in the feature space, making them close to each other in the embedding space, while maintaining a large distance from features at other locations or other samples. This contrastive constraint can maximize the shared semantic and structural information across modalities, ensuring that the fused representation has both modal consistency and complementarity. Finally, the two types of modal features optimized by contrastive learning are concatenated or weighted along the channel dimension to form a richer, more robust, and more expressive multimodal fusion feature. Step 4: Autocorrelation learning; An autocorrelation learning module is constructed to calculate and analyze the inter-patch correlations within fused features, in order to establish an accurate model of normal patterns; this module simultaneously calculates both local and global correlations of features: Local correlation computation: A learnable Gaussian kernel function is introduced to compute the local dependencies between each patch and its neighboring patches; the scale parameter of the Gaussian kernel function is learnable, enabling the model to adaptively focus on short-range feature associations that are sensitive to minor anomalies; local correlation focuses on modeling subtle consistency of texture and local structure. Global correlation computation: Introducing a self-attention mechanism to calculate the global dependencies between each patch and all other patches; global correlation computation captures the macroscopic consistency of the overall structure, shape, and long-distance repeating patterns of industrial workpieces; The autocorrelation learning module trains the model through contrastive learning, enabling the local and global correlations of normal samples to maintain specific correlation patterns, thereby achieving an accurate characterization of normal features. This results in an accurate modeling of the distribution of normal features, providing a reliable basis for subsequent anomaly detection. Step 5: Determine correlation differences and calculate anomaly scores; When evaluating model performance and performing anomaly detection, the difference between local and global correlations is used as the core criterion. Anomaly identification and scoring are calculated based on this difference. During model training, the optimization objective is used to make the local and global correlations of normal samples significantly different. Because the local texture relationships of normal regions are highly consistent, and their overall structure maintains stable long-distance correlations from a global perspective, a large correlation difference is formed between the two. In contrast, the local structure of anomaly samples is disrupted, and their correlation relationships within the overall scope are also disturbed, leading to a significant reduction in the difference between local and global correlations. To quantify these correlation differences, KL divergence or its distance metric function is used to compare the local and global correlation distributions, obtaining a patch-level correlation difference map. Furthermore, this correlation difference is fused with the pixel-level reconstruction error obtained from the feature reconstruction module to comprehensively reflect the dual deviation of the anomaly region in both local and global structures. By thresholding the fused score map, the final anomaly score map is generated. Regions with higher scores are identified as having anomalies, thus achieving high-precision, pixel-level localization of surface defects in industrial workpieces.

2. The unsupervised multimodal industrial anomaly detection method according to claim 1, characterized in that, Step 2 describes the use of Berlin noise to generate a binary mask to determine the spatial distribution region of anomalies; For RGB images RGB pseudo-samples are generated by filling with a preset texture image. The process is represented as follows: in, It's a mask. The reversal, It is the opacity parameter. This indicates element-wise multiplication; For single-channel depth maps Anomaly depth values ​​are simulated using random numbers that follow a standard normal distribution to generate pseudo-sample depth maps. The process is represented as follows: ； in, It is generated by combining random numbers with the average depth of the depth map.

3. The unsupervised multimodal industrial anomaly detection method according to claim 2, characterized in that, Step 3, which describes multimodal feature fusion through a contrastive learning mechanism, specifically includes: inputting RGB and deep modal features from the same spatial location into a multilayer perceptron for projection processing, mapping them to query vectors or key vectors; applying the InfoNCE loss function to constrain corresponding features of the two modalities to enhance their matching degree in the feature space, thereby making RGB features and deep features at the same location more consistent in semantic representation; calculating the similarity between vectors using cosine similarity, and using two trainable fully connected layers to map the feature vectors processed by the multilayer perceptron to query vectors or key vectors, i.e. and , Indicates the sample index. Indicates the patch index; loss function Represented as: ； in, For batch size, For non-zero patch count, This is the matrix transpose; finally, the two modal features processed by the multilayer perceptron are connected along the channel dimension to obtain the fused features.

4. The unsupervised multimodal industrial anomaly detection method according to claim 3, characterized in that, Step 4, which involves calculating the local and global associations of features, specifically includes: first, projecting the input features into a query vector, a key vector, a value vector, and a learnable parameter vector for local associations. ; Computing local correlations: Utilizing a learnable Gaussian kernel function to focus on neighborhood information, the first... The patch and the first Local associations between patches are represented as follows: ； in, and For patch location, For the reason The obtained scale parameters; Calculate global associations: Capture long-distance dependencies using a self-attention mechanism, represented as: ； in, For query vector, For key vectors, For matrix transpose, Key vector Dimensions This represents the operation of converting attention scores (similarity) into a probability distribution.

5. The unsupervised multimodal industrial anomaly detection method according to claim 4, characterized in that, Step 5 describes the calculation of anomaly identification and anomaly scores based on the difference between local and global correlations. KL divergence is used to measure the local and global correlation distributions to obtain patch-level correlation difference values, which are then compared with the reconstructed features. and input features The deviations between them are used together to calculate the final outlier score: ； in, This indicates the local correlation of the patch calculated using KL divergence. and global association The model learns to distinguish between normal and abnormal patterns by increasing the correlation differences between normal samples and suppressing the correlation differences between abnormal samples during training. This enables the model to effectively detect anomalies.

Images