A reconstruction-based industrial image defect detection method, system, storage medium and product
By improving the encoder and decoder structure, and combining linear attention mechanism and adaptive feature perturbation, the problems of insufficient encoder extraction capability and decoder identity mapping in the prior art are solved, and more efficient industrial image defect detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN SHUJU INTELLIGENT MFG TECH CO LTD
- Filing Date
- 2026-03-17
- Publication Date
- 2026-06-09
AI Technical Summary
In existing reconstruction-based industrial image defect detection algorithms, the encoder's insufficient extraction capability leads to the inability to effectively extract abnormal features, and the decoder suffers from identity mapping problems, resulting in missed detections and false alarms.
The DINOv3 network is used as the encoder, combined with an improved structure of bottleneck layer and decoder. A linear attention mechanism and adaptive feature perturbation are used. Through feature perturbation and group supervision during the training phase, the reconstruction capability of the decoder is optimized by combining loss functions of cosine distance and structural similarity index, and high-resolution patch set features of reference sample map are introduced.
It improves the ability to extract abnormal features, reduces the false negative rate, suppresses the identity mapping problem of the decoder, reduces the false alarm rate, and improves the accuracy and stability of defect detection.
Smart Images
Figure CN121883482B_ABST
Abstract
Description
Technical Field
[0001] A reconstruction-based industrial image defect detection method, system, storage medium, and product are disclosed, which are used for defect detection in reconstructed industrial images and belong to the field of industrial image processing technology. Background Technology
[0002] Defect detection is a crucial step in industrial production. Intelligent defect detection methods based on image analysis are gradually replacing traditional manual quality inspection methods. However, industrial defect detection faces challenges such as the difficulty in collecting defect samples and the diversity of defect types. Supervised learning methods based on defect samples cannot cover all possible defect patterns. Therefore, an increasing number of unsupervised industrial anomaly detection algorithms have been proposed. These methods are trained using only normal samples and learn normal patterns to identify anomalies, effectively avoiding the problem of scarce defect samples.
[0003] Reconstruction-based industrial anomaly detection algorithms are a common type of unsupervised algorithm. They aim to learn and reconstruct normal features from normal samples during training, and then reconstruct normal features from input anomaly images during the inference phase, calculating the reconstruction feature error to obtain the anomaly region. Currently, common technical problems with reconstruction-based industrial anomaly detection algorithms include:
[0004] 1. The encoder's extraction capability is insufficient, and abnormal features cannot be effectively extracted, leading to missed detections;
[0005] 2. The decoder replicates the encoder identity mapping problem. When an abnormal image is input, the features reconstructed by the decoder are consistent with the features reconstructed by the encoder, leading to missed detections.
[0006] 3. Insufficient decoder reconstruction capability leads to the inability to effectively reconstruct normal features, resulting in false positives, and the inability to repair abnormal features into normal features, resulting in false negatives. Summary of the Invention
[0007] To address the problems mentioned above, the present invention aims to provide a reconstruction-based industrial image defect detection method, system, storage medium, and product. This addresses the following issues in existing technologies: insufficient encoder extraction capability leading to ineffective extraction of abnormal features and resulting in missed detections; decoder copying encoder identity mapping, causing missed detections when an abnormal image is input, as the decoder reconstructs features identical to the encoder features; insufficient decoder reconstruction capability leading to ineffective reconstruction of normal features and resulting in false alarms; and the inability to repair abnormal features into normal features, leading to missed detections.
[0008] To achieve the above objectives, the present invention adopts the following technical solution:
[0009] A reconstruction-based method for industrial image defect detection includes the following steps:
[0010] Step 1: Construct a reference sample library:
[0011] Collect more than 5,000 normal sample images and randomly select one to be added to the reference sample library. Calculate the sum of distances between the remaining normal sample images and all reference sample images in the reference sample library. The one with the largest distance is added to the reference sample library. If the number of reference sample images in the reference sample library meets the given requirement, the final reference sample library is constructed. Otherwise, calculate the sum of distances between the remaining normal sample images and all sample images in the reference sample library to obtain the reference sample images.
[0012] Step 2: Construct a reconstruction network and train it using normal sample images not in the reference sample library and reference sample images in the reference sample library to obtain a trained reconstruction network.
[0013] The reconstruction network includes an encoder that extracts coded features from the input image, a bottleneck layer that aggregates the coded features extracted by the encoder, and a decoder that decodes the coded features aggregated by the bottleneck layer and the patch set features of the matched reference sample image to obtain the reconstructed features.
[0014] Step 3: Abnormal Defect Location:
[0015] Based on the industrial image to be detected, match the reference sample image with the minimum Euclidean distance from the reference sample library;
[0016] Simultaneously, the industrial image to be detected and the corresponding reference sample image are input into the trained reconstruction network to obtain reconstruction features;
[0017] The distance between the encoded features and the reconstructed features is calculated to generate an anomaly score map, and then the defect area is located by threshold segmentation and contour search.
[0018] Furthermore, in step 1, the specific steps for calculating the sum of distances between the remaining normal sample images and all reference sample images in the reference sample library are as follows:
[0019] Adjust all remaining normal sample images and all reference sample images in the reference sample library to a fixed size (224, 224).
[0020] Then, the feature vectors of all normal samples are extracted using the convolutional neural network ResNet18, and the spatial dimensions are compressed through a global average pooling layer to obtain the 512-dimensional image features of each image output.
[0021] Finally, the Euclidean distances between the image features of any normal sample image not in the reference sample library and the image features of each reference sample image are calculated and summed to obtain the distance sum. The formula is:
[0022]
[0023] in, The first image feature not in the reference sample library is the normal sample image. dimension, The first image feature of the reference sample image dimension.
[0024] Furthermore, the encoder uses the DINOv3 network to extract features from layers 3 to 10 of the input image as encoding features at the corresponding scale. The input image serves as a training sample during the training phase, a test sample during the testing phase, and an industrial image to be detected during the detection phase.
[0025] The bottleneck layer consists of two MLP structures connected in sequence. The MLP structure consists of a first linear layer, a GELU activation function layer, a first dropout layer, a second linear layer, and a second dropout layer connected in sequence.
[0026] The decoder stacks 8 decoding layers, each outputting 8 layers of decoding features. Each decoding layer replaces the softmax attention mechanism with a linear attention mechanism. The input to the first decoding layer is the encoded feature fused from the bottleneck layer and the patch set features of the reference sample image with the same dimension as the encoded feature. The input to the 2nd to 8th decoding layers is the decoded feature fused from the previous decoding layer and the patch set features of the reference sample image with the same dimension as the decoded feature. The patch set features refer to the reference sample image with the smallest Euclidean distance from the reference sample library based on the input image, and the matched reference sample image is cropped into a patch set with the same size as the current encoded feature or decoded feature, and then the feature dimension is aggregated by a linear layer.
[0027] Furthermore, the input to each decoding layer is:
[0028]
[0029] in, This refers to the encoded features output by the bottleneck layer or the decoded features output by the previous decoding layer. To control the reference strength using learnable gating parameters, each layer of the decoder uses independent gating parameters. The gating parameter is a learnable scalar that is mapped by the Sigmoid function to obtain a weighted coefficient in the interval [0,1], which is used to adjust the reference strength of the reference feature in the current layer. Indicates and The patch set features of the reference sample image corresponding to the feature dimension. This indicates the result after fusion.
[0030] Furthermore, the formula for the linear attention mechanism is:
[0031]
[0032]
[0033] Among them, the feature mapping function for ,parameter These are the hyperparameters of the feature mapping function. These are the input variables of the feature mapping function, in the linear attention formula. The value is and , , , These represent query, key, and value characteristics, respectively.
[0034] Furthermore, during the training of the reconstructed network, the encoder parameters are frozen, and only the bottleneck layer and decoder are optimized, specifically:
[0035] A feature perturbation layer is introduced before the bottleneck layer to apply spatial adaptive Gaussian noise to the encoded features. The amplitude of the spatial adaptive Gaussian noise is dynamically adjusted according to the L2 norm of the encoded features. The larger the L2 norm, the stronger the response, and the larger the perturbation in the region, and the weaker the response, the smaller the perturbation in the region. For loss calculation, the 8 layers of encoded features obtained by the encoder and the 8 layers of decoder are divided into two groups: the first 4 layers and the last 4 layers. After aggregating the features of each group, a combined loss function is calculated. This loss function combines cosine distance and structural similarity index, and the two losses are fused by weighted summation. Cosine distance measures directional similarity, while structural similarity index considers brightness, contrast, and structural information.
[0036] Bottleneck layer and decoder optimizations are performed using the StableAdamW optimizer.
[0037] Furthermore, the specific steps of step 3 are as follows:
[0038] Step 3.1: Match the reference sample image with the smallest Euclidean distance from the reference sample library based on the industrial image to be detected;
[0039] Step 3.2: Input the industrial image to be detected into the trained reconstruction network, extract the coding features of layers 3-10 through the encoder of the trained reconstruction network, and then input them into the decoder after aggregation through the bottleneck layer.
[0040] Step 3.3: The first layer of the decoder fuses the encoded features output by the bottleneck layer with the patch set features of the reference sample map with the same dimension as the encoded features to obtain the decoded features. The input of the decoding layers 2-8 fuses the decoded features output by the previous decoding layer with the patch set features of the reference sample map with the same dimension as the decoded features to finally obtain the reconstructed features, that is, the multi-layer decoding features.
[0041] Step 3.4: Calculate the cosine distance between the coding features of each layer and the corresponding decoding features of the layer, and then perform the aggregation operation to obtain the corresponding cosine distance score map, generate a multi-scale anomaly score map, and obtain an anomaly score map of the same size as the industrial image to be detected by upsampling.
[0042] Step 3.5: Normalize the cosine distance score of the anomaly score map, and then linearly map it to the pixel value range of [0, 255] to obtain the visualized anomaly score map, where the value range of the cosine distance score of the anomaly score map is [-1, 1].
[0043] Step 3.6: Binarize the visualized anomaly score map. After binarization, mark the regions with cosine distance scores greater than a given preset threshold as generating a defect mask map; otherwise, mark them as normal. After marking, generate the defect mask map.
[0044] Step 3.7: Use a contour finding algorithm to extract the boundary contour of the defect region in the defect mask image, thereby achieving precise defect localization. The contour finding algorithm includes OpenCV's findContours.
[0045] A reconstruction-based industrial image defect detection system includes a memory, a processor, and a computer program stored in the memory, characterized in that: the processor executes the computer program to implement the steps of the reconstruction-based industrial image defect detection method.
[0046] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method.
[0047] A computer program product includes a computer program that, when executed by a processor, implements the steps of the reconstruction-based industrial image defect detection method.
[0048] Compared with the prior art, the beneficial effects of this invention are as follows:
[0049] I. The encoder (DINOv3) in this invention introduces strong representation capabilities: it can stably extract semantics and fine-grained textures in the global context, and can simultaneously depict normal and abnormal patterns, reducing missed detections caused by insufficient feature extraction by the encoder.
[0050] Second, this invention can suppress the "identity mapping" problem of the decoder, improve the anomaly reconstruction capability, and thus improve the detection rate: adaptive feature perturbation is added during the training phase, Dropout is introduced in the bottleneck layer, and features are grouped for supervision; the loss adopts "cosine distance (emphasizing global directional consistency) + SSIM (emphasizing local structure and contrast)", without focusing on the layer-by-layer and pixel-by-pixel copying capability, avoiding smooth reconstruction while achieving stable convergence.
[0051] Third, the present invention reduces false alarms by matching reference sample images: matching the most similar normal sample as an external reference for each image to be tested, suppressing hardware environment interference such as lighting, camera, production line fluctuations, etc., and reducing false alarms caused by domain offset.
[0052] Fourth, the present invention can reconstruct under the guidance of a reference sample image: the decoder introduces a high-resolution local patch reference (i.e., patch set features) of the reference sample image to supplement the texture and edges of the normal sample image, promote the reconstruction of normal regions of the normal sample image into normal regions, and help to recover missing and abnormal regions, thereby improving the overall reconstruction quality of the decoder, thereby improving the identification of anomalies and reducing false alarms in normal regions. Attached Figure Description
[0053] Figure 1 This is a schematic diagram of the reconstruction network structure in this invention; in the figure: the upper dashed box is the encoder as a whole, and only the features from the 3rd to the 10th layers are taken as the corresponding 8-layer encoder features; the bottleneck layer MLP is used to aggregate the 8-layer encoded features; the three dashed boxes below are the decoder part, from top to bottom: stacked decoding layers, fusion layer to obtain the input of each corresponding decoding layer, and linear layer with patch set features of the matched reference sample map.
[0054] Figure 2 This is a schematic diagram of the MLP structure in the bottleneck layer of this invention. Detailed Implementation
[0055] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.
[0056] A reconstruction-based method for industrial image defect detection includes the following steps:
[0057] Step 1: Construct a reference sample library:
[0058] In actual production, differences in raw materials and manufacturing processes can lead to variations in workpieces from the same or different batches. Furthermore, environmental noise introduced by the dynamic production line and camera noise during the imaging process of intelligent defect detection equipment can cause inconsistencies in the images of normal workpieces submitted for inspection. To improve system robustness and reduce maintenance costs, a large number of normal workpiece sample images are first collected. Then, based on these samples, a subset of normal samples with significant differences is selected as an external reference sample library. The selection method is as follows: First, one normal sample image is randomly selected and added to the reference sample library. Then, the sum of the distances between the remaining normal sample images and all sample images in the reference sample library is calculated. The image with the largest distance is added to the reference sample library, and this process is repeated until the required number of reference sample images is met. Specifically:
[0059] Collect more than 5,000 normal sample images and randomly select one to be added to the reference sample library. Calculate the sum of distances between the remaining normal sample images and all reference sample images in the reference sample library. The one with the largest distance is added to the reference sample library. If the number of reference sample images in the reference sample library meets the given requirement of 500, the final reference sample library is constructed. Otherwise, calculate the sum of distances between the remaining normal sample images and all sample images in the reference sample library to obtain the reference sample images.
[0060] The specific steps for calculating the sum of distances between the remaining normal sample images and all reference sample images in the reference sample library are as follows:
[0061] Adjust all remaining normal sample images and all reference sample images in the reference sample library to a fixed size (224, 224).
[0062] Then, the feature vectors of all normal samples are extracted using the convolutional neural network ResNet18, and the spatial dimensions are compressed through a global average pooling layer to obtain the 512-dimensional image features of each image output.
[0063] Finally, the Euclidean distances between the image features of any normal sample image not in the reference sample library and the image features of each reference sample image are calculated and summed to obtain the distance sum. The formula is:
[0064]
[0065] in, The first image feature not in the reference sample library is the normal sample image. dimension, The first image feature of the reference sample image Dimension. Euclidean distance. There is a distance for each remaining normal sample image and each reference sample image, and the sum of the distances is the Euclidean distance between each remaining normal sample image and all reference sample images. The result is obtained by adding them together.
[0066] Step 2: Construct a reconstruction network and train it using normal sample images not in the reference sample library and reference sample images in the reference sample library to obtain a trained reconstruction network.
[0067] The reconstruction network includes an encoder that extracts coded features from the input image, a bottleneck layer that aggregates the coded features extracted by the encoder, and a decoder that decodes the coded features aggregated by the bottleneck layer and the patch set features of the matched reference sample image to obtain the reconstructed features.
[0068] The encoder uses the DINOv3 network to extract features from layers 3 to 10 of the input image as the corresponding scale encoding features. The input image is the training sample in the training phase, the test sample in the testing phase, and the industrial image to be detected in the detection phase. The DINOv3 network is trained on unlabeled large-scale data using a teacher-student knowledge distillation framework. The core advantages of introducing this model as the encoder are: (1) It can effectively model the long-distance dependency between any position in the image through the multi-head self-attention mechanism, thus possessing a strong ability to extract and understand global contextual information; (2) Through regularization technology (REG) and Gram matrix anchoring mechanism, it effectively alleviates the feature degradation problem that is common in Vision Transformer in dense prediction tasks, so that the model can maintain global semantic understanding while also possessing excellent local dense feature extraction capabilities, which is suitable for reconstruction tasks that require simultaneous attention to global semantics and local details.
[0069] The bottleneck layer consists of two MLP structures connected in sequence. The MLP structure consists of a first linear layer, a GELU activation function layer, a first dropout layer, a second linear layer, and a second dropout layer connected in sequence. Figure 1 The MLP in the code is the bottleneck layer, which includes two MLP structures, each corresponding to a specific layer. Figure 2 It consists of two fully connected layers and a non-linear activation function, with Dropout layers added after the activation function and at the final output layer. The first fully connected layer expands the feature dimension to a high-dimensional space, with an increase factor typically set to 4, and enhances feature expressiveness through the non-linear activation function. The second fully connected layer projects the features back to the low-dimensional space, controlling the number of parameters and computational complexity. The Dropout rate is typically set to 0.1–0.3, randomly discarding some activation values to prevent overfitting and suppress the decoder from directly copying the encoder's features. Specifically, it consists of a linear layer (Linear) - an activation layer (GELU) - a Dropout layer (Dropout) - a linear layer (Linear) - a Dropout layer (Dropout).
[0070] The decoder stacks eight decoding layers, each outputting eight layers of decoding features. Each decoding layer replaces the softmax attention mechanism with a linear attention mechanism. The input to the first decoding layer is the encoded feature fused from the bottleneck layer and a patch set feature of a reference sample image with the same dimension as the encoded feature. The inputs to the second through eighth decoding layers are the decoded features fused from the previous layer and a patch set feature of a reference sample image with the same dimension as the decoded feature. Here, the patch set feature refers to the reference sample image that matches the minimum distance from the reference sample library based on the input image, and the matched reference sample image is cropped into a patch set with the same size as the current encoded or decoded feature (input image size is...). Based on the consistent size of the encoded feature map extracted from DINOv3, and with a downsampling factor of 16, the cropping window is... (Strictly aligned with the decoded feature map) and then aggregated through a linear layer to obtain the feature dimensions.
[0071] The input to each decoding layer is:
[0072]
[0073] in, This refers to the encoded features output by the bottleneck layer or the decoded features output by the previous decoding layer. To control the reference strength using learnable gating parameters, each layer of the decoder uses independent gating parameters. The gating parameter is a learnable scalar that is mapped by the Sigmoid function to obtain a weighted coefficient in the interval [0,1], which is used to adjust the reference strength of the reference feature in the current layer. Indicates and The patch set features of the reference sample image corresponding to the feature dimension. This represents the result after fusion. Compared to the softmax attention mechanism, the computational complexity of the linear attention mechanism is reduced from O(L²d) to O(Ld²), significantly improving inference efficiency. L is the sequence length, which is the number of image blocks, and d is the feature dimension.
[0074] The formula for the linear attention mechanism is:
[0075]
[0076]
[0077] Among them, the feature mapping function for ,parameter These are the hyperparameters of the feature mapping function. These are the input variables of the feature mapping function, in the linear attention formula. The value is and , , , These represent query, key, and value characteristics, respectively.
[0078] During the training of the reconstructed network, the encoder parameters are frozen, and only the bottleneck layer and decoder are optimized, specifically:
[0079] A feature jitter layer is introduced before the bottleneck layer to apply spatially adaptive Gaussian noise to the encoded features. The amplitude of the spatially adaptive Gaussian noise is dynamically adjusted according to the L2 norm of the encoded features. Regions with larger L2 norms and stronger responses experience larger jitters, while regions with weaker responses experience smaller jitters. For loss calculation, the features obtained from the 8 layers of encoded features from the encoder and the 8 layers of decoder are divided into two groups: the first 4 layers and the last 4 layers. After aggregating the features of each group, a combined loss function is calculated. This loss function combines cosine distance and the Structure Similarity Index Measure (SSIM), fusing the two losses through a weighted summation. The cosine distance measures directional similarity, while the structure similarity index considers brightness, contrast, and structural information. Grouping the loss calculation can effectively avoid the decoder copying the encoded features layer by layer and region by region, suppressing identity mapping, and losing cosine distance and SSIM instead of MSE loss, thus avoiding smooth reconstruction.
[0080] The bottleneck layer and decoder optimization adopt the StableAdamW optimizer. Based on AdamW's adaptive learning rate and decoupled weight decay, a more stable gradient update strategy is introduced to further improve the stability and convergence effect of the training process.
[0081] When constructing the reference sample library, the sum of the Euclidean distances between each remaining normal sample image and all reference sample images is calculated to evaluate whether the image with the greatest difference between the remaining normal sample image and the reference sample image already in the sample library should be added to the sample library.
[0082] However, in the actual training or testing process, we only need to calculate the Euclidean distance between the image to be tested and all reference sample images, compare which one has the smallest Euclidean distance, and use the one with the smallest distance as the reference sample image for the current test image.
[0083] Step 3: Abnormal Defect Location:
[0084] Based on the industrial image to be detected, match the reference sample image with the minimum Euclidean distance from the reference sample library;
[0085] Simultaneously, the industrial image to be detected and the corresponding reference sample image are input into the trained reconstruction network to obtain reconstruction features;
[0086] The distance between the encoded features and the reconstructed features is calculated to generate an anomaly score map, and then the defect area is located by threshold segmentation and contour search.
[0087] The specific steps are as follows:
[0088] Step 3.1: Match the reference sample image with the smallest Euclidean distance from the reference sample library based on the industrial image to be detected;
[0089] Step 3.2: Input the industrial image to be detected into the trained reconstruction network, extract the coding features of layers 3-10 through the encoder of the trained reconstruction network, and then input them into the decoder after aggregation through the bottleneck layer.
[0090] Step 3.3: The first layer of the decoder fuses the encoded features output by the bottleneck layer with the patch set features of the reference sample map with the same dimension as the encoded features to obtain the decoded features. The input of the decoding layers 2-8 fuses the decoded features output by the previous decoding layer with the patch set features of the reference sample map with the same dimension as the decoded features to finally obtain the reconstructed features, that is, the multi-layer decoding features.
[0091] Step 3.4: Calculate the cosine distance between the coding features of each layer and the corresponding decoding features of the layer, and then perform the aggregation operation (i.e., average the multiple cosine distance scores obtained by the coding features of each layer and the corresponding decoding features of the layer) to obtain the corresponding cosine distance score map, generate a multi-scale anomaly score map, and obtain an anomaly score map of the same size as the industrial image to be detected by upsampling.
[0092] Step 3.5: Normalize the cosine distance score of the anomaly score map, and then linearly map it to the pixel value range of [0, 255] to obtain the visualized anomaly score map, where the value range of the cosine distance score of the anomaly score map is [-1, 1].
[0093] Step 3.6: Binarize the visualized anomaly score map. After binarization, mark the regions with cosine distance scores greater than a given preset threshold as generating a defect mask map; otherwise, mark them as normal. After marking, generate the defect mask map.
[0094] Step 3.7: Use a contour finding algorithm to extract the boundary contour of the defect region in the defect mask image, thereby achieving precise defect localization. The contour finding algorithm includes OpenCV's findContours.
[0095] Example
[0096] like Figure 1As shown, the input industrial image to be detected first undergoes multi-layer encoder feature extraction via an encoder, followed by a bottleneck layer to obtain aggregated features. Note that there is no feature jitter layer during the testing phase. Simultaneously, the image to be detected is matched with the closest image from a reference sample library as a reference sample image. The decoder simultaneously receives the aggregated encoded features and the patch set features of the reference sample image, performing layer-by-layer decoding to obtain multi-layer decoded features. Then, the encoded and decoded features are grouped and cosine distances are calculated. For each group, an aggregation operation (i.e., averaging) is performed to obtain the corresponding cosine distance score image. Finally, post-processing such as binarization and contour finding is performed to identify anomalous regions.
[0097] The above are merely representative embodiments among the many specific applications of this invention, and do not constitute any limitation on the scope of protection of this invention. All technical solutions formed by transformation or equivalent substitution fall within the scope of protection of this invention.
Claims
1. A reconstruction-based method for detecting defects in industrial images, characterized in that, Includes the following steps: Step 1: Construct a reference sample library: Collect more than 5,000 normal sample images and randomly select one to be added to the reference sample library. Calculate the sum of distances between the remaining normal sample images and all reference sample images in the reference sample library. The one with the largest distance is added to the reference sample library. If the number of reference sample images in the reference sample library meets the given requirement, the final reference sample library is constructed. Otherwise, calculate the sum of distances between the remaining normal sample images and all sample images in the reference sample library to obtain the reference sample images. Step 2: Construct a reconstruction network and train it using normal sample images not in the reference sample library and reference sample images in the reference sample library to obtain a trained reconstruction network. The reconstruction network includes an encoder that extracts coded features from the input image, a bottleneck layer that aggregates the coded features extracted by the encoder, and a decoder that decodes the coded features aggregated by the bottleneck layer and the patch set features of the matched reference sample image to obtain the reconstructed features. Step 3: Abnormal Defect Location: Based on the industrial image to be detected, match the reference sample image with the minimum Euclidean distance from the reference sample library; Simultaneously, the industrial image to be detected and the corresponding reference sample image are input into the trained reconstruction network to obtain reconstruction features; The distance between the encoded features and the reconstructed features is calculated to generate an anomaly score map, and then the defect area is located by threshold segmentation and contour search. The encoder uses the DINOv3 network to extract features from layers 3 to 10 of the input image as encoding features at the corresponding scale. The input image is used as a training sample during the training phase, a test sample during the testing phase, and an industrial image to be detected during the detection phase. The bottleneck layer consists of two MLP structures connected in sequence. The MLP structure consists of a first linear layer, a GELU activation function layer, a first dropout layer, a second linear layer, and a second dropout layer connected in sequence. The decoder stacks 8 decoding layers, each outputting 8 layers of decoding features. Each decoding layer replaces the softmax attention mechanism with a linear attention mechanism. The input to the first decoding layer is the encoded feature fused from the bottleneck layer and the patch set features of the reference sample image with the same dimension as the encoded feature. The input to the 2nd to 8th decoding layers is the decoded feature fused from the previous decoding layer and the patch set features of the reference sample image with the same dimension as the decoded feature. The patch set features refer to the reference sample image with the smallest Euclidean distance from the reference sample library based on the input image, and the matched reference sample image is cropped into a patch set with the same size as the current encoded feature or decoded feature, and then the feature dimension is aggregated by a linear layer. During the training of the reconstructed network, the encoder parameters are frozen, and only the bottleneck layer and decoder are optimized, specifically: A feature perturbation layer is introduced before the bottleneck layer to apply spatial adaptive Gaussian noise to the encoded features. The amplitude of the spatial adaptive Gaussian noise is dynamically adjusted according to the L2 norm of the encoded features. The larger the L2 norm, the stronger the response, and the larger the perturbation in the region, and the weaker the response, the smaller the perturbation in the region. For loss calculation, the 8 layers of encoded features obtained by the encoder and the 8 layers of decoder are divided into two groups: the first 4 layers and the last 4 layers. After aggregating the features of each group, a combined loss function is calculated. This loss function combines cosine distance and structural similarity index, and the two losses are fused by weighted summation. Cosine distance measures directional similarity, while structural similarity index considers brightness, contrast, and structural information. Bottleneck layer and decoder optimizations are performed using the StableAdamW optimizer.
2. The industrial image defect detection method based on reconstruction according to claim 1, characterized in that, In step 1, the specific steps for calculating the sum of distances between the remaining normal sample images and all reference sample images in the reference sample library are as follows: Adjust all remaining normal sample images and all reference sample images in the reference sample library to a fixed size (224, 224). Then, the feature vectors of all normal samples are extracted using the convolutional neural network ResNet18, and the spatial dimensions are compressed through a global average pooling layer to obtain the 512-dimensional image features of each output. Finally, the Euclidean distances between the image features of any normal sample image not in the reference sample library and the image features of each reference sample image are calculated and summed to obtain the distance sum. The formula is: in, The first image feature not in the reference sample library is the normal sample image. dimension, The first image feature of the reference sample image dimension.
3. The industrial image defect detection method based on reconstruction according to claim 2, characterized in that, The input to each decoding layer is: in, This refers to the encoded features output by the bottleneck layer or the decoded features output by the previous decoding layer. To control the reference strength using learnable gating parameters, each layer of the decoder uses independent gating parameters. The gating parameter is a learnable scalar that is mapped by the Sigmoid function to obtain a weighted coefficient in the interval [0,1], which is used to adjust the reference strength of the reference feature in the current layer. Indicates and The patch set features of the reference sample image corresponding to the feature dimension. This indicates the result after fusion.
4. The industrial defect detection method based on reconstruction according to claim 3, characterized in that, The formula for the linear attention mechanism is: Among them, the feature mapping function for ,parameter These are the hyperparameters of the feature mapping function. These are the input variables of the feature mapping function, in the linear attention formula. The value is and , , , These represent query, key, and value characteristics, respectively.
5. The industrial image defect detection method based on reconstruction according to claim 4, characterized in that, The specific steps of step 3 are as follows: Step 3.1: Match the reference sample image with the smallest Euclidean distance from the reference sample library based on the industrial image to be detected; Step 3.2: Input the industrial image to be detected into the trained reconstruction network, extract the coding features of layers 3-10 through the encoder of the trained reconstruction network, and then input them into the decoder after aggregation through the bottleneck layer. Step 3.3: The first layer of the decoder fuses the encoded features output by the bottleneck layer with the patch set features of the reference sample map with the same dimension as the encoded features to obtain the decoded features. The input of the decoding layers 2-8 fuses the decoded features output by the previous decoding layer with the patch set features of the reference sample map with the same dimension as the decoded features to finally obtain the reconstructed features, that is, the multi-layer decoding features. Step 3.4: Calculate the cosine distance between the coding features of each layer and the corresponding decoding features of the layer, and then perform the aggregation operation to obtain the corresponding cosine distance score map, generate a multi-scale anomaly score map, and obtain an anomaly score map of the same size as the industrial image to be detected by upsampling. Step 3.5: Normalize the cosine distance score of the anomaly score map, and then linearly map it to the pixel value range of [0, 255] to obtain the visualized anomaly score map, where the value range of the cosine distance score of the anomaly score map is [-1, 1]. Step 3.6: Binarize the visualized anomaly score map. After binarization, mark the regions with cosine distance scores greater than a given preset threshold as generating a defect mask map; otherwise, mark them as normal. After marking, generate the defect mask map. Step 3.7: Use a contour finding algorithm to extract the boundary contour of the defect region in the defect mask image, thereby achieving precise defect localization. The contour finding algorithm includes OpenCV's findContours.
6. A reconstruction-based industrial image defect detection system, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1-5.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-5.
8. A computer program product, comprising a computer program, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-5.