A self-supervised defect detection method based on heatmap pseudo-defects

Abstract

This invention relates to the field of product defect detection and discloses a self-supervised defect detection method based on heatmap-based pseudo-defects. The pseudo-defect image construction steps include: extracting features from a normal source image to generate a heatmap, locating anchor points and extracting defect patches, and then fusing the patches into the normal target image to construct an labeled pseudo-defect image; a self-supervised model training step: training an encoder-decoder model using pseudo-defects and normal images, and learning the feature reconstruction pattern of normal samples by minimizing the difference between the encoder and decoder reconstructed features; and a defect detection step: using the trained model, detecting defects by calculating the similarity score between the encoder features and decoder reconstructed features of the image to be tested, with regions having low similarity scores being identified as defects. This invention constructs realistic pseudo-defects using heatmaps and combines multi-scale feature fusion and attention mechanisms to improve the model's detection accuracy and generalization ability for diverse defects.

Description

Technical Field

[0001] This invention relates to the field of product defect detection, specifically a self-supervised defect detection method based on heatmap pseudo-defects. Background Technology

[0002] In modern industrial manufacturing, automated visual inspection of product surface defects is a crucial step in ensuring product quality and improving production efficiency. With the development of machine vision and deep learning technologies, using automated equipment to replace manual labor for quality inspection has become a mainstream trend.

[0003] In existing technologies, deep learning-based defect detection methods are mainly divided into two categories: supervised learning and unsupervised / self-supervised learning. Supervised learning methods train deep neural networks on a large number of precisely labeled defect samples, achieving high detection accuracy. The other category, self-supervised learning methods, aims to alleviate the difficulty of obtaining large-scale defect samples in industrial scenarios. These methods typically use only readily available normal samples for training. Their core idea is to allow the model to fully learn the feature distribution of normal samples and identify inputs that significantly deviate from this distribution as defects. One specific self-supervised technique involves artificially constructing pseudo-defects on normal images and then training the model to identify or reconstruct these pseudo-defects, thereby learning the differences between normal and abnormal images.

[0004] While existing technologies have addressed the reliance on labeled defect data to some extent by constructing pseudo-defects, several shortcomings remain in practical applications. Existing pseudo-defect construction methods exhibit significant randomness in location selection and morphological generation; for example, they may generate defects by pasting random noise or arbitrary tiles from other images. This approach ignores the inherent structure and texture features of the object being detected, leading to pseudo-defects that may differ significantly from real defects in both location and appearance. For instance, a pseudo-defect might appear in a suspended background area where a physical defect should not exist. When models are trained on such data, they are prone to learning "shortcut" features such as unnatural boundaries or texture differences between pseudo-defects and the background, rather than the intrinsic patterns of real defects. This limits the model's ability to generalize and recognize real, diverse defects, thus impacting detection performance.

[0005] Furthermore, at the model learning level, some existing self-supervised models fail to effectively distinguish and utilize the importance of features at different scales when processing features extracted by the encoder. Industrial defects vary in shape and size; minor scratches are mainly reflected in high-resolution low-level features, while large areas of dirt are more reflected in low-resolution high-level semantic features. If the model fails to establish an effective mechanism to fuse multi-scale feature information and focus on key features that are more decisive for the reconstruction task, its feature representation of "normal" samples will be insufficiently comprehensive and robust. This reconstruction method, which does not fully integrate and filter features, weakens the model's ability to distinguish complex normal textures from minor anomalies, thereby limiting the accuracy of the final defect detection and the precision of the localization. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a self-supervised defect detection method based on heatmap pseudo-defects, which solves the problems of weak model generalization ability and low detection accuracy caused by unreasonable pseudo-defect construction and insufficient feature learning in existing self-supervised defect detection methods.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A self-supervised defect detection method based on heatmap pseudo-defects is proposed. This method first generates pseudo-defect images with accurate annotations and reasonable shapes by using normal source images and target images through a pseudo-defect construction process. Then, the generated pseudo-defect images and normal images are used to jointly train a self-supervised model containing an encoder, decoder and multi-scale feature fusion network; Finally, the trained model is used to detect and locate defects in new test images.

[0008] The specific technical solution of the present invention is achieved through the following steps: The pseudo-defect construction step aims to construct pseudo-defect images from defect-free images for subsequent model training. This step specifically includes: Feature heatmap generation: First, a normal source image is provided. A pre-trained convolutional neural network (e.g., a ResNet50 network) is used as a feature extractor to perform forward propagation on the source image to extract its deep features, obtaining a feature distribution. To quantify the saliency of different regions in an image, it is necessary to calculate the feature distribution. The L2 norm of the feature vector at each spatial location is calculated, and all L2 norm values ​​are normalized according to the following formula to generate the final feature heatmap: ; In the formula, These are the normalized feature vectors; The L2 norm of the current feature vector; The smallest L2 norm eigenvalue; It is the eigenvalue of the largest L2 norm.

[0009] Anchor point positioning and candidate box size determination: On the characteristic heatmap generated in step 1, for all thermal values The points are sorted, and the image spatial location corresponding to the point with the highest heat value is selected as the anchor point for subsequent extraction of candidate boxes.

[0010] To determine the size of the candidate bounding box, first consider the overall height of the source image. and width Using the Gamma function Calculate the base height of the candidate box and base width : ; In the formula, This is the base height of the candidate box; This is the base width of the candidate box; It is the Gamma function; The overall height of the source image; The width of the source image is given.

[0011] To increase the diversity of defect morphology, a scaling factor is randomly sampled from a standard normal distribution with a mean of 1 and a variance of 0.5. And utilize this scaling factor Adjust the base dimensions to obtain the final candidate box height. and width : ; In the formula, This is the final candidate box height; This is the final candidate box width; This is the scaling factor for random sampling from the standard normal distribution; The base height; The base width is denoted as .

[0012] Fusion and annotation: Based on the anchor points determined in step 2 and the final candidate box height and width The corresponding image region is extracted from the source image as a candidate bounding box region. A normal target image is then taken, and an anchor point for the fusion region is determined on the target image using the same method as in steps 1 and 2. The candidate bounding box region extracted from the source image is then fused to the anchor point region of the target image, thereby generating a pseudo-defect image.

[0013] Finally, the pseudo-defect image is double-labeled: first, the implanted candidate box region is segmented and labeled at the pixel level to generate its contour positioning information; second, the entire pseudo-defect image is classified and labeled as a defective category.

[0014] Self-Supervised Model Construction and Detection Steps: This step aims to build, train, and use a self-supervised model to achieve defect detection. Specifically, this step includes: Model framework and data processing: A self-supervised model is constructed, consisting of an encoder, a decoder, and a multi-scale feature fusion network. The encoder and decoder employ similar network structures, such as both being composed of ResNet34 modules. Before training, all images (including normal and pseudo-defect images) are preprocessed to a uniform 256×256 pixel size, and data augmentation is performed using methods such as random rotation and flipping. Finally, the images are randomly cropped to 224×224 pixels as the model input.

[0015] Multi-scale feature fusion: To fuse feature information from different levels, a multi-scale feature fusion network was designed. In the encoder, for the first... Hierarchical feature maps Connect it with the next higher level (the first) Feature map of layer (i≥2) Perform a fusion operation to generate an interactive feature. Before fusion, 1×1 convolutions are used to align the number of channels of features at different levels.

[0016] Subsequently, an attention mechanism is used to apply the interactive features to the current level. Here, the interactive features... As a query for the attention mechanism, the original features of the current layer Simultaneously serving as both key and value, the enhanced features are calculated. : ; In the formula, The enhanced features; The interaction feature serves as the query in the attention mechanism; For the first The hierarchical feature map serves as both the key and value in the attention mechanism. For the first The dimension of layer features; It is a normalized exponential function; This is the transpose operation of a matrix.

[0017] The processed features from all levels are integrated to obtain the final multi-scale fused features, which are then input into the decoder.

[0018] Model training: The goal of model training is to enable the decoder to accurately reconstruct the normal features extracted by the encoder. During training, the encoder is calculated in the first... Feature maps extracted hierarchically Reconstructed feature maps recovered by the decoder at the corresponding level Cosine similarity between To quantify reconstruction quality: ; In the formula: For the first Hierarchical similarity score; For the encoder in the first Feature maps extracted hierarchically; For the decoder in the first Reconstructed feature map of hierarchical restoration; This is the transpose operation for a matrix; Let L2 be the L2 norm of the vector.

[0019] Based on this similarity score, define the model's total loss function. : ; In the formula, This is the total loss function of the model; The total number of layers for feature comparison; For the first Hierarchical similarity score; For the encoder in the first Feature maps extracted hierarchically; For the decoder in the first Reconstructed feature map of hierarchical restoration; This is the transpose operation for a matrix; Let L2 be the norm of the vector.

[0020] Defect detection: A test image is input into the trained self-supervised model. The model performs one forward propagation and calculates the similarity scores between feature maps at each level. The similarity scores at each level are then merged into an anomaly score map with the same size as the test image.

[0021] On this anomaly score map, because the model cannot effectively reconstruct the features of unseen defective regions, the similarity scores corresponding to defective regions will be significantly lower than those of normal regions. Therefore, by setting a preset threshold, regions with scores below the threshold can be identified and located as defective regions.

[0022] This invention provides a self-supervised defect detection method based on heatmap pseudo-defects. It has the following beneficial effects: 1. In the pseudo-defect construction step of this invention, regions with significant structure or texture in the image are located as anchor points by generating feature heatmaps. Candidate box regions extracted from these anchor points are used as pseudo-defects, which visually better match the inherent characteristics of objects, making the constructed pseudo-defect images closer to defects in real-world scenes. Using these high-quality, morphologically reasonable pseudo-defect samples to train the self-supervised model enables the model to learn more robust and representative features of normal samples, thereby improving the model's generalization ability in detecting real and diverse defects.

[0023] 2. The self-supervised model of this invention incorporates a multi-scale feature fusion network, which fuses feature maps extracted from different levels of the encoder. Specifically, it interacts with upper-layer feature maps containing high-level semantic information and current-layer feature maps containing low-level texture and detail information. This design enables the model to utilize contextual information at different scales simultaneously when reconstructing features, forming a more comprehensive and three-dimensional feature representation of normal samples, thereby more effectively learning the inherent distribution patterns of normal images.

[0024] 3. Building upon multi-scale feature fusion, this invention further introduces an attention mechanism. This mechanism processes the fused features, enabling them to automatically learn and focus on the feature regions most important for the reconstruction task, while suppressing interference from irrelevant or redundant information. This allows the model to more accurately grasp the key features of normal samples during training. Therefore, in the defect detection step, when the model encounters defect regions that cannot be accurately reconstructed, its corresponding similarity score will decrease more significantly, thus achieving more accurate defect localization and higher detection accuracy. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a detailed flowchart of the pseudo-defect construction steps of the present invention; Figure 3 This is a schematic diagram of the structure of the self-supervised model of the present invention. Detailed Implementation

[0026] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see the appendix Figure 1 , Figure 1 This is a schematic flowchart of a self-supervised defect detection method based on heatmap pseudo-defects according to an embodiment of the present invention. The present invention provides a self-supervised defect detection method based on heatmap pseudo-defects, which can be executed by a defect detection system, which can be implemented on one or more computing devices. The method may include the following steps: S100, execute the pseudo-defect construction step. This step is executed by a pseudo-defect construction module 100, whose function is to generate a pseudo-defect image that is reasonable in position and shape and has accurate annotation information using a normal image that does not contain defects, as the data input for subsequent model training.

[0028] S200, Perform the self-supervised model training step. This step is performed by a self-supervised model training module 200, whose function is to optimize and train a preset self-supervised network model using the pseudo-defect images and normal images generated in step S100, so that the model learns and establishes the ability to reconstruct features of normal images.

[0029] S300, Perform the defect detection step. This step is performed by a defect detection module 300, whose function is to use the self-supervised model trained in step S200 to analyze any input image to be tested, and output a judgment on whether the image has defects and the specific location information of the defects.

[0030] In a specific embodiment, the complete execution flow of the above method is as follows: First, the pseudo-defect construction module 100 receives a normal source image and a normal target image. Within this module, a feature heatmap generation unit 110 first processes the source image. This unit 110 uses a pre-trained convolutional neural network (in this embodiment, a ResNet50 network) as a feature extractor to extract the deep feature distribution of the source image. Subsequently, this unit 110 calculates the L2 norm of the feature vector at each spatial location in the feature distribution and processes it according to a preset normalization formula to generate a feature heatmap that can quantify the saliency of each region of the image.

[0031] Next, an anchor point and size determination unit 120 analyzes the feature heatmap output by the feature heatmap generation unit 110. This unit 120 retrieves and sorts all heatmap values, selecting the location with the largest value as the anchor point for defect implantation. Simultaneously, to determine the size of the pseudo-defect, this unit 120 first calculates a basic candidate box height and width based on the Gamma function and the original height and width of the source image. Then, to make the generated defect sizes more diverse, this unit 120 randomly samples a scaling factor from a standard normal distribution with a specified mean (e.g., 1) and variance (e.g., 0.5), and adjusts the aforementioned basic size using this scaling factor to obtain the final candidate box size used for extracting the image region.

[0032] Then, a fusion and annotation unit 130 extracts the corresponding candidate box regions from the source image based on the anchor points and candidate box sizes determined by the anchor point and size determination unit 120. This unit 130 then fuses these candidate box regions into a specified location in the target image, thereby generating an image containing the pseudo-defect. To support subsequent training, this unit 130 also performs a double annotation operation on the generated pseudo-defect image: first, it generates pixel-level contour localization information for the implanted candidate box regions as segmentation annotations; second, it labels the entire pseudo-defect image as a defective category as a classification annotation.

[0033] After the pseudo-defects are constructed, the self-supervised model training module 200 begins operation. This module first establishes a self-supervised model containing an encoder, decoder, and a multi-scale feature fusion network through its internal model building unit 210. In this embodiment, both the encoder and decoder are composed of ResNet34 network modules to maintain structural symmetry. After receiving training data (including normal images and pseudo-defect images generated in the preceding steps), unit 210 performs uniform preprocessing operations on all images, including resizing to 256×256 pixels, data augmentation such as random rotation and flipping, and finally random cropping to 224×224 pixels.

[0034] During model training, the encoder is responsible for extracting multi-scale feature maps of the input image at different levels. The multi-scale feature fusion network then processes these feature maps, specifically by: for the encoder's... Hierarchical feature maps are first fused with the previous level (the first level). The feature maps of the two layers are used to generate an interactive feature; before fusion, a 1×1 convolution is used to align the channel numbers of the two feature maps. Then, an attention mechanism is used to apply this interactive feature to the layer. The original feature map at each level is processed by query-key-value computation to obtain an enhanced feature map, which is then passed to the decoder.

[0035] A model optimization unit 220 is responsible for calculating the training loss of the model and updating the model parameters. For the encoder and decoder in the... For each corresponding layer of feature maps, unit 220 calculates a similarity score by measuring the cosine similarity between the two. The model's total loss function It is then defined as in all At each comparison level, corresponding The arithmetic mean of the values. Unit 220 employs the Adam optimizer, continuously minimizing the total loss function via backpropagation. Continue until the model parameters converge.

[0036] Finally, when defect detection is required, the defect detection module 300 receives a test image. Within this module, an inference and score map generation unit 310 inputs the test image into a trained self-supervised model, performs one forward propagation, and calculates the similarity scores between the encoder and decoder feature maps at each level. This unit 310 fuses the similarity score maps from all levels and upsamples them to the same size as the original test image to generate an anomaly score map.

[0037] A defect determination unit 320 analyzes the anomaly score map. In this map, normal regions that can be well reconstructed by the model have high similarity scores; while defective regions that the model has not learned from, because their features cannot be effectively reconstructed, have significantly lower similarity scores. Therefore, by setting a preset score threshold and binarizing the anomaly score map, the defect determination unit 320 can accurately identify regions with scores below the threshold as defective regions, thereby completing defect detection and localization.

[0038] Please see the appendix Figure 2 , Figure 2 This is a detailed flowchart of the pseudo-defect construction steps according to an embodiment of the present invention. This step aims to generate pseudo-defect images that are reasonable in location, shape, and size, and have precise annotations, using normal image samples, thus providing high-quality input data for subsequent self-supervised model training. In this embodiment, this step is specifically implemented through the following steps.

[0039] First, a normal source image without any known defects is provided. To locate a structurally or texturally significant region on this source image to simulate the location of a real defect, a feature heatmap needs to be generated. In this embodiment, a ResNet50 network pre-trained on the ImageNet dataset is used as the feature extractor. The source image is input into the ResNet50 network and a forward propagation is performed to extract feature distributions that characterize the high-level semantic information of the image from the network's deep convolutional layers.

[0040] After obtaining the feature distribution, the information in the feature distribution needs to be quantified to generate a feature heatmap. Specifically, the L2 norm of the feature vector at each spatial location in the feature distribution is calculated. The magnitude of this L2 norm value directly reflects the saliency of the corresponding image region. Subsequently, all calculated L2 norm values ​​are normalized to map them to a uniform numerical range, thereby generating a two-dimensional feature heatmap. In this heatmap, the brightness value of each pixel represents the feature saliency of the corresponding location in the source image.

[0041] Next, anchor points for extracting candidate boxes are located on the generated feature heatmap. Specifically, all pixels in the feature heatmap are traversed, and the point with the highest heatmap value is retrieved and located. The spatial coordinates of this highest heatmap point are selected as the anchor point for implanting false defects. To increase the diversity of the final generated samples, a heatmap value threshold can be set, and all points with heatmap values ​​higher than this threshold are included in a set of candidate anchor points.

[0042] After determining the anchor points, a reasonable size needs to be determined for the pseudo-defects, namely the height and width of the candidate bounding box. This process consists of two steps: First, to ensure that the defect size has a certain proportional relationship with the size of the object itself, the Gamma function is used, combined with the original height and width of the source image, to calculate a basic candidate bounding box height and width. Then, to simulate the randomness of defect size in the real world, a scaling factor is randomly sampled from a pre-defined standard normal distribution with a mean of 1 and a variance of 0.5. Multiplying this scaling factor by the previously calculated basic height and width yields the final candidate bounding box height and width used to extract the image region.

[0043] Finally, image fusion and annotation are performed. Another normal target image, also free of defects, is taken. Using the same method as the source image—feature extraction and normalization—a feature heatmap is generated for this target image, and an anchor point for the fusion operation is determined. Then, based on the anchor point determined in the source image and the calculated candidate bounding box height and width, the corresponding rectangular image region is extracted from the source image as the candidate bounding box region. This candidate bounding box region is then fused to the anchor point region of the target image, which can be achieved through direct overlay. At this point, an image containing pseudo-defects is constructed.

[0044] To enable the pseudo-defect image to be used for supervised training of the subsequent model, it needs to be doubly labeled. The first layer is segmentation labeling, which generates a binary mask image with the exact same size as the pseudo-defect image. The pixel values ​​of the corresponding candidate box regions in the mask image are marked as 1, while the pixel values ​​of the remaining background regions are marked as 0, thus providing the model with precise pixel-level location information of the defect. The second layer is classification labeling, which assigns a "defective" category label to the entire constructed pseudo-defect image, while all the original normal images are assigned a "no-defect" category label.

[0045] Please see the appendix Figure 3 , Figure 3 This is a schematic diagram of a self-supervised model according to an embodiment of the present invention. The model is designed to learn and reconstruct features of normal images, thereby gaining the ability to identify abnormal (i.e., defective) regions. Structurally, the model mainly includes an encoder 310, a decoder 320, and a multi-scale feature fusion network 330 disposed between the encoder 310 and the decoder 320.

[0046] In this embodiment, the overall framework of the model is based on an encoder-decoder structure. The function of the encoder 310 is to receive a preprocessed input image and extract its hierarchical feature maps at different scales layer by layer. Specifically, the encoder 310 uses a ResNet34 network module as its basic building block. This network has multiple convolutional blocks, each of which can extract features at different abstraction levels, from high-resolution detail features such as color and texture at the bottom layer to low-resolution semantic features containing rich contextual information at the top layer.

[0047] The function of decoder 320 is to receive the features processed by multi-scale feature fusion network 330 and to restore these feature maps layer by layer to the same spatial resolution and feature dimension as the corresponding layer feature map of encoder 310. To facilitate subsequent direct similarity calculation of the features output by encoder and decoder, decoder 320 adopts a network structure similar to encoder 310, also composed of ResNet34 network modules, and gradually expands the spatial size of feature maps through upsampling operations (e.g., bilinear interpolation) to match the size of the corresponding layer of encoder.

[0048] The multi-scale feature fusion network 330 is a key component in this embodiment. Its function is to interact with and enhance the multi-scale features extracted by the encoder 310 to generate more expressive features. This network processes the output feature maps of multiple selected levels (three levels in this embodiment) in the encoder 310. For the first level of the multi-scale feature fusion network 330... Feature maps output by layer (i≥2) The multi-scale feature fusion network 330 will combine it with features from its previous layer (i.e., the first layer). Feature maps of layers with lower resolution To integrate.

[0049] Before fusion, since feature maps at different levels typically have different numbers of channels, a 1×1 convolutional layer is first used to transform the channel dimensions of one or two feature maps to ensure they have the same number of channels, thus allowing for subsequent fusion operations. After the fusion operation (e.g., through element-wise addition or channel concatenation of feature maps), an interactive feature is generated. .

[0050] Subsequently, to enable the model to focus on more important information in the feature map, this interactive feature... It is used in an attention mechanism to process the original feature map of the current layer. Weighted enhancement is applied. Specifically, in the computation of this attention mechanism, interaction features... The original feature map of the current layer is used as a query. This is then used as both the key and the value. A set of attention weights is obtained by calculating the dot product of the query and the key, followed by scaling and normalization using a softmax function. These weights are then applied to the value (i.e., the original feature map). On this basis, a feature map that has undergone attention enhancement and highlights important feature regions can be generated. After processing by the multi-scale feature fusion network 330, these enhanced feature maps It is transmitted to the corresponding level of decoder 320 as input for its reconstruction task.

[0051] The description in this section corresponds to Figure 1 The self-supervised model training step S200 in the overall process is shown. The purpose of this step is to optimize the parameters of the self-supervised model using existing data, enabling it to establish an accurate reconstruction capability for normal image features.

[0052] Self-supervised model training steps: First, the training data is prepared and preprocessed. The training dataset consists of constructed pseudo-defect images with double annotations and original, normal images without any defects. To improve the model's robustness and reduce overfitting, a uniform and standardized data preprocessing procedure needs to be performed on all images in this dataset.

[0053] In one specific embodiment, the preprocessing flow includes: First, scaling all input images to a uniform size of 256×256 pixels. Next, performing data augmentation operations on the scaled images, including randomly flipping the images horizontally with a 50% probability, randomly flipping them vertically with a 50% probability, and randomly rotating them within a range of -10 to +10 degrees. Finally, randomly cropping a 224×224 pixel region from the augmented images as the final input to the self-supervised model.

[0054] During training, the model aims to learn a mapping from encoder features to decoder reconstructed features, enabling the decoder to accurately reconstruct the features extracted by the encoder for normal images or normal image regions. When the model receives an input image (whether normal or containing pseudo-defects), the image data flows through the encoder, generating feature maps at multiple predefined levels. These feature maps are then processed by a multi-scale feature fusion network and fed into the decoder. The decoder then outputs the reconstructed feature maps at the corresponding levels.

[0055] The optimization objective of model training is defined by a loss function. This loss function aims to quantify the difference between the raw features extracted by the encoder and the reconstructed features recovered by the decoder. Specifically, a loss function is selected from the encoder and decoder. The hierarchy corresponding to each spatial dimension (in this embodiment) =3), for the first Each level acquires the feature maps extracted by the encoder. and the feature map reconstructed by the decoder The similarity score for this level is obtained by calculating the cosine similarity between the two feature maps. .

[0056] The model's total loss function Defined as all Each level ( The loss function is the arithmetic mean of the original features of the encoder and the reconstructed features of the decoder. The smaller the value of this loss function, the more similar the original features of the encoder and the reconstructed features of the decoder are as a whole. By minimizing this loss function, which is equivalent to maximizing the cosine similarity of all corresponding level features, the model is driven to learn the inherent feature distribution and reconstruction pattern of normal images.

[0057] To minimize the loss function, this embodiment uses the Adam optimizer to iteratively update all trainable parameters of the model (including weights and biases in the encoder, decoder, and multi-scale feature fusion network). In each iteration, the total loss function is calculated using the backpropagation algorithm. The gradients of each model parameter are calculated, and the parameters are updated based on these gradients. This training process is repeated for multiple epochs until the total loss function converges on the validation set or reaches the preset number of training epochs, marking the completion of model training.

[0058] The description in this section corresponds to Figure 1 The defect detection step S300 in the overall process is shown. This step aims to apply a trained self-supervised model to analyze any input image to be tested, so as to achieve automatic defect detection and accurate localization.

[0059] Defect detection and location steps: When an image needs to be detected, it is first input into a trained self-supervised model. The model then performs a complete forward propagation on the input. During this forward propagation, the data of the image flows sequentially through the encoder, the multi-scale feature fusion network, and the decoder, generating the outputs of each layer.

[0060] In the core stage of the reasoning process, the system computational encoder at the 1st... Feature maps extracted hierarchically Reconstructed feature maps recovered by the decoder at the corresponding level Similarity score between This calculation is performed in all preset scenarios. This process is performed at each corresponding level, thereby generating a two-dimensional similarity score map for each level.

[0061] To obtain a comprehensive assessment of the degree of anomaly in the entire image, it is necessary to consider the above... The similarity score maps at different levels are fused into a single anomaly score map. In one specific embodiment, this fusion operation is accomplished by upsampling all levels of similarity score maps to unify their spatial resolution, then aligning them and calculating a pixel-wise average at each pixel location. Subsequently, this fused composite score map is upsampled using bilinear interpolation and other methods to restore it to a size completely consistent with the original input size of the image under test, thus obtaining the final anomaly score map.

[0062] In this anomaly score map, the value of each pixel represents the normality of the corresponding spatial location in the test image or its similarity to normal samples. Since the self-supervised model is trained by learning feature reconstruction patterns from a large number of normal samples, it can effectively reconstruct features in normal regions of the test image, resulting in higher scores for these regions on the anomaly score map. Conversely, for defective regions not encountered during training, their feature patterns differ significantly from normal samples, preventing effective reconstruction and leading to a significant decrease in feature similarity scores. These defective regions then appear as isolated, low-scoring areas on the anomaly score map.

[0063] To ultimately identify and segment defects from the anomaly score image, the system performs binarization on the image using a preset score threshold. Specifically, all pixels with scores below the preset threshold are identified as defect pixels, while all pixels with scores equal to or higher than the threshold are identified as normal background pixels. This operation generates a binary mask image of the same size as the image under test, clearly marking the precise outline and location of the defect area, thus simultaneously completing defect detection (determining its existence) and location (marking its position).

[0064] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A self-supervised defect detection method based on heatmap pseudo-defects, characterized in that, This includes the following interrelated steps: The pseudo-defect image construction step involves extracting features from a normal source image to generate a feature heatmap that can represent the saliency of the image region, locating an anchor point for implanting a defect on the feature heatmap, extracting a defect patch from the source image based on the anchor point, and then fusing the defect patch into the anchor point region of a normal target image to construct a labeled pseudo-defect image. The self-supervised model training step uses the pseudo-defect image and normal image constructed in the preceding steps to train a self-supervised model containing an encoder and a decoder. The training aims to enable the model to learn the feature reconstruction pattern of normal images by minimizing the difference between the feature map extracted by the encoder and the feature map reconstructed by the decoder. The defect detection step utilizes the self-supervised model obtained through the aforementioned training step to detect a test image. This detection is achieved by calculating the similarity score between the encoder feature map generated by the model and the decoder reconstructed feature map of the test image, wherein regions with low similarity scores are identified as defect regions. The specific steps for constructing the pseudo-defect include: The feature distribution of the source image is extracted using a pre-trained convolutional neural network, and the feature distribution is normalized to generate the feature heatmap. The location with the highest thermal value on the characteristic heat map is selected as the anchor point; The height and width of the candidate box are determined using the Gamma function and a random scaling factor. Candidate box regions are extracted from the source image based on the anchor points and the height and width of the candidate boxes, and then fused into the target image. The fused pseudo-defect image is then classified and segmented. The self-supervised model further includes a multi-scale feature fusion network located between the encoder and decoder, and the training process is as follows: The image is input into the encoder, and multi-scale feature maps are extracted at different levels. The multi-scale feature fusion network fuses and enhances the multi-scale feature maps; The decoder receives the fused and enhanced features and recovers the reconstructed feature maps corresponding to each level of the encoder; A loss function is defined by calculating the similarity score between the multi-scale feature map of the encoder and the reconstructed feature map of the decoder, and the model parameters are optimized by minimizing the loss function. The multi-scale feature fusion network fuses and enhances the multi-scale feature maps in the following ways: For the encoder's first Hierarchical feature map , integrating with its previous level, namely the first Hierarchical feature maps Generate an interactive feature ; Using attention mechanisms to analyze the interaction features and the first... The hierarchical feature maps are processed to generate enhanced features.

2. The self-supervised defect detection method based on heatmap pseudo-defects according to claim 1, characterized in that, The method for normalizing the feature distribution is as follows: calculate the L2 norm of each feature vector in the feature distribution, and then normalize it using the following formula: ； In the formula, These are the normalized feature vectors; The L2 norm of the current feature vector; The smallest L2 norm eigenvalue; It is the eigenvalue of the largest L2 norm.

3. The self-supervised defect detection method based on heatmap pseudo-defects according to claim 2, characterized in that, The method for determining the height and width of the candidate box is as follows: Through the Gamma function With respect to the height of the source image and width Calculate the base height and base width ; A scaling factor is randomly sampled from a standard normal distribution with a mean of 1 and a variance of 0.

5. ; The base height and base width With the scaling factor Multiply by each other to get the final candidate box height. and width .

4. The self-supervised defect detection method based on heatmap pseudo-defects according to claim 1, characterized in that, The loss function The calculation method is as follows: ； In the formula, This is the total loss function of the model; The total number of layers for feature comparison; For the first The similarity score for each level is calculated as follows: ; For the encoder in the first Feature maps extracted hierarchically; For the decoder in the first Reconstructed feature map of hierarchical restoration; This is the transpose operation for a matrix; Let L2 be the norm of the vector.

5. The self-supervised defect detection method based on heatmap pseudo-defects according to claim 1, characterized in that, The attention mechanism is calculated as follows: ； In the formula, The enhanced features; The interaction feature serves as the query in the attention mechanism; For the first The hierarchical feature map serves as both the key and value in the attention mechanism. For the first The dimension of layer features; It is a normalized exponential function; This is the transpose operation for a matrix.

6. The self-supervised defect detection method based on heatmap pseudo-defects according to claim 1, characterized in that, The defect detection steps specifically include: The image to be tested is input into the trained self-supervised model for one forward propagation; Calculate the similarity scores between the encoder and the decoder at each level of feature maps; The similarity scores at each level are merged into an anomaly score map with the same size as the image under test; On the abnormal score map, areas with scores below a preset threshold are identified as defective areas.

7. The self-supervised defect detection method based on heatmap pseudo-defects according to claim 1, characterized in that, The segmentation and annotation of the pseudo-defect image is to generate pixel-level contour positioning information for the implanted candidate box region; the classification and annotation is to label the entire pseudo-defect image as a defective category.

Images