Defect detection method and device based on residual contrast learning and medium

Abstract

The present application relates to a defect detection method, device and medium based on residual contrast learning, and the core is a defect recognition model, comprising: an encoder feature extraction module for inputting an image and extracting multi-level semantic features as encoded features; a decoder feature reconstruction module for layer-by-layer reconstruction of the last feature map of the encoded features to generate a reconstructed feature aligned with the last feature map in the same scale; a residual calculation and projection module for calculating the residual between the encoded features and the reconstructed features, generating image block-level residual features representing image abnormal information, and projecting to a preset contrast learning space to obtain image block-level residual embedding features; and a defect recognition module guiding normal samples and known category defect features to fit to respective corresponding centers in the residual space through residual contrast learning; and outputting a defect recognition result through the image block-level residual embedding features and the fitting result.

Description

Technical Field

[0001] This invention relates to the field of defect detection, and in particular to a defect detection method, device and medium based on residual contrastive learning. Background Technology

[0002] In the field of industrial image anomaly detection, traditional methods mainly rely on manual visual inspection, which suffers from low efficiency, high cost, and fatigue. With the development of deep learning technology, especially deep learning-based image anomaly detection methods, which have gradually become mainstream, their application in industrial manufacturing shows great potential. However, although existing technologies achieve high image patch-level accuracy on certain datasets, they still face many challenges in real-world industrial scenarios, such as: a scarcity of defect samples, high levels of background noise in images, and the existence of unknown defect categories.

[0003] Current mainstream deep learning methods can be divided into two main paradigms based on their core ideas: feature embedding-based methods and reconstruction-based methods. In addition, there are supervised methods developed under specific supervised samples.

[0004] The core idea of ​​feature embedding-based methods is to assume a model pre-trained on a large natural image dataset, capable of extracting powerful, general features. During training, only normal samples are used, allowing the model to learn its "normal" feature patterns. During inference, anomalous samples exhibit features different from the learned normal patterns and are thus detected. Different implementation architectures are also available.

[0005] Teacher-student architecture: A fixed, pre-trained model is used as the "teacher" network, whose weights are frozen during training. A trainable "student" network is trained to mimic the features produced by the teacher network on normal images. For anomalous images, the student network fails to mimic them well, resulting in feature discrepancies.

[0006] Single-class architecture: This treats the problem as a single-classification problem, aiming to learn a compact boundary (such as a hypersphere) of a "normal" region in the feature space. During testing, samples falling outside the boundary are considered anomalous. This also utilizes techniques for creating anomalous samples, such as CutPaste, which synthesizes anomalous samples by cutting and pasting parts of normal images, transforming the problem into a binary classification problem; or it uses comparative learning to bring features closer to normal samples and push features further away from synthesized anomalous samples.

[0007] Distribution mapping: Normal image features extracted by the pre-trained model are fitted to a simple prior distribution (such as a multivariate Gaussian distribution) through a learnable mapping module. During testing, the features of abnormal images will have a very low likelihood probability under this distribution.

[0008] Training memory: During the training phase, features of a large number of normal samples (usually features of image patches) are stored in a "training memory". During inference, the features of the test image are compared with the normal features in the training memory using a nearest neighbor search; regions that are far away are considered anomalous.

[0009] The core idea of ​​reconstruction-based methods is to train a model (such as an autoencoder) to learn how to reconstruct abnormal images into normal images, or in other words, to reconstruct the features of normal images. Since the model has only seen normal samples, it cannot effectively reconstruct the abnormal parts it has not seen before. Therefore, anomalies are detected by comparing the differences (reconstruction error) between the original image and the reconstructed image. Reconstruction networks also come in different model architectures:

[0010] Autoencoders: Using an encoder-decoder structure, the model is trained to compress and reconstruct the input image, with the goal of making the reconstructed image as similar as possible to the original normal image.

[0011] Generative Adversarial Networks (GANs) utilize a generator and a discriminator. The generator is trained to reconstruct normal images, while the discriminator distinguishes between real, normal images and reconstructed images. Abnormal images are poorly reconstructed and easily identified by the discriminator.

[0012] Transformer: Utilizing the powerful global modeling capabilities of the Transformer to replace the convolutional network in the autoencoder as the encoder and decoder of the reconstruction network, it can better capture long-distance dependencies and is beneficial for reconstructing normal images with complex structures.

[0013] Diffusion model: The denoising process in the diffusion model gradually reconstructs a normal image from noise. For anomalous images, the denoising process or intermediate gradients will differ significantly from those of normal images.

[0014] Supervised anomaly detection involves adding a small number of anomalous samples to normal samples, or combining pseudo-anomaly synthesis techniques with supervised learning. However, due to the imbalance of sample categories, some methods combine reinforcement learning to select batches with high information content, or use attention loss in weakly supervised settings to make the model focus on anomalous regions, which significantly improves the model's defect detection capability with few samples.

[0015] Although existing technologies have achieved image-level defect recognition accuracy of over 97% on open-source datasets such as MVTec AD and VisA, current methods still have significant limitations: unsupervised methods can locate unknown defects but cannot identify their specific categories; while supervised methods, although capable of both location and recognition of known categories, are powerless against new defect types not encountered during the training phase. Given the high uncertainty and uncontrollability of unknown defects in industrial scenarios, how to maintain accurate defect location capabilities while further identifying defect categories, especially the identification of unknown defect types, has become a key technical challenge urgently needing breakthroughs in the field of industrial quality inspection. Summary of the Invention

[0016] To address the aforementioned technical problems, this invention provides a defect detection method based on residual contrastive learning, comprising:

[0017] Acquire the image to be identified and input it into the defect identification model; the defect identification model includes:

[0018] The encoder feature extraction module is used to extract multi-level semantic features from the input image as encoded features.

[0019] The decoder feature reconstruction module is used to reconstruct the last layer feature map of the encoded features layer by layer, generating reconstructed features that are aligned with the last layer feature map at the same scale.

[0020] The residual calculation and projection module is used to calculate the residual between the encoded features and the reconstructed features, generate image block-level residual features that characterize image anomaly information, and project them onto a preset contrast learning space to obtain image block-level residual embedding features.

[0021] The defect identification module guides the defect features of normal samples and known categories to fit to their respective centers in the residual space through residual contrast learning; and outputs the defect identification results by embedding image block-level residual features and fitting results.

[0022] Furthermore, the processing steps of the residual calculation and projection module include:

[0023] In the multi-level semantic features of the coding features, the first few layers and the last few layers are fused respectively to obtain the first coding fusion feature and the second coding fusion feature;

[0024] The first coding fusion feature, the second coding fusion feature, and the reconstructed feature are divided into several image blocks in the spatial dimension. Each image block corresponds to a local feature vector to calculate the image block-level residual features.

[0025] The image block-level residual features are projected to obtain the image block-level residual embedding features.

[0026] Furthermore, the decoder feature reconstruction module employs an 8-layer Transformer decoder;

[0027] The residual calculation and projection module divides the features of the first 8 layers of the encoder into layers 1-4 and layers 5-8, and performs fusion of the two parts.

[0028] The fused encoded features and reconstructed features are divided into several image patches in the spatial dimension to obtain the first... The residual features of each image patch are represented as follows:

[0029] ;

[0030] in, For the first Residual features of each image patch; For encoder number The first layer Encoded features of an image patch; For encoder number The first layer Reconstruction features of an image patch Number of image patches;

[0031] By introducing a multilayer perceptron projection head, the image block-level residual features are mapped from the original feature space to the contrastive learning embedding space to obtain the image block-level residual embedding features.

[0032] Furthermore, the processing steps of the defect identification module include:

[0033] Define a learnable reconstruction center and use feature reconstruction loss to make the residual features of normal samples approach the reconstruction center;

[0034] Based on the number of known defect categories ,definition Each learnable category center corresponds to Given a known defect center and a positive sample center, and using image block-level residual contrastive learning loss, a clustered distribution with compact intra-class and large inter-class spacing is obtained in the contrastive space;

[0035] Based on the residual embedding features and cluster distribution, the output includes in-domain categories and out-of-domain labels; where the in-domain categories are normal categories or a known defect category; and the out-of-domain labels are unknown defects.

[0036] Furthermore, based on the residual embedding features and cluster distribution, the steps for outputting in-domain categories and out-of-domain annotations include:

[0037] Calculate the distance from each image block-level residual feature to the center of each class;

[0038] Calculate the energy value corresponding to each image patch;

[0039] Determine whether the energy value corresponding to each image patch is greater than the set energy threshold. If not, the identification result is an in-domain category, and the specific category is determined based on the distance. If yes, the identification result is an out-of-domain label. The energy threshold is determined by the high quantile of the ID energy distribution under the target false alarm rate (FPR).

[0040] Furthermore, the feature reconstruction loss is determined by a weighted sum of the local mean square error loss and the global cosine similarity loss, and is expressed as:

[0041] ;

[0042] ;

[0043] ;

[0044] in, For feature reconstruction loss, For local mean square error loss, For global cosine similarity loss, The weighting coefficients for feature reconstruction loss are... For batch size, The number of image patches, For the first The first image patch Batch residual characteristics To reconstruct the center, For the first The residual features of each batch are flattened vectors along the spatial dimension.

[0045] Furthermore, the image block-level residual contrastive learning loss is expressed as:

[0046] ;

[0047] in, For image block-level residual contrastive learning loss, The number of image patches, For the first Residual embedding features of each image patch Corresponding to positive samples, The corresponding set is the negative sample set. The second temperature coefficient; It is a similarity function;

[0048] To address the issue of uneven defect sample distribution, an imbalanced weighting mechanism is introduced on top of the image block-level residual contrastive learning loss. This mechanism assigns higher weights to defect categories with fewer samples, multiplying the image block-level residual contrastive learning loss by the frequency of each defect category.

[0049] Furthermore, to simultaneously optimize reconstruction localization and residual spatial separability, a joint loss is constructed, which is determined by weighting the feature reconstruction loss and the image block-level residual contrastive learning loss, and is expressed as:

[0050] ;

[0051] in, For joint losses, For feature reconstruction loss, For image block-level residual contrastive learning loss, This represents the joint loss weighting coefficient.

[0052] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-described defect detection methods.

[0053] On the other hand, the present invention also provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-mentioned defect detection methods.

[0054] This invention provides a defect detection method, device, and medium based on residual contrastive learning. By introducing encoding extraction and decoding reconstruction, combined with residual space construction and contrastive learning mechanism, it solves the shortcomings of traditional defect detection schemes in semi-open set identification, false negative and missed detection control, new defect discovery, and model generalization ability. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the framework of an embodiment of the defect detection method based on residual contrastive learning of the present invention;

[0056] Figure 2 This is a schematic diagram of the contrastive learning method in one embodiment of the defect detection method based on residual contrastive learning of the present invention;

[0057] Figure 3 This is a flowchart of an embodiment of the training process of the defect detection method based on residual contrastive learning of the present invention;

[0058] Figure 4 This is a flowchart of an embodiment of the inference process of the defect detection method based on residual contrastive learning of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0060] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., these directional indications are only used to explain the relative positional relationships and movement of the components in a specific posture. If the specific posture changes, the directional indications will also change accordingly. Furthermore, if the embodiments of the present invention involve descriptions such as "first," "second," "S1," "S2," "step one," "step two," etc., these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance, or implicitly indicating the number of technical features indicated or the order of method execution. Those skilled in the art will understand that anything that does not violate the inventive concept and is within the scope of the present invention should be included in the protection scope of the present invention.

[0061] like Figure 1-4 As shown, to address the contradictions in existing technologies for industrial defect detection—namely, the difficulty in locating defects and the difficulty in classifying them, especially the insufficient ability to identify semi-open sets—this invention provides a defect detection method based on residual contrastive learning, comprising:

[0062] Acquire the image to be identified and input it into the defect identification model; the defect identification model includes:

[0063] (a) Encoder Feature Extraction Module (Input: Output: )

[0064] The encoder feature extraction module is used to extract multi-level semantic features from the input image as encoded features; where the input image is denoted as... , Let be the spatial dimensions of the input image, representing pixels in the height and width directions respectively; and let be the output encoded features, denoted as . , , For the first The spatial dimension of layer-encoded features This is the dimension for encoding features.

[0065] Specifically, the encoder feature extraction module can optionally employ a pre-trained visual feature extraction network to learn representations of the input image, outputting multi-level semantic features as encoded features:

[0066] (1)

[0067] in, Indicates the encoder's first... Visual feature extraction network of layers, Indicates the encoder's first... Layer (or the first) The feature output (at each scale / stage) is used to provide "semantic anchors" for subsequent reconstruction and residual construction. In the DINOv2-B encoder used in this method... The option is 12.

[0068] It is worth noting that the encoder of this invention can employ a visual feature extraction network pre-trained on large-scale data based on self-supervision or weak supervision. In addition to ViT-type structures, convolutional neural networks or hybrid architectures can also be used (such as CNNs with pyramid / multi-scale outputs, hierarchical Transformers like Swing, or CNN-Transformer hybrid backbones), as long as they can output one or more layers of semantic features for subsequent reconstruction and residual construction.

[0069] (ii) Noise-adding module for neck network (input: Output: )

[0070] To improve robustness, random deactivation / perturbation can be introduced into the feature layer to mask certain channels or spatial locations:

[0071] (2)

[0072] in, Indicates the neck network Features after layer enhancement; Indicates the neck network The feature mask tensor of the layer; Element-by-element multiplication; middle, and The spatial dimensions of the corresponding feature map (pixel positions in the height and width directions); The channel dimension of the corresponding feature map; The probability of inactivation is denoted as . This operation is equivalent to injecting noise into the feature domain to construct "pseudo-anomalies," forcing the decoder to learn stable structural information rather than random textures.

[0073] (III) Decoder Feature Reconstruction Module (Input: or Output: )

[0074] The decoder feature reconstruction module is used to reconstruct the last layer feature map of the encoded features (which can be multi-level semantic features directly output by the encoder, or multi-level semantic features output by the encoder after being enhanced by the neck network) layer by layer, and generate reconstructed features that are aligned with the same scale as the feature map of that layer.

[0075] Preferably, a feature reconstruction decoder is constructed to reconstruct the feature map of the last layer of the encoder after neck network enhancement layer by layer, generating reconstructed features that are aligned with the encoded features of that layer at the same scale.

[0076] (3)

[0077] in, The reconstructed features are the output of the decoder; For the Transformer decoder, an 8-layer Transformer decoder can be selected. Its goal is to reconstruct the features of the first 8 layers of encoding while "mainly preserving the normal structure", so that the abnormal regions are highlighted in the residuals and the pixel details of the shallow layers of the DINO model are preserved.

[0078] (iv) Residual Calculation and Projection Module (Calculation Process: Input:) or ,as well as Output: (Projection process: Input:) Output: )

[0079] The residual calculation and projection module is used to calculate the residual between the encoded features and the reconstructed features, generate image block-level residual features that characterize image anomaly information, and project them onto a preset contrast learning space to obtain image block-level residual embedding features.

[0080] Specifically, the processing steps of the residual calculation and projection module include:

[0081] a1: In the multi-level semantic features of the encoding features, the first few layers and the last few layers are fused separately to obtain the first encoding fusion feature and the second encoding fusion feature; Example: Divide the first 8 layers of encoder features into layers 1-4 and layers 5-8, and fuse the two parts;

[0082] a2: Divide the first coded fusion feature, the second coded fusion feature, and the reconstructed feature into several image blocks in a spatial dimension. Each image block corresponds to a local feature vector to calculate the image block-level residual features. For example, divide the fused coded features and reconstructed features into several image blocks in a spatial dimension. Each image block corresponds to a local feature vector to obtain the first image block-level residual features. The residual features of each image patch are represented as follows:

[0083] (4)

[0084] in, For the first The residual features of each image patch are used to form the residual features as follows: , ; For encoder number The first layer Encoded features of an image patch; For encoder number The first layer Reconstruction features of an image patch Number of image patches ( , The spatial dimension of the input image. (This refers to the factor by which the feature is downsampled compared to the original resolution). This residual feature characterizes the degree of deviation of a local region from the reconstruction of the normal structure and is the direct carrier of anomalous semantics.

[0085] a3: Project the image block-level residual features to obtain the image block-level residual embedding features. Specifically, a multilayer perceptron (MLP) projection head can be optionally introduced. The image block-level residual features are mapped from the original feature space to the contrastive learning embedding space to obtain the image block-level residual embedding features:

[0086] (5)

[0087] in, For the first The residual embedding features of each image patch are used to form the projected residual features, i.e.: residual embedding features , To compare spatial dimensions, For projection head functions, For the first The residual vector of each image patch, express Normalization is used to constrain the distribution of embedded vectors and stabilize contrastive learning training.

[0088] (v) Defect Identification Module (Input: Output: Recognition results, optionally a pixel-level anomaly category map)

[0089] The defect identification module guides the defect features of normal samples and known categories to fit to their respective centers in the residual space through residual contrastive learning; and outputs the defect identification results through residual embedding features and fitting results.

[0090] Specifically, the processing steps of the defect identification module include:

[0091] b1: Define a learnable refactoring center , The feature reconstruction loss is used to make the residual features of normal samples approach the reconstruction center; specifically, the feature reconstruction loss can be selected from formulas (10)-(12).

[0092] b2: Based on the number of known defect categories ,definition Center for Learnable Categories , , To decode spatial dimensions, , There are learnable category centers, each corresponding to a positive sample center and... Given a set of known defect centers and a set of known defect categories, which can be optionally represented as... ; By using image block-level residual contrastive learning loss, a clustered distribution with compact intra-class and large inter-class spacing is obtained in the contrast space; preferably, the image block-level residual contrastive learning loss adopts formula (13)-(14).

[0093] b3: Based on the residual embedding features and cluster distribution, output the in-domain category and the out-of-domain label; where the in-domain category is the normal category or a known defect category; the out-of-domain label is the unknown defect. Preferably, it includes:

[0094] c1: Calculate the distance from each image patch-level residual feature to the center of each category. Specifically, this distance can be calculated using cosine similarity, or other metrics such as Euclidean distance or Mahalanobis distance, and used as the logits score of the residual image patch belonging to each category. More specifically, the score of each residual image patch belonging to each category can be determined by the distance from each image patch-level residual feature to the center of each category. The category corresponding to the highest score (normal category or a known defect category) is then selected as the in-domain recognition result. This can be represented as:

[0095] (6)

[0096] in, For the first The image patch in the first Scores in each category For the first Residual embedding features of each image patch; For the first One learnable category center; Number of image patches; Number of defect categories;

[0097] c2: Calculate the energy value corresponding to each image patch; specifically, using an energy function for calculation:

[0098] (7)

[0099] in, For the first Energy value of each image patch This is the first temperature coefficient, used to control the sensitivity to the logits distribution. The number of anomaly categories. For the first The image patch in the first Scores in each category;

[0100] c3: Determine whether the energy value corresponding to each image patch is greater than the set energy threshold. If not, the recognition result is an in-domain category, and the specific category is determined based on the distance. If yes, the recognition result is an out-of-domain label. The energy threshold is determined by the high quantile of the ID energy distribution under the target false alarm rate (FPR).

[0101] Specifically, the judgment rule is expressed as follows:

[0102] (8)

[0103] in, For the first The energy value of each image patch, OOD is the out-of-domain annotation, indicating an out-of-domain anomalous region with an unknown defect that has not been seen before; ID is the in-domain category, indicating the in-domain category that has been seen before, specifically normal or a known defect; This is the energy threshold.

[0104] More preferably, the energy threshold can be dynamically adjusted to achieve the target false positive rate (FPR). Below, the highest quantile of the ID energy is taken, and it is represented as:

[0105] (9)

[0106] in, Energy threshold The quantile function represents the normal distribution. Indicates ID energy.

[0107] (vi) Construction of loss function

[0108] (1) Feature reconstruction loss:

[0109] The feature reconstruction loss is divided into two parts, determined by a weighted sum of the local mean square error loss and the global cosine similarity loss, expressed as:

[0110] (10)

[0111] (11)

[0112] (12)

[0113] in, For feature reconstruction loss, For local mean square error loss, For global cosine similarity loss, The weighting coefficients for feature reconstruction loss are... For batch size, The number of image patches, For the first The first image patch Batch residual characteristics To reconstruct the center, For the first The residual features of each batch, flattened along the spatial dimension, are vectors with the shape of... .

[0114] (2) Image block-level residual contrastive learning loss:

[0115] Based on the constraints of residual reconstruction, contrastive learning optimization is performed on the residual embedding after MLP projection. To address the issue of uneven defect sample distribution, an imbalanced weighting mechanism is introduced on top of this contrastive loss, assigning higher weights to defect categories with fewer samples. Optionally, the image block-level residual contrastive learning loss can be multiplied by the frequency of each defect category. Specifically, the image block-level residual contrastive learning loss is expressed as:

[0116] (13)

[0117] in, For image block-level residual contrastive learning loss, The number of image patches, For the first Residual embedding features of each image patch Corresponding to positive samples, The corresponding set is the negative sample set. The second temperature coefficient; The similarity function can be optionally defined as:

[0118] (14)

[0119] The contrastive learning loss brings the representations of homogeneous image patches closer together and pulls the representations of heterogeneous image patches further apart in the residual embedding space, so that unknown anomalies naturally deviate from the normal pattern clustering area in the embedding space, thereby supporting the semi-open set decision.

[0120] (3) Joint losses

[0121] To simultaneously optimize reconstruction localization and residual spatial separability, this invention constructs a joint loss, which is determined by weighting the aforementioned feature reconstruction loss and image block-level residual contrastive learning loss, and is expressed as:

[0122] (15)

[0123] in, For joint losses, For feature reconstruction loss, For image block-level residual contrastive learning loss, This represents the joint loss weighting coefficient.

[0124] In summary, this invention provides a defect detection method based on residual contrastive learning, and presents a semi-open set defect recognition model. Its overall process is based on a framework of "pre-trained feature extraction + feature reconstruction + residual contrastive learning," which can simultaneously satisfy the following requirements in the semi-open set defect detection scenario: ① accurate localization of image block-level defect regions (pixel level); ② reliable classification of known defect categories during the training phase (in-domain categories); ③ discriminative discovery of defect types not seen during the training phase (out-of-domain annotation), achieving the integration of localization, known classification, and unknown recognition. Compared with existing technologies, the technical solution of this invention has at least the following beneficial effects:

[0125] (1) Achieving a unified output of "localization, classification, and open set rejection": Existing methods often can only achieve one of the following: anomaly localization or known classification, making it difficult to take into account the discovery of unknown defects in semi-open set scenarios. This invention enables the model to simultaneously output pixel-level defect regions, known defect categories, and unknown defect labels within the same framework through the chain of "multi-layer feature reconstruction → image block-level residual construction → residual to category prototype / category center discrimination". This effect is mainly achieved by the feature reconstruction module, the residual construction mechanism, and the energy / threshold-based open set decision module.

[0126] (2) Achieving a capability extension from unsupervised to semi-supervised with a small number of annotations, significantly reducing annotation costs: Existing unsupervised reconstruction / embedding methods can usually only distinguish between normal and abnormal, making it difficult to obtain stable multi-class recognition capabilities when defect categories are scarce. This invention introduces category prototypes (category centers) and establishes alignment / contrast constraints of "residual-prototype" in the residual embedding space, enabling a small number of labeled samples to form a transferable category benchmark, thereby extending the unsupervised residual framework to semi-supervised known category recognition; this effect is brought about by the category prototype / center learning mechanism + image block-level residual contrastive learning loss (InfoNCE, etc.).

[0127] (3) More reliable identification of unknown defects with calibrable and reproducible thresholds, reducing false alarms: Existing open set discrimination often relies on empirical thresholds or unstable classification confidence, making it difficult to control the false alarm rate across products / operating conditions. This invention constructs OOD scores based on "distance → logits → energy function" and uses the high quantile of ID energy distribution under the target false alarm rate FPR to determine the threshold, making the rejection rules quantifiable and reproducible, thereby improving the stability of unknown defect discovery; this effect is brought about by the energy function construction + quantile threshold calibration mechanism (including temperature coefficient adjustment).

[0128] (4) More robust to noise background and working condition disturbances, and stronger generalization: Industrial images often have complex textures, lighting changes and noise interference, which makes existing methods prone to overfitting random textures. This invention introduces random deactivation / perturbation (pseudo-anomalies) in the feature domain and combines it with reconstruction and contrast targets, which prompts the model to learn stable structural representations and suppress noise sensitivity, thereby improving cross-scene generalization; this effect is brought about by the joint optimization of the neck network noise module + reconstruction loss and contrast loss.

[0129] (5) More precise positioning and more prominent abnormal semantics: This invention reconstructs and utilizes shallow to medium-level features to form residuals. The residuals directly characterize the deviation of local areas from the "normal structure reconstruction", making the defective areas more prominent in the residual space, which is conducive to pixel-level positioning and subsequent discrimination. This effect is brought about by the multi-layer feature reconstruction strategy + image block-level residual representation + reconstruction loss of local MSE and global cosine combination.

[0130] It is worth noting that the core of this invention lies in proposing a defect detection method based on residual contrastive learning. By introducing encoding extraction and decoding reconstruction, combined with residual space construction and contrastive learning mechanisms, it solves the shortcomings of traditional defect detection schemes in areas such as semi-open set identification, false negative and missed detection control, new defect discovery, and model generalization ability. However, in order to expand the scope of patent protection and prevent others from circumventing the technical solution of this invention through similar technical means, some alternative solutions are provided below. These alternative solutions include substitutions for some structures, devices, or method steps in the technical solution of this invention, as well as other implementations of the complete technical solution, all of which can achieve the purpose of this invention. Examples:

[0131] (1) "Weighted contrastive loss" can be specified into a variety of mature contrastive loss functions, such as TripletLoss, N-Pair Loss, Proxy-Anchor Loss or Supervised Contrastive Loss. Although these functions differ in their weighting strategies for positive and negative samples, their core purpose is to optimize the relative distance between features and label vectors, achieving the same or similar technical effects.

[0132] (2) Substitution of residual contrastive learning objectives and semi-open set decision method

[0133] This invention employs residual projection followed by comparison and optimization with class centers, and constructs an energy function based on logits for OOD (Out of Detail) discrimination. Alternative methods include, but are not limited to:

[0134] The contrastive learning loss can be replaced by InfoNCE with supervised contrastive loss, Triplet class loss, Proxy / prototype class loss, etc.; the similarity metric can be replaced by cosine similarity with Euclidean distance, Mahalanobis distance, etc.

[0135] The category center can be replaced by "learnable parameter center" or "batch / global prototype center (mean of the same category)" or "memory prototype".

[0136] The open set decision can be replaced by "maximum class score threshold", "distance to nearest class center threshold", "tail threshold based on statistical modeling", etc., in addition to the "energy function threshold". The threshold determination method can be replaced by "ID energy quantile under target FPR" or "validation set calibration quantile / adaptive temperature calibration". All of the above substitutions can achieve rejection or marking of unknown defect regions as OOD.

[0137] In summary, the inventive points of this invention include:

[0138] 1. Core Invention Points

[0139] The core of this invention lies in constructing a unified residual contrastive learning framework to solve the coexistence challenge of high-precision positioning, known category classification, and unknown category discovery in industrial defect detection. Its core inventive points can be specifically broken down as follows:

[0140] A semi-supervised extension mechanism based on category prototypes extends the unsupervised residual framework to semi-supervised, semi-open set detection. Building upon the unsupervised anomaly detection framework of "feature reconstruction—residual representation," a small number of labeled samples are introduced to construct / learn category prototypes (including normal prototypes and known defect category prototypes). Alignment / contrast constraints between "residuals and prototypes" are established in the residual embedding space: labeled samples are used to explicitly bring their residual representations closer to their corresponding category prototypes and further away from other category prototypes; unlabeled samples participate in training through clustering constraints with normal prototypes or consistency constraints based on prototype similarity. This expands unsupervised detection from simply distinguishing between normal and abnormal to a semi-supervised, semi-open set detection method that simultaneously possesses the ability to "identify known defect categories (ID) + reject unknown defects (OOD)."

[0141] Residual Projection and Prototype Contrastive Learning: Constructing a compact intra-class and separable inter-class residual discrimination space. Image block-level residuals are mapped to the contrast embedding space via a projection head. Using class prototypes as contrast "anchors / surrogates," a supervised contrastive learning objective is constructed, enabling residuals of the same class to converge to their corresponding prototypes in the embedding space and separating residuals of different classes from each other. Simultaneously, prototypes can be implemented in the form of learnable parameters or moving mean prototypes (memory prototypes) to enhance stability and transferability under small sample annotation conditions, thereby improving the ability to distinguish known defect categories.

[0142] Semi-open set decision and threshold calibration based on prototype energy function: Unifying the rejection of unknown defects and the localization output. A block-level energy function is constructed using the distance (equivalent to logits) of the residual embedding into each category prototype. An energy threshold is used to complete the ID / OOD decision: if the energy is higher than the threshold, it is judged as an unknown defect (OOD); otherwise, it is classified into the best-matching known category (including the normal class). The threshold can be calibrated based on the high quantile of the energy distribution of the in-domain sample (ID) under a preset false alarm rate (FPR), thus achieving a unified output of pixel-level localization, known category classification, and unknown defect detection within the same framework.

[0143] 2. Secondary Invention Points

[0144] To ensure the effectiveness and robustness of the core inventive points in complex industrial environments, this invention also includes the following secondary inventive points:

[0145] Training optimization strategy for imbalanced data: To address the common problem of uneven distribution of industrial defect samples, an imbalanced weighting mechanism is introduced on the basis of contrastive loss. This mechanism assigns higher loss weights to defect categories with fewer samples, forcing the model to pay equal attention to all categories and preventing the model optimization from being dominated by the majority class samples.

[0146] Category Prototype EMA Update Mechanism: In a semi-supervised setting, the category prototype carries the information transmission from "a small number of annotations to a stable category representation." The category prototype is a learnable parameter, initialized by the mean of the residual embeddings of the labeled samples, and iteratively updated using exponential moving average (EMA) / momentum update. The new category center = old category center + current category mean. Simultaneously, norm constraints (normalizing cluster centers) and center separation regularization (adding loss to the distance between centers) are applied to the prototype to maintain the separability of each category prototype in the embedding space. Through this mechanism, the unsupervised residual framework can stably obtain the discrimination benchmark of "normal + known defect categories" with only a small number of annotations, thus supporting subsequent semi-open set decision-making.

[0147] Feature enhancement and robustness module: A feature noise-injection module is introduced into the neck network as a powerful and effective regularization method. By actively injecting noise, the encoder is forced to learn more robust feature representations that are invariant to disturbances, thereby improving the model's generalization ability in real noisy industrial environments.

[0148] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute any of the above-described defect detection methods.

[0149] On the other hand, the present invention also provides a computer storage medium storing executable program code; the executable program code is used to execute any of the above-described defect detection methods.

[0150] For example, the program code can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the program code in the terminal device.

[0151] The terminal device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the terminal device may also include input / output devices, network access devices, buses, etc.

[0152] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0153] The memory can be an internal storage unit of the terminal device, such as a hard drive or RAM. The memory can also be an external storage device of the terminal device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units of the terminal device. The memory is used to store the program code and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output.

[0154] The aforementioned terminal equipment and computer storage media are created based on the aforementioned defect detection method. Their technical functions and beneficial effects will not be elaborated here. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A defect detection method based on residual contrastive learning, characterized in that, include: Acquire the image to be identified and input it into the defect identification model; The defect identification model includes: The encoder feature extraction module is used to extract multi-level semantic features from the input image as encoded features. The decoder feature reconstruction module is used to reconstruct the last layer feature map of the encoded features layer by layer, generating reconstructed features that are aligned with the last layer feature map at the same scale. The residual calculation and projection module is used to calculate the residual between the encoded features and the reconstructed features, generate image block-level residual features that characterize image anomaly information, and project them onto a preset contrast learning space to obtain image block-level residual embedding features. The defect identification module guides the features of normal samples and known defect categories to fit towards their respective centers in the residual space through residual contrastive learning; and outputs the defect identification result by combining image patch-level residual embedding features and the fitting result; including: defining a learnable reconstruction center, using feature reconstruction loss to make the residual features of normal samples approach the reconstruction center; and based on the number of known defect categories... ,definition Each learnable category center corresponds to Given a known defect center and a positive sample center, the image uses image block-level residual contrastive learning loss to obtain a cluster distribution in the contrastive space that is compact within classes and has large inter-class spacing. Based on the residual embedding features and the cluster distribution, the in-domain category and out-of-domain label are output. The in-domain category is either the normal category or a known defect category; the out-of-domain label is for unknown defects.

2. The defect detection method according to claim 1, characterized in that, The processing steps of the residual calculation and projection module include: In the multi-level semantic features of the coding features, the first few layers and the last few layers are fused respectively to obtain the first coding fusion feature and the second coding fusion feature; The first coding fusion feature, the second coding fusion feature, and the reconstructed feature are divided into several image blocks in the spatial dimension. Each image block corresponds to a local feature vector to calculate the image block-level residual features. The image block-level residual features are projected to obtain the image block-level residual embedding features.

3. The defect detection method according to claim 2, characterized in that, The decoder feature reconstruction module employs an 8-layer Transformer decoder; The residual calculation and projection module divides the features of the first 8 layers of the encoder into layers 1-4 and layers 5-8, and performs fusion of the two parts. The fused encoded features and reconstructed features are divided into several image patches in the spatial dimension to obtain the first... The residual features of each image patch are represented as follows: ； in, For the first Residual features of each image patch; For encoder number The first layer Encoded features of an image patch; For encoder number The first layer Reconstruction features of an image patch Number of image patches; By introducing a multilayer perceptron projection head, the image block-level residual features are mapped from the original feature space to the contrastive learning embedding space to obtain the image block-level residual embedding features.

4. The defect detection method according to claim 1, characterized in that, The steps for outputting in-domain categories and out-of-domain labels based on residual embedding features and cluster distribution include: Calculate the distance from each image block-level residual feature to the center of each class; Calculate the energy value corresponding to each image patch; Determine whether the energy value corresponding to each image patch is greater than the set energy threshold. If not, the identification result is an in-domain category, and the specific category is determined based on the distance. If yes, the identification result is an out-of-domain label. The energy threshold is determined by the high quantile of the ID energy distribution under the target false alarm rate (FPR).

5. The defect detection method according to any one of claims 1 to 4, characterized in that, The feature reconstruction loss is determined by a weighted sum of the local mean square error loss and the global cosine similarity loss, and is expressed as: ； ； ； in, For feature reconstruction loss, For local mean square error loss, For global cosine similarity loss, The weighting coefficients for feature reconstruction loss, For batch size, The number of image patches, For the first The first image patch Batch residual characteristics To restructure the center, For the first The residual features of each batch are flattened vectors along the spatial dimension.

6. The defect detection method according to claim 5, characterized in that, The image block-level residual contrastive learning loss is represented as: ； in, For image block-level residual contrastive learning loss, The number of image patches, For the first Residual embedding features of each image patch Corresponding to positive samples, The corresponding set is the negative sample set. The second temperature coefficient; It is a similarity function; To address the issue of uneven defect sample distribution, an imbalanced weighting mechanism is introduced on top of the image block-level residual contrastive learning loss. This mechanism assigns higher weights to defect categories with fewer samples, multiplying the image block-level residual contrastive learning loss by the frequency of each defect category.

7. The defect detection method according to claim 6, characterized in that, To simultaneously optimize reconstruction localization and residual spatial separability, a joint loss is constructed, which is determined by weighting the feature reconstruction loss and the image block-level residual contrastive learning loss, and is expressed as: ； in, For joint losses, For feature reconstruction loss, For image block-level residual contrastive learning loss, This represents the joint loss weighting coefficient.

8. A terminal device, characterized in that, It includes a memory and a processor; the memory stores program code that can be executed by the processor; the program code is used to execute the defect detection method according to any one of claims 1-7.

9. A computer storage medium, characterized in that, It stores executable program code; the executable program code is used to execute the defect detection method according to any one of claims 1-7.

Images