A defect detection method based on deep contrastive learning

Abstract

The present invention discloses a defect detection method based on deep contrast learning, which relates to the field of computer vision technology, including obtaining the original image data of industrial workpieces, and performing normalization, denoising, cropping and resizing to form standardized image data; inputting the standardized image data into a deep contrast learning model based on Transformer, and outputting the global feature vector of the workpiece image; inputting the optimized positive and negative sample pairs into a meta-learning unit, and performing rapid adaptive training on the deep contrast learning model through a meta-learning mechanism; applying the trained contrast learning model to real-time defect detection, performing defect judgment on the newly input workpiece image, and outputting the defect detection result of the workpiece. The present invention improves the accuracy of defect detection, while greatly reducing the training time and data requirements, and significantly improves the limitations of the prior art in a small sample environment.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a defect detection method based on deep contrast learning. Background Art

[0002] In the development of industrial automation and intelligent manufacturing, defect detection of industrial workpieces is an important part of ensuring product quality. At present, traditional defect detection methods mainly rely on rule-based image processing technology or manual detection. These methods perform well in simple scenarios, but with the increase in the complexity of workpieces and the improvement of detection accuracy and efficiency requirements, their limitations gradually become apparent. Especially in small sample environments, traditional methods have limited ability to extract workpiece features and are difficult to accurately identify subtle defects on the surface of workpieces, resulting in low detection accuracy. In recent years, the application of deep learning technology in the field of image processing has made significant progress, especially convolutional neural networks (CNN) and other deep neural networks have shown strong feature extraction capabilities in target detection tasks. However, these technologies usually rely on large-scale annotated data. For small sample scenarios, their generalization ability is poor and they are prone to overfitting problems, which in turn affects the accuracy and robustness of detection.

[0003] Existing deep learning methods face a significant drop in accuracy when performing industrial workpiece defect detection in a small sample environment. Since it is often difficult to obtain a large amount of industrial image data with defect annotations, it is difficult for deep learning models to be effectively trained with a small amount of data, resulting in unsatisfactory performance of the models in practical applications. In addition, when faced with new types of workpieces or new types of defects, existing models have poor adaptability and cannot be quickly adjusted to cope with new detection tasks. Therefore, in a small sample environment, how to improve the accuracy of defect detection has become a key issue that needs to be urgently addressed in the current industrial field. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a defect detection method based on deep contrast learning to solve the problem of low accuracy of industrial workpiece defect detection in a small sample environment.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a defect detection method based on deep contrast learning, which includes obtaining original image data of an industrial workpiece, and performing normalization, denoising, cropping and resizing to form standardized image data;

[0008] Input the standardized image data into the Transformer-based deep contrast learning model to output the global feature vector of the artifact image;

[0009] The global feature vector is input into the positive and negative sample pair generation unit of contrastive learning to generate positive sample pairs and negative sample pairs;

[0010] The feature distribution of positive and negative sample pairs is optimized through the loss function of deep contrastive learning to obtain optimized positive and negative sample pairs;

[0011] The optimized positive and negative sample pairs are input into the meta-learning unit, and the deep contrastive learning model is quickly adapted and trained through the meta-learning mechanism;

[0012] The trained contrastive learning model is applied to real-time defect detection, defect judgment is performed on the newly input workpiece image, and the defect detection result of the workpiece is output.

[0013] As a preferred solution of the defect detection method based on deep contrast learning described in the present invention, wherein: the original image data of the industrial workpiece is obtained, and the specific steps are:

[0014] Collect high-definition workpiece images through industrial cameras, use OpenCV to read the collected image files, and load them into memory for processing;

[0015] Check whether the read image is in RGB format. If the image is not in RGB format, convert it to RGB format.

[0016] After ensuring that the image is in RGB format, the image size is further checked, and finally the inspected and standardized raw image data of the industrial workpiece is output.

[0017] As a preferred solution of the defect detection method based on deep contrast learning of the present invention, wherein: the forming of standardized image data comprises the following specific steps:

[0018] Normalizing the pixel values ​​of the raw image data of the inspected and normalized industrial workpiece;

[0019] Based on the normalized image, denoising is performed through bilateral filtering;

[0020] The denoised image is cropped to the key part containing the artifact, while keeping the size of the cropped image consistent to form standardized image data.

[0021] As a preferred solution of the defect detection method based on deep contrast learning of the present invention, the global feature vector of the output workpiece image is specifically comprised of the following steps:

[0022] Convert the normalized image data into patches for input to the Transformer-based deep contrastive learning model.

[0023] Divide the image into non-overlapping patches, each of size P×P;

[0024] Flatten each patch to convert it into a one-dimensional vector, and map each patch to a fixed dimension D through linear projection;

[0025] Let N be the number of patches, then the patch expression is:

[0026]

[0027] Among them, H represents the height of the image and W represents the width;

[0028] After linear projection, the combination of all patches is represented as an input feature matrix X∈R N×D ;

[0029] Through the linear transformation of the input feature matrix X, three sets of matrices, namely the query matrix Q, the key matrix K and the value matrix V, are generated;

[0030] The attention weight matrix A is calculated through the interdependence between each patch, and the expression is:

[0031]

[0032] Among them, D k Denotes the dimensions of the query vector and key vector, K T represents the transpose of the key matrix K;

[0033] Based on the attention weight A and the value matrix V, calculate the output feature Z = AV;

[0034] The input patch is concatenated and linearly transformed through the outputs of multiple independent attention heads to generate a global feature vector, which is expressed as:

[0035]

[0036] Among them, F gl is the global feature vector of the output, H represents the number of attention heads in the multi-head self-attention mechanism, represents the normalization factor, h represents the index of the current attention head, i represents the index of the patch into which the input image is divided, and A h (X i ) represents the attention weight of the i-th patch extracted from the attention matrix of the h-th head, W hrepresents the linear transformation matrix of the h-th attention head, X i The i-th patch feature vector of the input.

[0037] As a preferred solution of the defect detection method based on deep contrast learning described in the present invention, the specific steps of generating positive sample pairs and negative sample pairs are as follows:

[0038] For the global feature vector F generated from the multi-head self-attention mechanism gl Normalize and get the global eigenvector

[0039] For each original image I of the workpiece, multiple images with different perspectives are generated through data augmentation;

[0040] Based on each enhanced image, the global feature vector of the enhanced image is extracted through the Transformer model

[0041] Pair the global feature vectors of the same workpiece from different perspectives to generate positive sample pairs;

[0042] Randomly select images I of different artifacts and calculate the global feature vector for each image

[0043] The global feature vectors from different artifacts are paired to generate negative sample pairs.

[0044] As a preferred solution of the defect detection method based on deep contrast learning described in the present invention, the optimized positive and negative sample pairs are obtained by specifically performing the following steps:

[0045] For each pair of positive and negative samples, the cosine similarity is calculated, and the InfoNCE loss function is applied to optimize the neural network model;

[0046] Use the Adam optimizer to update the parameters of the neural network model based on the InfoNCE loss;

[0047] Calculate the loss L and backpropagate the loss Update the neural network model parameters until the neural network model reaches a predetermined training round;

[0048] Map the input samples to the feature space of the trained neural network model to further optimize the positive and negative sample pairs.

[0049] As a preferred solution of the defect detection method based on deep contrast learning of the present invention, wherein: the deep contrast learning model is rapidly adapted and trained by the meta-learning mechanism, and the specific steps are as follows:

[0050] Divide the optimized positive and negative sample pairs into multiple meta-tasks T i , each task contains positive and negative sample pairs of artifacts;

[0051] On the support set of each task, the temperature parameters are adjusted using an adaptive temperature regulation mechanism;

[0052] After adjusting the parameters in the support set, the weight w is dynamically assigned to each task through dynamic task weights. i The expression is:

[0053] w i =(1+exp(-γ·(L i (Q i ,θ')-L avg ))) -1 ;

[0054] Among them, -γ represents the hyperparameter that adjusts the sensitivity of task difficulty to weight, L avg represents the average loss on the query set of all tasks, and θ' represents the model's performance on the meta-task T i The parameters after a gradient update on the support set, L i (Q i ,θ') represents the meta-task T i In the query set Q i losses;

[0055] Combined with the initial update of the support set and the query set loss, the dynamic weights w of each task are i Perform global parameter updates;

[0056] Through multiple rounds of task iterations, the deep contrastive learning model is gradually optimized.

[0057] As a preferred solution of the defect detection method based on deep contrast learning of the present invention, the specific steps of outputting the defect detection result of the workpiece are as follows:

[0058] Convert the input workpiece image into a grayscale image and perform standardization;

[0059] For the grayscale image after standardization, the Canny edge detection algorithm is used to extract the workpiece contour and generate the region of interest;

[0060] The extracted workpiece region of interest is input into the Transformer encoder, and the feature embedding vector of the input workpiece is extracted through the encoder;

[0061] Call the pre-optimized positive sample feature embedding vector Z from the sample library pos and negative sample feature embedding vector Z neg ;

[0062] Calculate the input workpiece feature z input and the positive sample feature embedding vector Z pos The cosine similarity sim pos ;

[0063] Calculate the input workpiece feature z input and the negative sample feature embedding vector Z neg The cosine similarity sim neg ;

[0064] Based on the calculated similarity, a preset threshold τ is used to determine whether the input workpiece has defects;

[0065] If sim pos <τ and sim neg >τ, the workpiece is judged to have defects, otherwise, it is judged to be free of defects.

[0066] In a second aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the defect detection method based on deep contrast learning as described in the first aspect of the present invention is implemented.

[0067] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the defect detection method based on deep contrast learning as described in the first aspect of the present invention is implemented.

[0068] The beneficial effects of the present invention are as follows: the present invention significantly improves the accuracy of industrial workpiece defect detection in a small sample environment by introducing a mechanism of deep contrastive learning. Secondly, by generating positive and negative sample pairs through contrastive learning, the model's ability to distinguish between different workpiece features is enhanced, reducing dependence on large-scale labeled data. At the same time, combined with the Transformer model's powerful global feature extraction capabilities and meta-learning mechanism, it is possible to quickly optimize model parameters in the case of a small number of samples, making it more adaptable when facing new types of workpieces or defects. This design not only improves the accuracy of defect detection, but also greatly reduces training time and data requirements, significantly improving the limitations of existing technologies in small sample environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0070] Figure 1This is a flow chart of the defect detection method based on deep contrast learning in Example 1.

[0071] Figure 2 This is a flow chart for obtaining original image data of an industrial workpiece in Example 1. DETAILED DESCRIPTION

[0072] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0073] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0074] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0075] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a defect detection method based on deep contrast learning, comprising the following steps:

[0076] S1. Obtain the original image data of the industrial workpiece, and perform normalization, denoising, cropping and size adjustment to form standardized image data.

[0077] Furthermore, high-definition workpiece images are collected through industrial cameras, and the collected image files are read using OpenCV and loaded into memory for processing;

[0078] It should be noted that when capturing high-definition workpiece images, appropriate exposure, white balance and resolution (such as 1920x1080 pixels) should be ensured, and the workpiece images should be stored in JPEG or PNG format.

[0079] Preferably, OpenCV is a widely used library for computer vision that can easily read, process and display images.

[0080] Check whether the read image is in RGB format (i.e., has 3 color channels). If the image is not in RGB format (such as a grayscale image), convert it to RGB format to ensure a unified color space for subsequent processing. The specific implementation steps are as follows:

[0081] After using OpenCV to read an image, the image will be represented as a multidimensional array with a shape of (height, width, channels). Channels represents the number of channels of the image. The number of channels of an RGB format image is 3, while the number of channels of a grayscale image is 1.

[0082] By checking the shape of the image, if the number of channels is 3, it means that the image is in RGB format; if the number of channels is 1, it means that the image is a grayscale image and needs to be converted to RGB format.

[0083] If the image is grayscale (i.e. the number of channels is 1), we can convert it to RGB format using OpenCV's cv2.cvtColor() function.

[0084] After ensuring that the image is in RGB format, the image dimensions, such as the width, height, and number of channels, are further checked to ensure that they meet expectations. Finally, the inspected and standardized raw image data of the industrial workpiece is output.

[0085] It should be noted that the specific process of checking the size of an image is to read the size information of the image, check the width, height and number of channels of the image as needed, and process the image size that does not meet the requirements;

[0086] For reading the size information of the image, it means that the image is usually represented as a three-dimensional array (heightxwidth x channels) in the memory, where: height: the height of the image (number of pixels); width: the width of the image (number of pixels); channels: the number of channels of the image (3 for RGB images and 1 for grayscale images).

[0087] Checking the width, height, and number of channels of the image as needed refers to checking whether the resolution of the image meets expectations, for example, whether it is the size required by the model (such as 224x224), and whether the number of channels is 3 (RGB).

[0088] For processing image sizes that do not meet the requirements, it means that if the image size does not meet the predetermined requirements, you can consider resizing it in subsequent steps (such as using cv2.resize()).

[0089] Furthermore, the pixel values ​​of the original image data of the inspected and normalized industrial workpiece are normalized;

[0090] Based on the normalized image, denoising is performed through bilateral filtering;

[0091] Crop the denoised image to the key part containing the artifact to reduce noise and interference and enhance the model's focus on the artifact features;

[0092] It should be noted that "the key part containing the workpiece" refers to the area in the image that actually contains the workpiece, rather than the irrelevant background or noise in the image. Specifically, it refers to: The main body of the workpiece: The part of the image related to the workpiece, usually the main area that needs to be analyzed, detected or classified. These areas contain the structural features of the workpiece or parts that may have defects; Remove irrelevant background: The workpiece image may contain a lot of unnecessary background (such as the environment and equipment background when the industrial camera is shooting). Through cropping, irrelevant background parts can be removed to ensure that only the workpiece itself is retained to reduce interference with the model; Focus on key features: The cropped image is more focused on the key feature parts of the workpiece, which are the content that the model needs to focus on. For example, the edges of the workpiece, surface details, and areas where defects may exist.

[0093] For example, the artifact is usually located in the center of the image, and the central area of ​​the image can be cropped out based on prior knowledge (for example, retaining 75% of the central part of the image); if the location of the artifact is uncertain, the target detection algorithm (such as YOLO or Faster R-CNN) can be combined to automatically locate and crop the key parts of the artifact.

[0094] Adjust the size of the cropped images to a consistent size, such as a fixed size (for example, 224x224). This step ensures that the image size input to the model is consistent to avoid affecting the model performance due to inconsistent sizes.

[0095] After normalization, denoising, cropping and resizing, the image will form standardized image data.

[0096] S2. Input the standardized image data into the Transformer-based deep contrast learning model and output the global feature vector of the workpiece image.

[0097] Furthermore, the normalized image data is converted into a patch form for input to the Transformer-based deep contrastive learning model;

[0098] Divide the image into non-overlapping patches, each of size P×P;

[0099] It should be noted that patch refers to a small area or block extracted from an image; patches is the plural form of patch, which means all small image blocks obtained after segmenting the entire image.

[0100] Flatten each patch to convert it into a one-dimensional vector, and map each patch to a fixed dimension D through linear projection to meet the input requirements of the Transformer;

[0101] Let N be the number of patches, then the patch expression is:

[0102]

[0103] Among them, H represents the height of the image and W represents the width;

[0104] After linear projection, the combination of all patches is represented as an input feature matrix X∈R N×D ;

[0105] The core goal of the self-attention mechanism is to capture global dependencies by calculating the mutual relationships (i.e., "attention") between elements in the input sequence. In image processing tasks, the input image is divided into multiple small blocks (patches), each patch is flattened into a one-dimensional vector, and then the self-attention mechanism is introduced through the following steps:

[0106] Through the linear transformation of the input feature matrix X, three sets of matrices, namely the query matrix Q, the key matrix K and the value matrix V, are generated;

[0107] The query matrix Q is expressed as: Q = XW Q ;

[0108] The bond matrix K is expressed as: K = XW K ;

[0109] The value matrix V is expressed as: V = XW V ;

[0110] Among them, W Q Indicates mapping the input data to the Query vector, W K Indicates mapping the input data to the Key vector, W V Indicates mapping input data to Value vector;

[0111] It should be noted that in the self-attention mechanism, the Query vector is used to calculate the similarity with the Key vectors of other elements, so as to determine the relationship between the current input element (such as a word or an image patch) and other elements; the Key vector is one of the representations of each input element, and it performs a dot product operation with the Query vector to measure the similarity. The Key is similar to the "index" or "key" in the database, which helps the model find the input element most relevant to the Query; the Value vector is the content representation of each input element, which will eventually be weighted and summed according to the similarity between the Quer and the Key to generate the final output. The Value vector contains the actual information of the input, and is used together with the Query and Key to calculate the final attention weighted result.

[0112] Where D is the fixed dimension of the input patch, D k is the dimension after mapping (usually the lower dimension).

[0113] It should also be noted that W Q ,W K ,W V Specifically, it refers to the learnable parameters in the model. The so-called learnable weight matrix refers to these matrices W Q ,W K ,W V The elements of are not fixed constants, but parameters that are constantly adjusted during the model training process. When the model is trained, these weight matrices are updated through forward and backward propagation of the input data, using optimization algorithms such as gradient descent, so that the model can better adapt to the characteristics of the input data.

[0114] The attention weight matrix A is calculated through the interdependence between each patch, and the expression is:

[0115]

[0116] Among them, D k Denotes the dimensions of the query vector and key vector, K T represents the transpose of the key matrix K;

[0117] It should be noted that by dividing This prevents the dot product from becoming too large, especially in D k In the case of large dimensions; if not scaled, the dot product value may become too large, causing the softmax gradient to become very small, thus affecting the training effect of the model.

[0118] Based on the attention weight A and the value matrix V, calculate the output feature Z = AV;

[0119] The input patch is concatenated and linearly transformed through the outputs of multiple independent attention heads to generate a global feature vector, which is expressed as:

[0120]

[0121] Among them, F gl is the global feature vector of the output, H represents the number of attention heads in the multi-head self-attention mechanism, represents the normalization factor, which is used to average the output results of multiple heads. h represents the index of the current attention head. The output W of each head h is h XdX both participate in the generation of global features, i represents the index of the patch into which the input image is divided, A h (X i) represents the attention weight of the i-th patch extracted from the attention matrix of the h-th head, W h represents the linear transformation matrix of the h-th attention head, X i The i-th patch feature vector of the input;

[0122] In the Transformer architecture, the multi-head self-attention mechanism calculates multiple attention heads in parallel, and each head extracts different features independently. By dividing by H, the outputs of multiple heads can be normalized to ensure that the final global feature is the average of the outputs of each head.

[0123] S3. Input the global feature vector into the positive and negative sample pair generation unit of contrastive learning to generate positive sample pairs and negative sample pairs.

[0124] Furthermore, the global feature vector F generated from the multi-head self-attention mechanism gl Normalize and get the global eigenvector The expression is:

[0125]

[0126] Among them, ||F gl | is the modulus of the global eigenvector;

[0127] For each original image I of the workpiece, multiple images with different perspectives are generated through data enhancement (such as rotation, cropping, flipping, color jittering, etc.);

[0128] Based on each enhanced image, the global feature vector of the enhanced image is extracted through the Transformer model The global feature vectors of the same workpiece from different perspectives are paired to generate positive sample pairs;

[0129] Example: If workpiece B generates images of 3 different perspectives after data enhancement Then the generated positive sample pair is:

[0130]

[0131] Randomly select images I of different artifacts and calculate the global feature vector for each image

[0132] Pair the global feature vectors from different artifacts to generate negative sample pairs;

[0133] Example: If the global feature vectors of workpiece B and workpiece C are and Then the generated negative sample pair is:

[0134] It should be noted that in contrastive learning, positive pairs should have the same global features (because they come from the same artifact), but their specific image representations are different; negative pairs come from images of different artifacts. To ensure the effectiveness of negative pairs, images from different artifacts should have obvious feature differences so that the model can learn to distinguish different artifacts.

[0135] S4. Optimize the feature distribution of positive and negative sample pairs through the loss function of deep contrastive learning to obtain optimized positive and negative sample pairs.

[0136] Furthermore, for each pair of positive and negative samples, the cosine similarity is calculated, and the InfoNCE loss function is applied to optimize the neural network model;

[0137] It should be noted that cosine similarity is used to measure the angle between two feature vectors in a high-dimensional space. The closer it is to 1, the more similar they are, and the closer it is to 0 or a negative value, the less similar they are.

[0138] The formula for calculating cosine similarity is:

[0139]

[0140] Among them, F i and F j Represents the feature vector of positive sample pair or negative sample pair, F i ×F j represents the product of two eigenvectors, ||F i | and ||F j || are the norms of the eigenvectors (L2 norms);

[0141] It should also be noted that in contrastive learning, the InfoNCE loss function is one of the most commonly used loss functions, which optimizes the feature representation of the model by maximizing the similarity of positive sample pairs while minimizing the similarity of negative sample pairs.

[0142] The loss formula is:

[0143]

[0144] Among them, F i is the feature vector of a query sample, F p represents the positive sample pair feature vector of the query sample, F j represents the feature vector of negative sample pairs (including all negative examples of the query sample), τ represents the temperature parameter, which is used to control the scaling of similarity and adjust the sensitivity of the model to different similarities (a smaller τ makes the model more sensitive to the differences between samples);

[0145] Use the Adam optimizer to update the parameters of the neural network model based on the InfoNCE loss;

[0146] It should be noted that the Adam optimizer is an adaptive learning rate optimization algorithm that can automatically adjust the learning rate of each parameter; SGD (stochastic gradient descent) is a commonly used optimization algorithm that directly updates model parameters based on the gradient of the loss function.

[0147] The formula for using the optimizer is (taking SGD as an example):

[0148]

[0149] Among them, θ represents the parameters of the model, η is the learning rate (learning rate), is the loss function The gradient of the model parameters θ;

[0150] Calculate the loss L and backpropagate the loss Update the neural network model parameters until the neural network model reaches the predetermined training round.

[0151] Back Propagation The expression is:

[0152]

[0153] Among them, the gradient is calculated by back propagation Guide the update direction of model parameters so that the model can gradually optimize the feature distribution of positive and negative sample pairs.

[0154] The input samples are mapped to the feature space of the trained neural network model, and the positive and negative sample pairs are further optimized so that the feature representations of the positive sample pairs are closer and the feature representations of the negative sample pairs are farther apart.

[0155] S5. Input the optimized positive and negative sample pairs into the meta-learning unit, and quickly adapt the deep contrastive learning model through the meta-learning mechanism.

[0156] Furthermore, the optimized positive and negative sample pairs are divided into multiple meta-tasks T i , each task contains positive and negative sample pairs of artifacts;

[0157] On the support set of each task, the temperature parameters are adjusted using an adaptive temperature regulation mechanism;

[0158] After adjusting the parameters in the support set, the weight w is dynamically assigned to each task through dynamic task weights. i The expression is:

[0159] w i =(1+exp(-γ·(Li (Q i ,θ')-L avg ))) -1 ;

[0160] Among them, -γ represents the hyperparameter that adjusts the sensitivity of task difficulty to weight, L avg represents the average loss on the query set of all tasks, and θ' represents the model's performance on the meta-task T i The parameters after a gradient update on the support set, L i (Q i ,θ') represents the meta-task T i In the query set Q i losses;

[0161] Combined with the initial update of the support set and the query set loss, the dynamic weights w of each task are i Perform global parameter updates;

[0162] Through multiple rounds of task iterations, the deep contrastive learning model is gradually optimized to make it more adaptable between different tasks.

[0163] It should be noted that the dynamic allocation weight w i The specific derivation process is as follows:

[0164] First, in the meta-learning process, each meta-task T needs to be evaluated i The difficulty is calculated by query set loss L i (Q i ,θ') to measure the task T i The model performance, specifically refers to the model on the query set Q i The loss is calculated using the updated parameters θ' on the support set.

[0165] If L i (Q i ,θ') is large, which means that the model performs poorly on the task and the task may be difficult.

[0166] If L i (Q i ,θ') is small, which means that the model performs well on the task and the task is relatively easy.

[0167] At the same time, in order to make the difficulty of different tasks comparable, the average query set loss L for all tasks is introduced avg , used as a reference benchmark.

[0168] Calculate the single task loss L i (Q i ,θ') and the average loss of all tasks L avgThe difference between them is used to quantify the difficulty of each task, the task loss ΔL i The expression is:

[0169] ΔL i =L i (Q i ,θ')-L avg ;

[0170] When L i (Q i ,θ')>L avg , that is, ΔL i >0, indicating task T i Harder than average mission.

[0171] When L i (Q i ,θ') <L avg , that is, ΔL i <0, indicating task T i Easier than average task.

[0172] Preferably, considering this requirement, the sigmoid function is an ideal choice. It is a common S-shaped curve function with an output value between (0,1)(0,1), which can smoothly map the input to a value between 0 and 1.

[0173] Through the form of sigmoid function, the loss difference ΔL i Converted to weight w i ;

[0174] The general form of the sigmoid function is:

[0175]

[0176] The loss difference ΔL i Substitute into the sigmoid function:

[0177]

[0178] Among them, -γ represents the hyperparameter that adjusts the sensitivity of task difficulty to weight, which is used to control the sensitivity of the input (i.e., loss difference) to the weight. A larger γ will increase the sensitivity of the weight to the loss difference, and conversely, a smaller γ will make the weight change more smoothly.

[0179] The loss difference ΔL i Substituting the above sigmoid expression, we can finally get the dynamic weight w for each task i expression:

[0180] w i=(1+exp(-γ·(L i (Q i ,θ')-L avg ))) -1 ;

[0181] Among them, when L i (Q i ,θ') is much larger than L avg , that is, when the task is difficult, the exponential term exp(-γ·ΔL i ) is close to 0, w i Close to 1, it means the task is of high importance;

[0182] When L i (Q i ,θ') is much smaller than L avg , that is, when the task is easier, the exponential term exp(-γ·ΔL i ) is larger, w i A value close to 0 indicates that the task is less important.

[0183] It should also be noted that the difference between the query set loss of the task and the global average loss is mapped to a weight w between 0 and 1 through the sigmoid function. i ; Weight w i Dynamically reflects the difficulty of each task, ensuring that the model gives more attention to harder tasks when updating global parameters.

[0184] S6. Apply the trained contrastive learning model to real-time defect detection, perform defect judgment on the newly input workpiece image, and output the defect detection result of the workpiece.

[0185] Furthermore, the input workpiece image is converted into a grayscale image and normalized to make it consistent with the distribution of the training data;

[0186] For the grayscale image after standardization, the Canny edge detection algorithm is used to extract the contour of the workpiece and generate the region of interest to ensure that the input of the model is the valid area of ​​the workpiece and remove irrelevant background information;

[0187] The extracted workpiece region of interest is input into the Transformer encoder, and the encoder extracts the feature embedding vector Z of the input workpiece input , which is a high-dimensional feature representation of the artifact image, used for comparison with the reference sample;

[0188] Call the pre-optimized positive sample (non-defective workpiece) feature embedding vector Z from the sample library pos And the negative sample (with defective artifacts) feature embedding vector Z neg ;

[0189] Calculate the input workpiece feature z input and the positive sample feature embedding vector Z pos The cosine similarity sim pos The expression is:

[0190]

[0191] Calculate the input workpiece feature z input and the negative sample feature embedding vector Z neg The cosine similarity sim neg The expression is:

[0192]

[0193] Based on the calculated similarity, a preset threshold τ is used to determine whether the input workpiece has defects;

[0194] If sim pos <τ and sim neg >τ, the workpiece is judged to have defects;

[0195] Otherwise, it is judged as no defect;

[0196] It should be noted that the setting of the threshold τ is usually determined according to the specific application scenario and data distribution through the following method:

[0197] In some fixed scenarios, a suitable threshold τ can be manually selected by observing the similarity distribution of training data and test data. Usually, the difference between the similarities of defect-free samples and defective samples is observed by visualizing the distribution graph of similarity (such as a histogram), and a threshold that can maximize the distinction between positive and negative samples is selected;

[0198] Secondly, use the validation set to tune parameters: input samples in the validation set that are known to be defective into the model and observe the detection results under different thresholds τ.

[0199] Then, we can calculate the accuracy, recall, F1 value and other indicators under different thresholds, and select the threshold τ that makes these indicators reach the optimal value.

[0200] Accuracy: The ratio of samples correctly classified by the model to the total number of samples.

[0201] Recall: The proportion of defective workpieces that are correctly detected as defective.

[0202] F1 value: The harmonic mean of accuracy and recall, which comprehensively considers the precision and recall capabilities of the model.

[0203] It should also be noted that when selecting the threshold τ, the following factors are usually considered:

[0204] Detection accuracy: Ensure that the threshold can maximize the distinction between positive and negative samples and reduce false positives and false negatives.

[0205] Environmental changes: If the types of artifacts vary or the detection environment changes significantly, it is recommended to use a dynamic threshold adjustment mechanism.

[0206] Data distribution: According to the similarity distribution characteristics of the artifact image, a threshold is selected that separates the positive samples from the negative samples.

[0207] Threshold selection in the example scenario:

[0208] Suppose in a specific defect detection task, after tuning the validation set, the following similarity distribution is observed:

[0209] Similarity of positive samples sim pos Most are between 0.8 and 1.0.

[0210] Similarity of negative samples sim neg Most are between 0.0 and 0.4.

[0211] According to this distribution, a threshold τ = 0.7 can be selected. At this time:

[0212] If sim pos <0.7, it is judged that the similarity between the input workpiece and the defect-free sample is low and there may be defects;

[0213] If sim neg >0.7, it is judged that the input workpiece has a high similarity with the defective sample and may contain defects.

[0214] This embodiment also provides a computer device, which is suitable for the defect detection method based on deep contrast learning, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the defect detection method based on deep contrast learning proposed in the above embodiment.

[0215] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0216] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the defect detection method based on deep contrast learning proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, disk or optical disk.

[0217] In summary, the present invention significantly improves the accuracy of industrial workpiece defect detection in a small sample environment by introducing the mechanism of deep contrastive learning. Secondly, by generating positive and negative sample pairs through contrastive learning, the model's ability to distinguish between different workpiece features is enhanced, reducing dependence on large-scale labeled data. At the same time, combined with the Transformer model's powerful global feature extraction capabilities and meta-learning mechanism, it can quickly optimize model parameters in the case of a small number of samples, making it more adaptable when facing new types of workpieces or defects. This design not only improves the accuracy of defect detection, but also greatly reduces training time and data requirements, significantly improving the limitations of existing technologies in small sample environments.

[0218] Example 2, referring to Table 1, is the second example of the present invention. In order to further verify the technical solution of the present invention, experimental simulation data of a defect detection method based on deep contrast learning is provided.

[0219] The experimental objects are workpieces on an industrial production line, and the types of workpieces selected include a class of common mechanical parts. In the experiment, 1,000 workpiece images were randomly collected from the production line, and high-definition images were collected using an industrial camera with an image resolution of 1024×1024 pixels. The existing contrast technology uses a defect detection model based on a traditional convolutional neural network (CNN), while the method of the present invention uses a deep contrast learning model based on Transformer.

[0220] Furthermore, in the preprocessing stage of the experiment, the collected workpiece images are first normalized to ensure that each pixel value is in the range of [0, 1]. The bilateral filtering algorithm is then used to denoise the image to filter out high-frequency noise in the image while retaining the boundary details of the workpiece. After denoising, the unnecessary parts of the image are cropped to focus on the key areas of the workpiece, and the image size is uniformly adjusted to 256×256 pixels to form standardized image data.

[0221] After preprocessing, the standardized image is input into the Transformer model. The model first divides the image into 16×16 patches and maps the dimension of each patch to a fixed 128-dimensional vector through linear projection. The global feature vector of the artifact image is generated through a multi-head self-attention mechanism. The global feature vector is used to generate positive and negative sample pairs. Negative sample pairs are generated based on images of different artifacts, and positive sample pairs are generated based on different perspectives of the same artifact.

[0222] Next, the model was trained through deep contrastive learning, the InfoNCE loss function was used to optimize the feature distribution of positive and negative sample pairs, and the Adam optimizer was used to update the model parameters. After 50 rounds of training, the loss value of the model tended to be stable, indicating that the feature extraction ability of the model has reached the best state.

[0223] Finally, the trained model is applied to the real-time defect detection task. 100 new workpiece images are randomly selected for detection to evaluate the accuracy, recall rate and detection speed of defect detection.

[0224] The details are shown in Table 1 below:

[0225] Table 1 Experimental data comparison table

[0226] Parameter name Performance of the method of the present invention Performance of CNN Model Improvement rate (%) Detection accuracy (%) 98.5 92.3 6.7 Recall rate (%) 97.8 89.5 9.3 Detection speed (ms / sheet) 15 35 -57.1 False alarm rate (%) 1.2 5.7 -79.0 Missing rate (%) 2.5 7.8 -68.0

[0227] It can be clearly seen from the above experimental data that the defect detection method based on deep contrastive learning of the present invention is superior to the traditional CNN defect detection method in multiple key indicators. First, in terms of detection accuracy, the method of the present invention reached 98.5%, while the traditional CNN model was only 92.3%, an improvement of 6.7%. This shows that the method of the present invention can more accurately extract global features when processing workpiece images, and reduce the model's sensitivity to noise and non-critical features through contrastive learning, thereby improving the accuracy of detection.

[0228] Further analysis of the recall rate shows that the recall rate of the method of the present invention is 97.8%, which is 9.3% higher than the 89.5% of the CNN model. The improvement in the recall rate shows that the method of the present invention can more efficiently capture the defect characteristics of the workpiece in the defect detection task and reduce the phenomenon of missed detection. The use of the Transformer's multi-head self-attention mechanism can better capture the global information in the image, especially significantly improve the recognition ability of large-area or complex morphological defects.

[0229] In terms of detection speed, the average detection time for each image of the method of the present invention is 15ms, while the detection time of the traditional CNN model is 35ms, which is a reduction of 57.1%. This result shows that although the computational complexity of the Transformer model is relatively high, the detection efficiency has been significantly improved through reasonable model optimization and the application of deep contrast learning methods, especially in industrial real-time detection scenarios, the method of the present invention can better meet the needs of rapid detection.

[0230] At the same time, the two indicators of false alarm rate and missed alarm rate also reflect the advantages of the method of the present invention. Experimental data show that the false alarm rate of the method of the present invention is only 1.2%, while the false alarm rate of the CNN model is as high as 5.7%, and the false alarm rate is reduced by 79.0%, which means that the present method can effectively reduce the misjudgment of normal workpieces and avoid excessive overhaul or unnecessary shutdown due to false alarms. In terms of missed alarm rate, the missed alarm rate of the method of the present invention is 2.5%, which is 68.0% lower than the 7.8% of the CNN model, indicating that the method of the present invention can detect defects in workpieces more stably, avoid defective workpieces from flowing into the subsequent links of the production line, and thus improve the reliability of quality control.

[0231] In summary, through the comparative analysis of experimental data, it is fully proved that the defect detection method based on deep contrastive learning of the present invention has significant advantages in detection accuracy, recall rate, detection speed, false alarm rate and missed alarm rate. At the same time, the global features extracted by the Transformer model combined with contrastive learning can more effectively identify workpiece defects, reduce false alarms and missed alarms, and have higher detection efficiency in real-time detection.

[0232] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A defect detection method based on deep contrast learning, characterized in that: include, Obtain the original image data of industrial workpieces, and perform normalization, denoising, cropping and size adjustment to form standardized image data; Input the standardized image data into the Transformer-based deep contrast learning model to output the global feature vector of the artifact image; The global feature vector is input into the positive and negative sample pair generation unit of contrastive learning to generate positive sample pairs and negative sample pairs; The feature distribution of positive and negative sample pairs is optimized through the loss function of deep contrastive learning to obtain optimized positive and negative sample pairs; The optimized positive and negative sample pairs are input into the meta-learning unit, and the deep contrastive learning model is quickly adapted and trained through the meta-learning mechanism; Apply the trained contrastive learning model to real-time defect detection, perform defect judgment on the newly input workpiece image, and output the defect detection result of the workpiece; The global feature vector of the output workpiece image is specifically comprised of the following steps: Convert the normalized image data into patches for input to the Transformer-based deep contrastive learning model. Divide the image into non-overlapping patches, each of size P×P; Flatten each patch to convert it into a one-dimensional vector, and map each patch to a fixed dimension D through linear projection; Let N be the number of patches, then the patch expression is: Among them, H represents the height of the image and W represents the width; After linear projection, the combination of all patches is represented as an input feature matrix X∈R N×D ; Through the linear transformation of the input feature matrix X, three sets of matrices, namely the query matrix Q, the key matrix K and the value matrix V, are generated; The attention weight matrix A is calculated through the interdependence between each patch, and the expression is: Among them, D k Denotes the dimensions of the query vector and key vector, K T represents the transpose of the key matrix K; Based on the attention weight A and the value matrix V, calculate the output feature Z = AV; The input patch is concatenated and linearly transformed through the outputs of multiple independent attention heads to generate a global feature vector, which is expressed as: Among them, F gl is the global feature vector of the output, H represents the number of attention heads in the multi-head self-attention mechanism, represents the normalization factor, h represents the index of the current attention head, i represents the index of the patch into which the input image is divided, and A h (X i ) represents the attention weight of the i-th patch extracted from the attention matrix of the h-th head, W h represents the linear transformation matrix of the h-th attention head, X i The i-th patch feature vector of the input; The specific steps of generating positive sample pairs and negative sample pairs are as follows: For the global feature vector F generated from the multi-head self-attention mechanism gl Normalize and get the global eigenvector For each original image I of the workpiece, multiple images with different perspectives are generated through data augmentation; Based on each enhanced image, the global feature vector of the enhanced image is extracted through the Transformer model Pair the global feature vectors of the same workpiece from different perspectives to generate positive sample pairs; Randomly select images I of different artifacts and calculate the global feature vector for each image Pair the global feature vectors from different artifacts to generate negative sample pairs; The optimized positive and negative sample pairs are obtained by specifically performing the following steps: For each pair of positive and negative samples, the cosine similarity is calculated, and the InfoNCE loss function is applied to optimize the neural network model; Use the Adam optimizer to update the parameters of the neural network model based on the InfoNCE loss; Calculate the loss L and backpropagate the loss Update the neural network model parameters until the neural network model reaches a predetermined training round; Map the input samples to the feature space of the trained neural network model to further optimize the positive and negative sample pairs; The meta-learning mechanism is used to quickly adapt the deep contrastive learning model. The specific steps are: Divide the optimized positive and negative sample pairs into multiple meta-tasks T i , each task contains positive and negative sample pairs of artifacts; On the support set of each task, the temperature parameters are adjusted using an adaptive temperature regulation mechanism; After adjusting the parameters in the support set, the weight w is dynamically assigned to each task through dynamic task weights. i The expression is: w i =(1+exp(-γ·(L i (Q i ,θ')-L avg ))) -1 ; Among them, -γ represents the hyperparameter that adjusts the sensitivity of task difficulty to weight, L avg represents the average loss on the query set of all tasks, and θ' represents the model's performance on the meta-task T i The parameters after a gradient update on the support set, L i (Q i ,θ') represents the meta-task T i In the query set Q i losses; Combined with the initial update of the support set and the query set loss, the dynamic weights w of each task are i Perform global parameter updates; Through multiple rounds of task iteration, the deep contrastive learning model is gradually optimized; The specific steps of outputting the defect detection result of the workpiece are as follows: Convert the input workpiece image into a grayscale image and perform standardization; For the grayscale image after standardization, the Canny edge detection algorithm is used to extract the workpiece contour and generate the region of interest; The extracted workpiece region of interest is input into the Transformer encoder, and the feature embedding vector of the input workpiece is extracted through the encoder; Call the pre-optimized positive sample feature embedding vector Z from the sample library pos and negative sample feature embedding vector Z neg ; Calculate the input workpiece feature z input and the positive sample feature embedding vector Z pos The cosine similarity sim pos ; Calculate the input workpiece feature z input and the negative sample feature embedding vector Z neg The cosine similarity sim neg ; Based on the calculated similarity, a preset threshold τ is used to determine whether the input workpiece has defects; If sim pos <τ and sim neg >τ, the workpiece is judged to have defects, otherwise, it is judged to be free of defects.

2. The defect detection method based on deep contrast learning according to claim 1, characterized in that: The specific steps of obtaining the original image data of the industrial workpiece are as follows: Collect high-definition workpiece images through industrial cameras, use OpenCV to read the collected image files, and load them into memory for processing; Check whether the read image is in RGB format. If the image is not in RGB format, convert it to RGB format. After ensuring that the image is in RGB format, the image size is further checked, and finally the inspected and standardized raw image data of the industrial workpiece is output.

3. The defect detection method based on deep contrast learning according to claim 2, characterized in that: The forming of standardized image data comprises the following specific steps: Normalizing the pixel values ​​of the raw image data of the inspected and normalized industrial workpiece; Based on the normalized image, denoising is performed through bilateral filtering; The denoised image is cropped to the key part containing the artifact, while keeping the size of the cropped image consistent to form standardized image data.

4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the defect detection method based on deep contrast learning described in any one of claims 1 to 3 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the defect detection method based on deep contrast learning described in any one of claims 1 to 3 are implemented.

Images