A surface defect detection method based on metric meta-learning

Abstract

The application discloses a surface defect detection method based on metric meta-learning, and belongs to the technical field of industrial visual monitoring, and the steps are as follows: acquiring defect images and non-defect images, and constructing a data set; a cascaded generative adversarial network based on a fusion multi-dimensional attention mechanism is used to generate high-resolution images, and the multi-dimensional attention mechanism is used to improve the local details and global feature expression capability of the images; a defect detection model is constructed, and weak supervision detection is performed on surface defects based on metric learning and meta-learning. Through metric learning and feature space constraint, the application performs confidence filtering and structure consistency constraint on unlabeled samples, thereby effectively improving the representation capability and generalization performance of the model, focusing on solving the problems of few defect samples, high labeling cost and difficult representation of micro-defects in the industrial scene, and improving the accuracy and fine-grained discrimination performance of defect detection.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of industrial visual monitoring, and particularly relates to a surface defect detection method based on metric meta-learning. BACKGROUND

[0002] Anomaly detection has important applications in the fields of industrial defect detection, medical image analysis, security monitoring, etc. Industrial surface defect detection, as a key link of intelligent manufacturing, still faces major challenges: (1) Defect data scarcity dilemma: The industrial good rate is usually higher than 95%, leading to a typical long-tail distribution feature of defect samples. Traditional supervised learning methods have a risk of feature space collapse in the extreme few-shot scenario where the sample size is less than Shot=5, and the model is easily disturbed by background noise, resulting in false detection. (2) Weak supervision learning reliability bottleneck: In industrial scenarios, only image-level weak labels (such as whole-image defect labels) can be obtained, and pixel-level labeling is costly. Existing weak supervision methods have a significant error propagation risk in defect positioning. (3) Cross-domain generalization capability defect: Domain shift problems caused by changes in equipment parameters and fluctuations in lighting conditions in industrial production lines lead to significant performance degradation of traditional detection models.

[0003] Due to the scarcity of abnormal sample data itself, traditional supervised learning methods rely on large-scale labeled samples for training, and it is difficult to achieve effective generalization of the model in the case of insufficient data. In recent years, generative adversarial networks (GAN) have received widespread attention due to their excellent performance in image generation. Data augmentation using GAN can to some extent alleviate the problem of sample shortage, but traditional generative adversarial networks (GAN) have significant limitations in industrial defect generation: ① Pattern collapse phenomenon caused by loss of high-frequency texture details; ② Insufficient multi-scale defect representation capability; ③ Lack of feature focusing mechanism for key defect areas. In addition, existing learning methods have adaptation barriers in industrial defect detection: ① Fuzzy decision boundary caused by lack of structured constraints in feature space; ② Single-granularity metric space is difficult to cope with complex defect morphology changes;

[0004] To solve the above problems, the application proposes a high-quality defect image generation method that combines multi-dimensional attention mechanism to enhance the feature extraction capability and stability of the generation network. At the same time, a few-shot meta-learning and weak supervision representation learning strategy based on multi-granularity similarity is introduced to make full use of a small amount of labeled data and a large amount of unlabeled data. In the weak supervision learning environment, due to the presence of only a small amount of accurately labeled samples, noise is often introduced in the automatic pseudo-labeling process of a large amount of unlabeled data, leading to misclassification and increasing the risk of model overfitting. Therefore, the application uses metric learning and feature space constraints to screen the confidence of unlabeled samples and constrain the structural consistency, thereby effectively improving the representation ability and generalization performance of the model. SUMMARY

[0005] In view of the above problems existing in the prior art, the present application provides a surface defect detection method based on metric meta-learning, which is reasonable in design and solves the problems of the prior art, and has good effects.

[0006] A surface defect detection method based on metric meta-learning, comprising the following steps:

[0007] Step 1: Obtain defect images and non-defect images, and construct a data set;

[0008] Step 2: Construct a defect image generation module based on a two-stage cascaded generative adversarial network, and improve the local detail and global feature expression ability of the image through a multi-dimensional attention mechanism;

[0009] Step 3: Construct a defect detection model, and perform weak supervision detection on the surface defects based on metric learning and meta-learning.

[0010] Further, in step 1, a large number of non-defect images are collected from an industrial production line, and a small number of defect samples are obtained by manual or semi-automatic means; the defect samples are subjected to a data expansion method to form a preliminary expanded data set, and the data expansion method includes image rotation, flipping, scaling and color change; the non-defect images are processed in combination with a defect mask set by an expert to construct a conditional input of the defect image, and the defect mask is represented by a binary image, wherein "1" represents a possible defect area, and "0" represents a background area.

[0011] Further, in step 2, the first-stage cascaded generative adversarial network comprises the following sub-steps:

[0012] Step 2.1: Let the initial input of the network be [I n ,M,z], is a non-defect image, is a defect mask, is a random noise vector, wherein H is the height, W is the width, and C is the number of channels; first, the initial input is subjected to a convolution layer, and the expression is: wherein represents a feature-level connection, FC is a full connection operation, Conv is a convolution operation, BN is a batch normalization operation, and ReLU is a nonlinear activation function;

[0013] Step 2.2: Then, a multi-dimensional attention residual block is used, and in each residual block, the input features are subjected to a convolution operation, and a channel and spatial attention fusion module is introduced;

[0014] The channel attention uses maximum pooling and average pooling on the input feature map along the spatial dimension to obtain a one-dimensional channel attention map, then performs nonlinear transformation through a multilayer perceptron, and then element-wise sums the two transformed channel attention maps and activates to obtain the feature A c ;

[0015] In the spatial attention process, maximum pooling and average pooling are used on the input feature map along the channel dimension to obtain a two-dimensional spatial attention map, then the channel dimension is spliced, mapped through convolution and activation, and the feature A is obtained s ;

[0016] A c and A s are element-wise multiplied with the input feature, and the residual feature F c is added to obtain where F represents the input feature;

[0017] Step 2.3: After N residual blocks, the output feature F res is obtained N = ResBlock N (F'), ResBlock N represents N residual blocks; the generator output low-resolution feature F LR = Tanh(Conv(ReLU(BN(Conv(F res ))))), Tanh is the hyperbolic tangent activation function.

[0018] Further, in step 2, the second stage cascaded generative adversarial network is specifically:

[0019] Let the first stage feature be F LR , expressed as:

[0020] F fusion = F LR + Conv 3×3 (F inter ));

[0021] where F inter is the feature residual extracted by the intermediate layer, and F fusion is the feature after residual fusion;

[0022] Pixel rearrangement or deconvolution layer is used for upsampling, gradually enlarged to the target resolution HxW to obtain the feature F HR , expressed as:

[0023] F HR = Upsample(F fusion );

[0024] Upsample is an up-sampling operation;

[0025] The final generator outputs a high-resolution image The expression is:

[0026]

[0027] The discriminator contains 6 convolutional blocks, each of which contains a 3x3 convolutional layer, a batch normalization layer and a nonlinear activation layer, for progressively extracting image features, and a multi-dimensional attention residual block is inserted after the 3rd-5th convolutional blocks to enhance important feature responses. The output feature map passes through a fully connected discriminator layer, and through a fully connected layer and sigmoid activation, outputs the probability value of image true or false judgment;

[0028] The first-stage cascaded generative adversarial network and the second-stage cascaded generative adversarial network use the same structure of the discriminator to ensure consistent discrimination standards, only the input resolution is different;

[0029] In the two stages, the generator and the discriminator use the same form of loss function, and the loss function uses an adversarial loss:

[0030]

[0031] wherein, is the generator loss function, E z is the expectation of the corresponding distribution of the input noise vector, D(·) is the discriminator function, and G(z) is the generator function, is the discriminator loss function, E x is the expectation of the distribution of the input real sample x, λ is a penalty coefficient, GP is a gradient penalty term, and its formula is represented as wherein represents the gradient, represents an interpolation sample.

[0032] Further, the step 3 includes the following sub-steps:

[0033] Step 3.1: Constructing a support set using labeled samples, the labeled samples including real defect images labeled by artificial means in step 1 and defect images synthesized by the generative adversarial network in step 2; samples to be classified or detected as a query set, the samples to be classified or detected being images collected from the production line in real time; unlabeled samples as a potential pseudo-label candidate set, the unlabeled samples being a large number of original non-defect images collected in step 1;

[0034] Step 3.2: Design a feature encoder with a multi-dimensional attention mechanism, which is composed of a parallel convolution branch module and a self-attention branch module; the input sample is first passed through the convolution branch module to extract texture information and edge features; at the same time, it is input into the self-attention branch module to capture long-range dependencies on context information through a multi-head attention mechanism;

[0035] Step 3.3: Construct a multi-granularity similarity calculation module based on support-query structure and category prototype;

[0036] Step 3.4: Design a weakly supervised pseudo-labeling module and a loss function;

[0037] Step 3.5: Construct a defect detection model based on the feature encoder, similarity calculation module, weakly supervised pseudo-labeling module, and softmax classifier, and train the model using a meta-learning training strategy.

[0038] Further, in step 3.2, the convolution branch module is specifically:

[0039] Let the input image be I, first pass through multiple convolution operations to extract local texture information, and the expression of each convolution operation is:

[0040] F conv =ReLU(BN(Conv 3×3 (I)));

[0041] After each convolution operation, a max-pooling layer is added for down-sampling, and the output feature of the convolution branch module is F' conv .

[0042] The self-attention branch module is specifically:

[0043] First, linearly map the input feature F' conv to generate multiple query vectors Q, key vectors K, and value vectors V, then calculate the attention matrix Attention(Q, K, V) through the scaled dot-product attention mechanism:

[0044]

[0045] Where d k is the dimension of the key vector;

[0046] The attention results of multiple heads are concatenated and then mapped back to the original dimension through linear transformation, and the final output F attn is obtained by residual connection with the input feature and layer normalization, and the expression is:

[0047] F attn =LayerNorm(F' conv +Concat(head1,...,headh )W O );

[0048] where head1,...,head h are the attention results of multiple heads, W O is a learnable weight matrix to map the concatenated channels back to the original number of channels, Concat is a concatenation operation, and LayerNorm is a normalization operation.

[0049] The outputs of the convolution branch module and the self-attention branch module are fused by weighting to obtain a unified high-resolution semantic embedding feature F fusion , and the expression is as follows:

[0050] F fusion =α·F conv +(1-α)·F attn ;

[0051] where α∈[0,1] is a learnable fusion weight.

[0052] Further, the step 3.3 includes the following sub-steps:

[0053] Step 3.3.1: For each class, extract the sample feature vectors corresponding to all classes in the support set output by the feature encoder {f i k}, where i is the sample number and k is the class number.

[0054] Step 3.3.2: Based on the idea of prototype network, the following weighted average formula is used to calculate:

[0055]

[0056] where N k is the number of samples of class k, and c k is the class center, which is used for subsequent similarity measurement.

[0057] Step 3.3.3: Take the class center c k as the reference representation for similarity calculation between the query set samples and the pseudo-labeled candidate set samples, and design two-level multi-granularity similarity calculation. The local similarity is to divide the image into blocks, and calculate the Euclidean distance d local for each block; the global similarity is to calculate the global feature cosine similarity distance d global , where the calculation process of d local and d global is as follows:

[0058]

[0059] where f llocal represents the l-th block in the spatial division of the feature map of the sample into m blocks of the same size, represents the center of the l-th block;

[0060]

[0061] wherein f global is the feature vector after global average pooling of the sample feature map;

[0062] Step 3.3.4: Fusion of local and global similarities, expression:

[0063] d fusion = βd local + (1-β)d global ;

[0064] wherein d fusion is the multi-granularity similarity calculation result, β is the fusion weight, which is adjusted according to actual training.

[0065] Further, the step 3.4 includes the following sub-steps:

[0066] Step 3.4.1: For unlabeled images, calculate the multi-granularity similarity between them and each class prototype to obtain pseudo labels Only when the maximum similarity meets a certain confidence threshold τ, the corresponding sample is included in the training support set to reduce the introduction of noise;

[0067] Step 3.4.2: Design a loss function, use cross-entropy loss, feature contrast loss and prototype loss to jointly optimize, and improve the robustness and generalization ability of the model;

[0068] The cross-entropy loss L CE is expressed as:

[0069]

[0070] wherein y k is the true label of class k, is the pseudo label predicted by class k; the cross-entropy loss is used to measure the accuracy of the model's classification prediction under the condition of given support set prototype; the model calculates the similarity between the query sample and all class prototypes, and obtains the prediction probability through softmax, which is compared with the true label to produce an error as a supervision signal for back propagation, which is used to train the classification module and feature encoding;

[0071] The prototype loss L proto is expressed as:

[0072]

[0073] wherein s(i) e k represents that the real class of sample i s(i) is k; the prototype loss is used to constrain the features of the same class support samples to be close to the corresponding class prototype center, and improve the intra-class consistency;

[0074] the feature contrast loss L contrast The expression is:

[0075]

[0076] wherein m is a set boundary isolation, f i is the feature vector of sample i, f j is the feature vector of sample j, y ij is the similarity label of sample i and sample j, y ij = 1 represents that the two are same class samples, y ij = 0 represents that the two are different class samples; the feature contrast loss is used to optimize the feature separability between classes, by constructing same class and different class sample pairs, calculating the relative distance between their features, and narrowing the distance between same class samples and pushing away different class samples, so as to improve the discriminability of the whole feature space;

[0077] The total loss is defined as:

[0078] L total = λ CE L CE + λ proto L proto + λ contrast L contrast ;

[0079] wherein λ CE , λ proto , λ contrast are loss weights, which are determined by cross-validation.

[0080] Further, the step 3.5 includes the following sub-steps:

[0081] Step 3.5.1: sampling a plurality of small tasks from the training data, each task being composed of a support set and a query set;

[0082] Step 3.5.2: for each task, calculating the loss using the support set, updating the parameters according to the gradient descent, and the expression is:

[0083]

[0084] wherein α is a learning rate hyperparameter, used to control the step size at each gradient update, is the gradient of the parameter θ, L task (·) is the task loss;

[0085] Step 3.5.3: On the basis of the update results of each task, the average gradient updates the initial parameters, and the expression is:

[0086]

[0087] Wherein, beta is a global learning rate hyperparameter for cross-task meta update;

[0088] Step 3.5.3: The updated initial parameters are brought into step 3.5.2 for multiple iterations, so that the model parameters have good rapid adaptability.

[0089] The beneficial technical effects brought by the present application are:

[0090] The present application proposes a multi-dimensional attention mechanism combined cascade generative adversarial network (CCGAN) and multi-granularity similarity measurement meta learning method, which optimizes data generation and feature learning method, improves the performance and applicability of the defect anomaly detection system. This method can be widely used in various anomaly detection tasks and has high practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0091] Figure 1 The principle diagram for generating high-resolution images in the cascade generative adversarial network in the present application;

[0092] Figure 2 Flowchart of the under-labeled weakly supervised learning method based on meta learning in the present application; DETAILED DESCRIPTION

[0093] The specific embodiments of the present application will be further described below in combination with specific embodiments:

[0094] A surface defect detection method based on meta learning, comprising the following steps:

[0095] Step 1: Obtain defect images and non-defect images, and construct a data set;

[0096] In step 1, a large number of non-defect images are collected from the industrial production line, and a small amount of defect samples are obtained by manual or semi-automatic method; After the defect samples are subjected to data expansion method, a preliminary expanded data set is formed, and the data expansion method includes image rotation, flipping, scaling and color change; The non-defect images are processed in combination with the defect mask set by the expert to construct the conditional input of the defect image, and the defect mask is represented by a binary image, wherein "1" represents the possible defect area, and "0" represents the background area.

[0097] Step 2: The high-resolution image is generated by the cascaded generative adversarial network based on the fusion multi-dimensional attention mechanism, which improves the expression of local details and global features of the image through the multi-dimensional attention mechanism; the cascaded generative adversarial network includes a first-stage cascaded generative adversarial network and a second-stage cascaded generative adversarial network, as shown in Figure 1 , wherein the first stage is used to generate a low-resolution defect image to provide global contour information for subsequent generation; the second stage generates a high-resolution image by combining the output of the first stage and the intermediate feature residual error, thereby improving the synthesis quality of defect local details;

[0098] The first-stage cascaded generative adversarial network includes the following sub-steps:

[0099] Step 2.1: Let the initial input of the network be [I n ,M,z], is a non-defect image, is a defect mask, is a random noise vector, and z can be expanded through a fully connected layer and reshaped into a feature map consistent with the image size, where H is the height, W is the width, and C is the number of channels; first, the initial input is passed through a convolutional layer, and the expression is: wherein represents feature-level connection, FC is a fully connected operation, Conv is a convolutional operation, BN is a batch normalization operation, and ReLU is a nonlinear activation function;

[0100] Step 2.2: Then, the multi-dimensional attention residual block is used again, and within each residual block, the input features are subjected to convolutional operation, and then the channel and spatial attention fusion modules are introduced;

[0101] The channel attention uses maximum pooling and average pooling along the spatial dimension of the input feature map to obtain a one-dimensional channel attention map, which is then subjected to nonlinear transformation through a multi-layer perceptron, the two transformed channel attention maps are summed element by element, and then activated to obtain the feature A c ;

[0102] In the spatial attention process, maximum pooling and average pooling are used along the channel dimension of the input feature map to obtain a two-dimensional spatial attention map, which is then spliced in the channel dimension, mapped through convolution and activation, and the feature A s is obtained.

[0103] A c and A s are multiplied element by element with the input features, and the residual feature F c is added to obtain wherein F represents the input feature, and

[0104] Step 2.3: After passing through N residual blocks, output feature F is obtained res = ResBlock N (F'), ResBlock N denotes N residual blocks; the generator outputs low-resolution feature F LR = Tanh(Conv(ReLU(BN(Conv(F res ))))), where Tanh is the hyperbolic tangent activation function.

[0105] The second stage of the cascaded generative adversarial network is specifically:

[0106] Let the first stage feature be F LR , which is expressed by the fusion module and cross-layer connection as:

[0107] F fusion = F LR + Conv 3×3 (F inter ));

[0108] where F inter is the feature residual extracted by the intermediate layer, and F fusion is the feature after fusing the residual;

[0109] Pixel rearrangement (PixelShuffle) or deconvolution layer is used for upsampling, gradually enlarged to the target resolution HxW to obtain feature F HR , expressed as:

[0110] F HR = Upsample(F fusion );

[0111] where Upsample is the upsampling operation;

[0112] The final generator outputs high-resolution image , expressed as:

[0113]

[0114] The discriminator contains 6 convolution blocks, each containing a 3x3 convolution layer, a batch normalization layer and a nonlinear activation layer, which are used to extract image features step by step. After the 3rd-5th convolution blocks, a multi-dimensional attention residual block (channel attention and spatial attention) is inserted to enhance important feature responses. The output feature map is passed through a fully connected discriminator layer, which outputs a probability value of image authenticity through a fully connected layer and a sigmoid activation.

[0115] The first stage cascaded generative adversarial network and the second stage cascaded generative adversarial network adopt the same structure of the discriminator to ensure consistent discrimination standards, and only the input resolution is different;

[0116] In the two stages, the generator and the discriminator adopt the same form of loss function, and the loss function adopts an adversarial loss:

[0117]

[0118]

[0119] wherein, is a generator loss function, E z is the expectation of the corresponding distribution of the input noise vector, D(·) is a discriminator function, an input image, and an output probability belonging to a real distribution, G(z) is a generator function, an input random noise, and an output generated feature, z represents a noise vector, is a discriminator loss function, E x is the expectation of the distribution of the input real sample x, λ is a penalty coefficient, and GP is a gradient penalty term, and its formula is represented as wherein represents a gradient, represents an interpolation sample, which is obtained by linear interpolation between a real sample x and a generated sample G(z).

[0120] Step 3: Construct a defect detection model, and perform weakly supervised detection on surface defects based on metric learning and meta-learning.

[0121] Step 3.1: Use the labeled samples to construct a support set for extracting class feature representation, the labeled samples including the real defect images labeled by manual means in step 1 and the defect images synthesized by the generative adversarial network in step 2; the samples to be classified or detected as a query set, the samples to be classified or detected are images collected from the production line in real time; the unlabeled samples as a potential pseudo-label candidate set, the unlabeled samples are a large number of original non-defect images collected in step 1;

[0122] Step 3.2: Design a feature encoder with a fusion multi-dimensional attention mechanism, and the encoder is composed of a parallel convolution branch module and a self-attention branch module; the input sample is first passed through the convolution branch module to extract texture information and edge features; at the same time, it is input to the self-attention branch module to capture long-range dependencies on context information through the multi-head attention mechanism;

[0123] The convolution branch module is specifically:

[0124] Let the input image be I, which is first subjected to a plurality of convolution operations to extract local texture information, and the expression of each convolution operation is:

[0125] Fconv =ReLU(BN(Conv) 3×3 (I)));

[0126] A max-pooling layer is added after each convolutional operation for downsampling, and the output feature of the convolutional branch module is F'. conv ;

[0127] The self-attention branch module employs multi-head attention and residual connections to enhance the model's ability to perceive long-range dependencies and global semantic context. The self-attention-based approach uses a feature-context weighted averaging method, dynamically calculating attention weights based on the similarity function between relevant pixel pairs. Specifically:

[0128] First, the input feature F' conv Multiple query vectors Q, key vector K, and value vector V are generated through linear mapping. Then, the attention matrix Attention(Q,K,V) is calculated using a scaled dot product attention mechanism.

[0129]

[0130] Where, d k It is the dimension of the key vector, used to prevent the inner product from becoming too large;

[0131] The attention results from multiple heads are concatenated, then mapped back to the original dimension through a linear transformation. A residual connection is then established with the input features, and the result is normalized to obtain the final output F. attn To further improve model stability and training efficiency, the expression is:

[0132] F attn =LayerNorm(F' conv +Concat(head1,...,head h W O );

[0133] Among them, head1,...,head h For the attention results of multiple heads, W O To map the concatenated channels back to the original number of learnable weight matrices, Concat is the concatenation operation and LayerNorm is the normalization operation.

[0134] The outputs of the convolutional branch module and the self-attention branch module are weighted and fused to obtain a unified high-resolution semantic embedding feature F. fusion The expression is:

[0135] F fusion =α·Fconv + (1 - a) · F attn ;

[0136] where a e [0, 1] is a learnable fusion weight.

[0137] Step 3.3: Constructing the similarity calculation module based on support-query structure and category prototype;

[0138] Step 3.3 includes the following sub-steps:

[0139] Step 3.3.1: For each category, extract the sample feature vectors corresponding to all categories in the support set output by the feature encoder {f i k}, where i is the sample number and k is the category number;

[0140] Step 3.3.2: Based on the idea of prototype network, the weighted average formula is used to calculate:

[0141]

[0142] where N k is the number of samples of category k, c k is the class center, which is used for subsequent similarity measurement;

[0143] Step 3.3.3: Take the class center c k as the reference representation for similarity calculation of query set samples and pseudo-labeled candidate set samples, in order to fully capture the information of images at multiple levels such as local structure, texture, and semantics, two-level multi-granularity similarity calculation is designed. Local similarity is to divide the image into blocks, and calculate the Euclidean distance d local for each block; global similarity is to calculate the global feature cosine similarity distance d global , where the calculation process of d local and d global is as follows:

[0144]

[0145] where f l local represents the l-th block in the spatial division of the sample feature map into m blocks of the same size, represents the l-th block center;

[0146]

[0147] where f global is the feature vector after global average pooling of the sample feature map;

[0148] Step 3.3.4: Fuse local and global similarity, the expression is:

[0149] d fusion = βd local + (1-β)d global ;

[0150] wherein d fusion is a multi-granularity similarity calculation result, β is a fusion weight, which is adjusted according to actual training.

[0151] Step 3.4: design a weakly supervised pseudo-labeling module and design a loss function;

[0152] Step 3.4 includes the following sub-steps:

[0153] Step 3.4.1: for unlabeled images, calculate the similarity between them and each category prototype to obtain pseudo-labels Only when the maximum similarity (or minimum distance) meets a certain confidence threshold τ, the corresponding sample is included in the training support set to reduce the introduction of noise;

[0154] The category prototype vector is not only used for pseudo-label generation of unlabeled samples, but also used for classification and detection operations of query set samples. Specifically, for query set samples, by calculating the similarity between them and each category prototype, the predicted category is determined, and is used for fast classification / detection in the inference stage, and the feature encoder and related module parameters are updated in the training stage by the classification error of the query set samples; in the training process, the similarity is used as the non-normalized category score, the category score is normalized by softmax to obtain the category probability distribution, and then the cross entropy is used to measure the consistency of the prediction result and the true label. For unlabeled samples, pseudo labels are generated by calculating the similarity with each category prototype and combining the confidence threshold, only when the maximum similarity meets the set threshold condition, the corresponding sample is included in the support set to expand the training data and enhance the model generalization ability. The category prototype similarity uniformly supports the classification decision in the inference stage and the weakly supervised expansion in the training stage, ensuring the consistency and effectiveness of the training and inference process.

[0155] Step 3.4.2: design a loss function, use cross-entropy loss, feature contrast loss and prototype loss to jointly optimize, and improve the robustness and generalization ability of the model;

[0156] The cross-entropy loss L CE (for labeled samples) is expressed as:

[0157]

[0158] wherein y k is the true label of category k, is the pseudo label predicted for category k;

[0159] The cross-entropy loss is used to measure the accuracy of the model's classification prediction for the query sample under the condition of given support set prototypes. The model calculates the similarity between the query sample and all category prototypes, obtains the prediction probability through softmax, and generates an error after comparing with the true label (or pseudo label), which is used as a supervision signal for back propagation to train the classification module and feature encoding.

[0160] Prototype loss L proto is used to minimize the distance between the support set sample and the corresponding category prototype, and the expression is:

[0161]

[0162] where s(i) e k represents that the real category s(i) of sample i (or the pseudo label category) is k;

[0163] Feature contrast loss L contrast The expression is:

[0164]

[0165] where m is the set boundary isolation, f i is the feature vector of sample i, f j is the feature vector of sample j, y ij is the similarity label of sample i and sample j, y ij = 1 indicates that the two are same category samples, and y ij = 0 indicates that the two are different category samples; the feature contrast loss is used to optimize the feature separability between categories, by constructing same and different category sample pairs, calculating the relative distance between their features, and pulling the same category samples and pushing the different category samples, so as to improve the discriminability of the overall feature space;

[0166] The total loss is defined as:

[0167] L total = λ CE L CE + λ proto L proto + λ contrast L contrast ;

[0168] where λ CE , λ proto , λ contrast are the loss weights, which are determined by cross-validation.

[0169] Step 3.5: Based on the feature encoder, the similarity calculation module, the supervised pseudo labeling module and the softmax classifier, a defect detection model is constructed, and a meta-learning training strategy is used to train the model, as shown in Figure 2 .

[0170] Step 3.5 includes the following sub-steps:

[0171] Step 3.5.1: Sample multiple small tasks from the training data, each task consisting of a support set and a query set;

[0172] Step 3.5.2: For each task, calculate the loss using the support set, update the parameters according to gradient descent, the expression is:

[0173]

[0174] where α is the learning rate hyperparameter, used to control the step size at each gradient update, is the gradient of the parameter θ, L task (·) is the task loss, which is the same as the overall loss function defined in step 3.4.2, but for different sampled task batches;

[0175] Step 3.5.3: Based on the update results of each task, average the gradient to update the initial parameters, the expression is:

[0176]

[0177] where β is the global learning rate hyperparameter for cross-task meta update, which controls the step size when adjusting the initial model parameters θ based on the average gradient of multiple small tasks, and it can balance the rapid adaptability and overall stability of the model between different tasks;

[0178] Step 3.5.3: Bring the updated initial parameters into step 3.5.2 for multiple iterations, so that the model parameters have good rapid adaptability.

[0179] Of course, the above description is not a limitation of the present application, and the present application is not limited to the above examples. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present application should also be within the scope of the present application.

Claims

1. A surface defect detection method based on metric meta-learning, characterized in that, The method comprises the following steps: Step 1: obtaining defect images and non-defect images, and constructing a data set; Step 2: generating high-resolution images based on a cascade generative adversarial network with a fusion multi-dimensional attention mechanism, and improving the local details and global feature expression ability of the images through the multi-dimensional attention mechanism; Step 3: constructing a defect detection model, and performing weakly supervised detection on surface defects based on metric learning and meta-learning; The step 3 comprises the following sub-steps: Step 3.1: constructing a support set using labeled samples, the labeled samples comprising real defect images labeled by manual means in step 1 and defect images synthesized by the generative adversarial network in step 2; samples to be classified or detected as a query set, the samples to be classified or detected being images collected in real time from a production line; and unlabeled samples as a potential pseudo-label candidate set, the unlabeled samples being a large number of original non-defect images collected in step 1; Step 3.2: designing a feature encoder with a fusion multi-dimensional attention mechanism, the encoder being composed of a parallel convolution branch module and a self-attention branch module; the input sample is first passed through the convolution branch module to extract texture information and edge features; and at the same time, the input sample is input to the self-attention branch module to capture long-range context information through a multi-head attention mechanism; Step 3.3: constructing a multi-granularity similarity calculation module based on a support-query structure and a category prototype; Step 3.4: designing a weakly supervised pseudo-labeling module and a loss function; Step 3.5: constructing a defect detection model based on the feature encoder, the similarity calculation module, the weakly supervised pseudo-labeling module and a softmax classifier, and training the model using a meta-learning training strategy.

2. The surface defect detection method based on metric meta-learning according to claim 1, wherein, In the step 1, a large number of non-defect images are first collected from an industrial production line, and a small number of defect samples are obtained by manual or semi-automatic means; after the defect samples are subjected to a data augmentation method, a preliminary augmented data set is formed, the data augmentation method comprising image rotation, flipping, scaling and color change; the non-defect images are processed in combination with a defect mask set by an expert to construct a conditional input of the defect images, the defect mask being represented by a binary image, wherein "1" represents a possible defect area, and "0" represents a background area.

3. The surface defect detection method based on metric meta-learning according to claim 2, characterized in that, In the step 2, the cascade generative adversarial network comprises a first-stage cascade generative adversarial network and a second-stage cascade generative adversarial network, and the first-stage cascade generative adversarial network comprises the following sub-steps: Step 2.1: Let the initial input of the network be , is a non-defect image, is a defect mask, is a random noise vector, where is the height, is the width, is the number of channels; the initial input is first passed through a convolutional layer, expressed as: where denotes feature-level concatenation, is a fully connected operation, is a convolutional operation, is a batch normalization operation, is a non-linear activation function; Step 2.2: then, a multi-dimensional attention residual block is used to perform convolution operation on the input features in each residual block, and a channel and spatial attention fusion module is introduced; The channel attention uses maximum pooling and average pooling respectively along the spatial dimension of the input feature map to obtain a one-dimensional channel attention map, and then a multilayer perceptron is used for nonlinear transformation. The transformed two channel attention maps are summed element by element, activated, and the feature ; In the spatial attention process, the input feature map is respectively subjected to max pooling and average pooling along the channel dimension to obtain a two-dimensional spatial attention map, which is then spliced in the channel dimension, subjected to convolution and activation, and mapped to obtain a feature ; will be described below. and Element-wise multiplication with input features and adding residual features , to obtain wherein denotes the input features; Step 2.3: After going through N residual blocks, output features are obtained , denote N residual blocks; Generator outputs low resolution features , is the hyperbolic tangent activation function.

4. The surface defect detection method based on metric meta-learning according to claim 3, characterized in that, In the step 2, the second-stage cascade generative adversarial network is specifically: Let the first stage feature be , expressed by the fusion module and cross-layer connection as: ； wherein, is the feature residual extracted for the intermediate layer, is the fused residual feature; upsample using pixel rearrangement or deconvolution layers, progressively enlarging to the target resolution obtaining features , the expression is: ； wherein is an up-sampling operation; The final generator outputs a high resolution image The expression is: ； The discriminator comprises six convolution blocks, each convolution block comprising a 3x3 convolution layer, a batch normalization layer and a nonlinear activation layer, for progressively extracting image features; a multi-dimensional attention residual block is inserted after the 3rd to 5th convolution blocks to enhance important feature responses; and the output feature map is subjected to a full connection discrimination layer, a full connection and a sigmoid activation, to output a probability value for judging whether the image is true or false; The first-stage cascaded generative adversarial network and the second-stage cascaded generative adversarial network adopt the same structure of the discriminator to ensure consistent discrimination standards, and only the input resolution is different; In the two stages, the generator and the discriminator adopt the same form of loss function, and the loss function adopts an adversarial loss: ； ； wherein, is the generator loss function, is the expectation of the input noise vector corresponding distribution, is the discriminator function, is the generator function, is the discriminator loss function, is the expectation of the distribution of input real samples x, is the penalty coefficient, is the gradient penalty term, which is expressed as wherein denotes the gradient, denotes the interpolated sample.

5. The surface defect detection method based on metric meta-learning according to claim 4, characterized in that, In step 3.2, the convolution branch module is specifically: Let the input image be I, first pass through multi-layer convolution operation to extract local texture information, and the expression of each layer of convolution operation is: ； A max pooling layer is added after each convolution operation to down-sample, and the output features of the convolution branch module are ; The self-attention branch module is specifically: First, the input features are linearly mapped to generate a plurality of query vectors Q, key vectors K, and value vectors V, and then an attention matrix A is calculated by a scaled dot-product attention mechanism : ； wherein is the dimension of the bond vector; The attention results of multiple heads are concatenated and mapped back to the original dimension through linear transformation, and the final output is obtained through residual connection with the input features and layer normalization , the expression is: ； wherein, is the attention result for the plurality of heads, is a learnable weight matrix that maps the concatenated channels back to the original number of channels, is a concatenation operation, is a normalization operation; The outputs of the convolution branch module and the self-attention branch module are fused by weighting to obtain unified high-resolution semantic embedding features The expression is: ； wherein, are learnable fusion weights.

6. The surface defect detection method based on metric meta-learning according to claim 5, wherein, The step 3.3 includes the following sub-steps: Step 3.3.1: For each class, extract the sample feature vectors corresponding to all classes in the support set of the feature encoder output where is the sample number, is the class number; Step 3.3.2: Based on the idea of the prototype network, the weighted average formula is used to calculate: ； in, For category The number of samples, Used as class centers for subsequent similarity measurement; Step 3.3.3: The class center is calculated as follows As the benchmark representation of similarity calculation of query set samples and pseudo-labeled candidate set samples, two-level multi-granularity similarity calculation is designed. Local similarity is to divide the image into blocks, and calculate the Euclidean distance of each block Global similarity is to calculate the global feature cosine similarity distance Wherein And The calculation process is as follows: ； wherein, represents dividing the feature map of the sample into blocks of the same size, the th block, represents the center of the th block; ； wherein, is the feature vector after global average pooling of the sample feature map; Step 3.3.4: Fuse the local and global similarities, and the expression is: ； wherein, is a multi-granularity similarity calculation result, is a fusion weight, which is adjusted according to actual training.

7. The surface defect detection method based on metric meta-learning according to claim 6, wherein, The step 3.4 includes the following sub-steps: Step 3.4.1: For unlabeled images, calculate their multi-granularity similarity with each category prototype to obtain pseudo labels Only when the maximum similarity meets a certain confidence threshold τ, the corresponding sample is included in the training support set to reduce noise introduction. Step 3.4.2: Design the loss function, and use the cross-entropy loss, the feature contrast loss and the prototype loss to jointly optimize to improve the robustness and generalization ability of the model; cross-entropy loss The expression is: ； wherein, is a true label of the class , is a predicted pseudo label of the class ; the cross-entropy loss is used to measure the accuracy of the model's classification prediction for the query sample under the condition of given support set prototypes; the model calculates the similarity between the query sample and all class prototypes, obtains the prediction probability through softmax, and produces an error after comparing with the true label, which is used as a supervision signal for back propagation to train the classification module and feature encoding; Prototype loss The expression is: ； wherein, representing the true class of the sample is ; the prototype loss is used to constrain the features of the same class support samples to be close to their corresponding class prototype center, and to improve the intra-class consistency; characteristic contrast loss The expression is: ； in, It is a set boundary isolation. For the sample eigenvectors, For the sample eigenvectors, For the sample and samples Similarity tags, This indicates that both are samples of the same category. This indicates that the two are samples of different categories; the feature contrast loss is used to optimize the feature separability between categories. By constructing pairs of similar and dissimilar samples, it calculates the relative distance between their features, bringing similar samples closer and dissimilar samples further apart, thereby improving the overall discriminative ability of the feature space. The total loss is defined as: ； wherein, , , are each loss weights, determined by cross-validation.

8. The surface defect detection method based on metric meta-learning according to claim 7, characterized in that, The step 3.5 includes the following sub-steps: Step 3.5.1: Sample multiple small tasks from the training data, and each task is composed of a support set and a query set; Step 3.5.2: For each task, calculate the loss using the support set, update the parameters according to the gradient descent, and the expression is: ； in, The learning rate hyperparameter controls the step size for each gradient update. For parameters Find the gradient. For mission losses; Step 3.5.3: On the basis of the update results of each task, average the gradient to update the initial parameters, and the expression is: ； wherein, is a global learning rate hyperparameter for cross-task meta-update; Step 3.5.3: Bring the updated initial parameters into step 3.5.2 for multiple iterations, so that the model parameters have good rapid adaptability.

Images