Small sample defect classification method based on double-branch distillation and task adaptive meta-learning

Abstract

The invention provides a small sample defect classification method based on double-branch distillation and task adaptive meta-learning, and the method comprises the steps: obtaining data, carrying out the preprocessing of the data, and obtaining a global image and a local image; performing internal loop training on the model, performing input training on global and local images through a double-branch feature extractor and a classifier to obtain a double-branch prediction result, calculating global loss and distillation loss, generating a self-adaptive task state by using a feature extractor parameter and the distillation loss, calculating element learning loss according to the task state, and calculating element learning loss according to the task state; updating the parameters of the double-branch feature extractor of the model through weighted loss; performing model outer cycle training, using meta learning loss accumulated by multiple inner cycles as a meta target, and updating parameters of meta learning branches; the small sample defect classification method based on double-branch distillation and task adaptive meta-learning shows strong generalization performance in the aspect of defect classification tasks which are unknown in defect category and need to be quickly adapted, and provides a new thought for realizing defect classification in a scene with rare data sets.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial defect classification, and particularly relates to a small sample defect classification method based on double-branch distillation and task adaptive meta-learning. BACKGROUND

[0002] With the development of intelligent manufacturing, the importance of industrial defect classification technology in fields such as manufacturing, automobiles, aerospace, etc. is increasingly prominent. In the field of industrial defect classification, due to the variety of defects generated in the actual use of products and the overall scarcity of samples, existing defect classification systems often lack sufficient rapid adaptation and generalization performance, and are insufficient to accurately identify and classify various defects.

[0003] Specifically, one of the core challenges in industrial defect classification systems is how to learn a feature representation with strong generalization ability based on limited defect samples, especially in the case of extremely unbalanced defect class distribution in each task. Existing methods usually rely on a large amount of labeled data to improve detection accuracy, which is not only costly and time-consuming in actual industrial scenarios, but also not feasible. At the same time, traditional methods often perform poorly when facing distribution differences between different data domains, such as differences caused by production lines, materials or environmental conditions, further exposing the shortcomings of their generalization ability and adaptation mechanism.

[0004] In order to solve the above problems, meta-learning technology has been gradually introduced into the field of industrial defect classification. Meta-learning enables the model to learn how to more effectively learn new tasks, so that it can quickly adapt and perform well when facing a small number of samples or completely new tasks. However, there is still a lack of effective model design in terms of how to efficiently extract and how to effectively use defect classification task meta-features to achieve model optimization, which limits the application of meta-learning in the field of industrial defect classification. Therefore, an innovative method is needed that can accurately learn defect task meta-features, taking into account the task adaptation ability and generalization ability, to meet the complex needs of the actual classification process of industrial defects. SUMMARY

[0005] To solve the problems existing in the prior art, the present application provides a small sample defect classification method based on double-branch distillation and task adaptive meta-learning, which comprises: obtaining defect data generated in the industrial production process, and preprocessing the defect data; inputting the preprocessed defect data into an image defect detection model to obtain a defect detection result;

[0006] Training an image defect detection model includes: acquiring a defect dataset; preprocessing the images in the dataset to obtain global branch data; cropping the global branch data to obtain local branch data; inputting the global and local branch data into a two-branch network for forward propagation to obtain global branch class prototypes, local branch class prototypes, global prediction results, and local prediction results; calculating cross-entropy loss and triplet loss based on the global branch class prototypes and global prediction results; calculating self-image distillation loss and inter-feature distillation loss based on the global branch class prototypes, local branch class prototypes, global prediction results, and local prediction results; acquiring model parameters; inputting the model parameters, self-image distillation loss, and inter-feature distillation loss into a task state generator to generate a task state vector; inputting the task state vector into a meta-learning network to obtain the meta-learning loss; weightedly fusing the meta-learning loss, cross-entropy loss, triplet loss, and distillation loss to obtain the total model loss; updating the model parameters according to the total model loss function; repeating the above process until the total model loss function converges, completing the model training.

[0007] The beneficial effects of this invention are:

[0008] This invention employs a dual-branch feature extraction architecture to extract dual-branch features from task images, improving the expressive power of the data input. A modified loss function, using self-image distillation loss and inter-feature distillation loss, enhances the model's multi-scale learning ability, while meta-learning loss improves the model's feature learning capability. This invention uses a prototype network as a classifier, learning an embedding function that maps the input to a metric space, enabling the model to summarize task-specific prototype vectors for different tasks, thus improving the model's task adaptability. This invention uses a meta-learning method to generate loss, fusing the dual-branch feature extractor parameters with the distillation loss to generate a task state vector. This meta-learning network generates a meta-learning loss, using the accumulated meta-learning loss from multiple tasks in the inner loop as the meta-objective to update the parameters of the meta-learning branches, improving the model's cross-task generalization ability. Attached Figure Description

[0009] Figure 1 This is an overall flowchart of the present invention;

[0010] Figure 2 This is a flowchart of the industrial classification method of the present invention;

[0011] Figure 3 This is a training result diagram of an embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] A few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning, such as Figure 1 As shown, the method includes: acquiring defect data generated during industrial production and preprocessing the defect data; inputting the preprocessed defect data into an image defect detection model to obtain defect detection results; training the image defect detection model includes: acquiring a defect dataset, preprocessing the images in the dataset to obtain global branch data; cropping the global branch data to obtain local branch data; inputting the global branch data and local branch data into a dual-branch network for forward propagation to obtain global branch class prototypes, local branch class prototypes, global prediction results, and local prediction results; and calculating the intersection of the global branch class prototypes and global prediction results. Cross-entropy loss and triplet loss are calculated; self-image distillation loss and inter-feature distillation loss are calculated based on the global branch class prototype, local branch class prototype, global prediction result, and local prediction result; model parameters are obtained, and the model parameters, self-image distillation loss, and inter-feature distillation loss are input into the task state generator to generate a task state vector; the task state vector is input into the meta-learning network to obtain the meta-learning loss; the meta-learning loss, cross-entropy loss, triplet loss, and distillation loss are weighted and fused to obtain the total model loss; the model parameters are updated according to the total model loss function; the above process is repeated until the total model loss function converges, completing the model training.

[0014] like Figure 1 The diagram shows the overall framework of a few-sample defect classification method based on bi-branch distillation and task-adaptive meta-learning, including a global branch, a local branch, and a meta-learning branch.

[0015] like Figure 2 As shown, a few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning specifically includes the following steps:

[0016] S1. Two-branch data preparation: This embodiment defines a two-branch learning process for industrial small-sample data scenarios, including the data used, the data categories and quantities set, the size and number of local branch images set, and the model's hyperparameters set. Regarding data preparation and preprocessing, defect data from the industrial production process, including image data, is collected and preprocessed through standardization, normalization, and cropping to generate a two-branch dataset suitable for model training.

[0017] S2, Forward Propagation and Two-Branch Loss Calculation: Supports using a two-branch set of data to obtain the two-branch class prototype through a feature extractor and classifier; and queries using a two-branch set of data to obtain the two-branch prediction result through a feature extractor and classifier.

[0018] The feature extractor employs a ResNet18 network with fully connected layers removed. The global image is input into this network to obtain global feature extraction results; similarly, local images are input to the same network to obtain local feature extraction results. The classifier uses a prototype network. The global and local feature extraction results for the support set are input into this prototype network to obtain global and local prototypes, respectively; the global and local feature extraction results for the query set are also input into this prototype network to obtain global and local prediction results, respectively.

[0019] Specifically, processing the input data using a ResNet18 network with fully connected layers removed involves: removing the pooling and fully connected layers at the ends of the ResNet18 network and adding adaptive pooling layers. The data is then input into the modified ResNet18 network to obtain the feature extraction results.

[0020] The prototype network processes global and local features separately, including: the prototype network consists of a prototype calculation module and a distance classification module; the prototype calculation module calculates the mean of the global and local feature extraction results of the support set to obtain the global prototype and the local prototype; the distance classification module calculates the Euclidean distance between the global and local feature extraction results of the query set and the corresponding prototype vector, and converts the distance into a probability distribution through the softmax function to obtain the global prediction result and the local prediction result.

[0021] Cross-entropy loss and triplet loss are calculated using the global prediction results, and self-image distillation loss and inter-feature distillation loss are calculated using the global and local prediction results. The specific formulas for calculating the loss functions are as follows:

[0022]

[0023]

[0024]

[0025]

[0026] in, The cross-entropy loss function; The loss function is for triples; The loss function is the self-image distillation function. The feature-inter-feature distillation loss function; Calculate the KL divergence; Number of local view images; ; One-hot encoding for category c; To query the global prediction results for the set; The probability of category c in the global prediction results of the query set; To retrieve the m-th local prediction result of the query set; Support a global prototype for category c; For category c, support the m-th local prototype; The global features obtained from global samples in the query set using a feature extractor; A global prototype of a support set that is of the same category as the global samples in the query set; A global prototype of a support set for categories different from the global samples of the query set; is the boundary hyperparameter of the triplet.

[0027] S3. Task state generation and meta-learning loss calculation: Using a task state generator, the norm of the weights of each layer of the feature extractor, the global prediction results and label distribution obtained by the classifier, and the self-image distillation loss and inter-feature distillation loss are combined to form a task state vector. The task state vector is then input into the meta-learning network for processing to obtain the meta-learning loss.

[0028] The process of inputting the task state vector into the meta-learning network includes: the meta-learning network consists of a meta-loss weight adjustment network and a dynamic loss generation network; the meta-loss weight adjustment network uses a multilayer perceptron and is implemented with two fully connected layers; the meta-loss weight adjustment network generates a dynamic loss generation network that adjusts the parameters based on the task state vector; the dynamic loss generation network uses a multilayer perceptron and is implemented with two fully connected layers; the task state vector and the adjusted parameters are input into the dynamic loss generation network to obtain the meta-learning loss.

[0029] Specifically, the generation process of meta-learning loss is as follows:

[0030] Step 1: The meta-loss weight adjustment network uses a multilayer perceptron (MLP) and consists of two fully connected layers. It generates a dynamic loss function based on the task state vector, and then adjusts the network parameters. , .

[0031] Step 2: Adjust the dynamic loss generation network parameters using the adjustment parameters. The formula is:

[0032]

[0033] in, Generate network parameters for dynamic loss.

[0034] Step 3: The dynamic loss generation network uses a multilayer perceptron, implemented with two fully connected layers, and uses adjusted parameters. The meta-learning loss is calculated by inputting the task state vector.

[0035] S4. Model training and parameter update: The total loss of the model data is obtained by weighting the cross-entropy loss, triplet loss, self-image distillation loss, inter-feature distillation loss, and meta-learning loss. In the inner loop, the weighted total loss is backpropagated to update the parameters of the global branch feature extractor and the parameters of the local branch feature extractor are updated using the exponential moving average mechanism. In the outer loop, the meta-learning loss accumulated in the inner loop is used as the meta-objective and the parameters of the dynamic loss generation network are updated through backpropagation.

[0036] The specific formula for the local branch exponential moving average mechanism of the model is as follows:

[0037]

[0038] in, These are the parameters for the global branch feature extractor. These are the parameters for the local branch feature extractor. This is the EMA update factor.

[0039] S5. Model Deployment and Inference: Based on the test set sample data, the model's global branch is used for classification tasks.

[0040] like Figure 3 As shown, the specific few-shot learning task uses the public dataset NEU_CLS, and the experimental results compared with MatchingNet, MAML, RelationNet, and ProtoNet are presented in [the table / document / etc.]. Figure 3 The last point is the method proposed in this invention. It can be seen that the accuracy is significantly improved after running the small sample defect classification method based on bibranch distillation and task adaptive meta-learning of this invention.

[0041] This invention employs a small-sample defect classification method based on bibranch distillation and task-adaptive meta-learning, which effectively solves the problems of scarce data, insufficient model generalization ability, and limited computing resources in industrial small-sample scenarios. It provides an efficient and accurate industrial defect classification method that can improve the accuracy and robustness of industrial defect classification and is applicable to various intelligent manufacturing and industrial defect classification scenarios.

[0042] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning, characterized in that, include: Acquire defect data generated during industrial production and preprocess the defect data; The preprocessed defect data is input into the image defect detection model to obtain the defect detection results; Training an image defect detection model includes: acquiring a defect dataset; preprocessing the images in the dataset to obtain global branch data; cropping the global branch data to obtain local branch data; inputting the global and local branch data into a two-branch network for forward propagation to obtain global branch class prototypes, local branch class prototypes, global prediction results, and local prediction results; calculating cross-entropy loss and triplet loss based on the global branch class prototypes and global prediction results; calculating self-image distillation loss and inter-feature distillation loss based on the global branch class prototypes, local branch class prototypes, global prediction results, and local prediction results; acquiring model parameters; inputting the model parameters, self-image distillation loss, and inter-feature distillation loss into a task state generator to generate a task state vector; inputting the task state vector into a meta-learning network to obtain the meta-learning loss; weightedly fusing the meta-learning loss, cross-entropy loss, triplet loss, and distillation loss to obtain the total model loss; updating the model parameters according to the total model loss function; repeating the above process until the total model loss function converges, completing the model training.

2. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 1, characterized in that, Preprocessing defective data includes data augmentation and standardization of the acquired image data.

3. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 1, characterized in that, The forward propagation of the dual-branch network on the input image includes: inputting global branch data and local branch data into the feature extractor for feature extraction; inputting the extracted global features and local features into the corresponding prototype network to obtain the global prototype, local prototype, global prediction result, and local prediction result.

4. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 3, characterized in that, The feature extractor performs feature extraction on global branch data and local branch data respectively. The feature extractor uses a ResNet18 network with the fully connected layers removed. The global image is input into the ResNet18 network with the fully connected layers removed to obtain the global feature extraction result. The local image is input into the ResNet18 network with the fully connected layers removed to obtain the local feature extraction result.

5. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 3, characterized in that, The prototype network processes global and local features separately, including: the prototype network consists of a prototype calculation module and a distance classification module; the prototype calculation module calculates the mean of the global and local feature extraction results of the support set to obtain the global prototype and the local prototype; the distance classification module calculates the Euclidean distance between the global and local feature extraction results of the query set and the corresponding prototype vector, and converts the distance into a probability distribution through the softmax function to obtain the global prediction result and the local prediction result.

6. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 1, characterized in that, The self-image distillation loss is: in, The loss function is the self-image distillation function. Calculate the KL divergence; Number of partial view images; To query the global prediction results for the query set; To retrieve the m-th local prediction result of the query set; The characteristic inter-distillation loss is: in, The feature-inter-feature distillation loss function; Calculate the KL divergence; Number of partial view images; The number of tasks categorized; Support a global prototype for category c; For category c, support the m-th local prototype.

7. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 1, characterized in that, The task state generator processes the input data by combining the norms of the weights of each layer of the feature extractor, the global prediction results and label distribution obtained by the classifier, and the self-image distillation loss and the inter-feature distillation loss to form a task state vector.

8. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 1, characterized in that, The process of inputting the task state vector into the meta-learning network includes: the meta-learning network consists of a meta-loss weight adjustment network and a dynamic loss generation network; the meta-loss weight adjustment network uses a multilayer perceptron and is implemented by two fully connected layers; the meta-loss weight adjustment network generates a dynamic loss generation network that adjusts the parameters based on the task state vector; the dynamic loss generation network uses a multilayer perceptron and is implemented by two fully connected layers; the task state vector and the adjusted parameters are input into the dynamic loss generation network to obtain the meta-learning loss.

9. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 1, characterized in that, The total loss of the model is: in, Cross-entropy loss; For triplet loss; For image distillation loss, Characteristic inter-distillation loss; This is the meta-learning loss; - All of these are balanced hyperparameters.

10. The few-sample defect classification method based on bibranch distillation and task-adaptive meta-learning according to claim 1, characterized in that, Updating model parameters based on the total loss function includes: in the inner loop, backpropagating the weighted total loss to update the global branch feature extractor parameters; using an exponential moving average mechanism to update the local branch feature extractor parameters; and in the outer loop, using the meta-learning loss accumulated in the inner loop as the meta-objective, updating the parameters of the dynamic loss generation network through backpropagation.

Images