Industrial defect classification open set identification method based on federal transfer learning

Abstract

The invention discloses an industrial defect classification open-set identification method based on federal transfer learning, and belongs to the technical field of federal learning and industrial defect classification, and the method comprises the steps: preparing data, and constructing a comprehensive negative sample data set; the training client performs input training on the data to obtain a corresponding classification model and an unknown category threshold; updating and uploading the model; carrying out weight calculation and aggregation on the model and the threshold value; and model deployment and inference: the target client receives the global model sent by the server, and the global model is used for defect classification of the to-be-detected data. According to the industrial defect classification open set identification method based on federal transfer learning, the classification capability of an unknown category is learned under the scene that the unknown category of data classification exists in the multi-source domain and the to-be-detected data, the classification capability is applied to the target domain user, the model and the threshold value are jointly trained in cooperation with multiple participants, and the classification efficiency of the to-be-detected data is improved. And under the condition that the data volume is limited and the computing resources are limited, the generalization ability and accuracy of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning and industrial defect classification, and in particular to a solution for accurately classifying data using federated transfer learning. Background Art

[0002] With the rapid development of artificial intelligence and machine learning, open set recognition has gradually become an important research direction in the field of computer vision and pattern recognition. In traditional classification tasks, the model assumes that the training and test data come from the same set of categories. However, in practical applications, especially in dynamic environments, the model may encounter unseen categories, which requires the ability of open set recognition. Open set recognition aims to enable the model to make reasonable judgments when encountering unknown categories to avoid misclassification.

[0003] There are many types of defects in the production process of industrial products, and they are difficult to predict. Existing defect classification systems often have difficulty accurately identifying and classifying various defects, including unknown defect categories. In these systems, a major technical challenge is how to learn the classification capabilities of unknown categories in multiple source domains and apply them to target domain users. Traditional machine learning and deep learning methods cannot distinguish unknown defect category data in this case, resulting in inaccurate classification.

[0004] Existing methods in this context often rely on a large amount of labeled data to improve detection accuracy, which is impractical in practical applications because it is costly, time-consuming and labor-intensive to obtain a large amount of labeled defect data. Traditional classification models can usually only process samples of known categories that have appeared in the training set, and cannot make correct judgments for samples of unknown categories. Open set recognition technology aims to identify and process samples of unknown categories that are not included in the training set. While identifying known categories, it can effectively detect and reject samples of unknown categories to prevent them from being misclassified as a known category. In actual industrial production, the defect types of products may be constantly updated and changed, and the system needs to have the ability to identify newly emerging defect types. Open set recognition technology can help improve the robustness and adaptability of defect detection systems to cope with the ever-changing production environment.

[0005] In order to solve the above problems, federated transfer learning technology and open set recognition technology have gradually been introduced into the field of industrial defect detection. Federated transfer learning collaborates with multiple participants to jointly train models and thresholds without sharing original data, and then draws on the knowledge of existing models to improve the model performance on the target task. Open set recognition technology aims to identify and process samples of unknown categories included in the training set. However, in problem scenarios with multiple source domains and unknown categories, how to effectively combine federated transfer learning technology and open set recognition technology to achieve cross-domain knowledge transfer and model optimization is still a difficult problem that needs to be solved. Summary of the invention

[0006] The purpose of the present invention is to provide an open set identification method for industrial defect classification based on federated transfer learning. In the face of multi-source domains and test data with unknown category data classification problems, the classification ability of unknown categories is learned in multiple source domains and applied to target domain users. Without sharing the original data, multiple participants are coordinated to jointly train models and thresholds, so as to improve the generalization ability and accuracy of the model under the condition of limited data volume and limited computing resources.

[0007] To achieve the above object, the present invention provides an open set identification method for industrial defect classification based on federated transfer learning, comprising the following steps:

[0008] S1. Data preparation: collect defect data generated in the local industrial production process, train the client to use the negative guide to generate negative sample data from the defect data, and perform data processing simultaneously;

[0009] S2, local model training, the training client inputs the processed data in S1 for training until the local model converges, solves the unknown categories in the data, and obtains the unknown category classification threshold;

[0010] S3, model update and upload, the training client uploads the trained model parameters, unknown category classification thresholds and local data feature data sets to the server;

[0011] S4, model weight calculation and aggregation, the server receives the model parameters, local data features, unknown category classification thresholds uploaded by the training client, and the second data features uploaded by the target client, measures the difference between the local data features and the second data features, and performs weighted aggregation on the model parameters of the training client and the unknown category classification threshold to obtain a global model and a global threshold;

[0012] S5. Model deployment and inference. The target client receives the global model and global threshold sent by the server to classify the industrial defects of the data to be tested.

[0013] Preferably, in S1, the training client uses a negative guide to generate negative sample features for the collected defect data, cooperates with GAN to generate negative sample data, mixes the generated negative sample data and defect data for balanced training, combines the external data set background to generate a comprehensive negative sample data set, and adopts an open set recognition method for training.

[0014] Preferably, the negative sample data generation comprises the following steps:

[0015] S101, extracting features of the defect data using a feature extractor, converting the features into feature vectors, and performing an inversion operation on some of the feature vectors;

[0016] S102, the negative guide uses data enhancement and random selection strategy training to generate negative sample data with different characteristics.

[0017] Preferably, in S4, the difference metric includes calculating a model weight of a single training client and calculating each threshold weight.

[0018] Preferably, the model weight calculation is specifically to measure the MMD distance between the local data feature data set and the second data feature data set, and calculate the model weight of a single training client.

[0019] Preferably, the calculation of each threshold weight is specifically to measure the Wasserstein distance between the local data feature data set and the second data feature data set to calculate each threshold weight.

[0020] Therefore, the present invention adopts the above-mentioned open set identification method for industrial defect classification based on federated transfer learning. When faced with the problem scenario of multi-source domains and unknown category data classification in the test data, the classification ability of unknown categories is learned in multiple source domains and applied to the target domain users. Without sharing the original data, multiple participants are coordinated to jointly train the model and threshold, so as to improve the generalization ability and accuracy of the model under the condition of limited data amount and limited computing resources.

[0021] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is an overall framework diagram of an embodiment of an industrial defect classification open set identification method based on federated transfer learning of the present invention;

[0023] Figure 2 It is a flowchart of the steps of an industrial open set classification method of an industrial defect classification open set identification method based on federated transfer learning in the present invention. DETAILED DESCRIPTION

[0024] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0025] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] Embodiment 1

[0027] The present invention provides an open set identification method for industrial defect classification based on federated transfer learning, and its overall framework is shown in the following figure: Figure 1 As shown, it includes a training client, a target client and a server, wherein the training client and the server are connected for communication, and the target client transmits data features to the server. Figure 2 This is a flowchart of the steps of the industrial open set classification method. In this embodiment, the federated transfer learning process is defined in the problem scenario of multiple source domains and unknown categories, including the data used, the data category and data quantity set by each user, the different data sets for the training client and the target client, the hyperparameters of the model, etc. The specific process includes the following steps:

[0028] S1. Data preparation and preprocessing. Each training client device collects defect data in the industrial production process locally, including image data, and performs preprocessing operations such as standardization and normalization. Use a negative guide to generate negative sample data, use GAN to generate a negative sample balanced data set, and then combine the external data set as the background to generate a comprehensive negative sample, and perform data processing to generate a data set for model training.

[0029] The process of generating negative samples is to first use the feature extractor to extract the vector of the data, select some features to be inverted, and let the feature extractor be Θ, then the feature vector is expressed as: f i =Θ(x i ). Perform the inversion operation on the feature vector to obtain the feature opposite to the known category. The feature inversion formula is:

[0030]

[0031] Use the negative guide to get negative samples and modify the features. The corresponding formula is:

[0032]

[0033] Get the corresponding negative sample features, and then use GAN to generate negative samples. The corresponding discriminator loss function is:

[0034]

[0035] The loss function of the generator is:

[0036]

[0037] Use GAN to generate negative samples, and then combine them with external datasets as background images to generate a comprehensive negative sample dataset.

[0038] S2, local model training, the training client inputs the data processed in step S1 for training, extracts feature vectors through feature extractor, x represents data, f(x) represents the extracted feature vector, selects DNN model for training, and defines cross entropy loss function. The specific formula for calculating loss function is as follows:

[0039]

[0040] L CE Expressed as the cross entropy loss function, Indicates the predicted category, y c Represents the true category, and the optimizer updates the model parameters. The specific formula is as follows:

[0041]

[0042] Where W is the model parameter, b is the bias term, and η is the learning rate. The local model is obtained until it converges. At the same time, the threshold is calculated. For each known category data, the center point of the category in the feature space is calculated. Let c k represents the class center of class k, X k Represents the set of all feature vectors of category k, and the class center calculation formula is:

[0043]

[0044] Calculate the distance from each input feature vector f(x) to the category center and the corresponding Euclidean distance. The calculation formula is:

[0045] d(f(x),c y )=‖f(x)-c y ‖2;

[0046] where d(f(x),c y ) represents the distance from the input sample to the category center, c yrepresents the distance from the input sample to the category center, and f(x) represents each feature vector.

[0047] Then calculate the mean and standard deviation of the category, and use the mean plus standard deviation method to set the threshold. The formula for calculating the threshold is:

[0048] T k =μ k +λσ k ;

[0049] where μ k represents the mean distance of category k, σ k It represents the standard deviation of the distance of category k, and λ is a hyperparameter that adjusts the dynamic change of the threshold.

[0050] S3, model update and upload, each training client device uploads the locally trained model parameters, threshold parameters and local data features to the server. The uploaded parameters are model parameter information, and the data features include feature vectors and statistical information of the data.

[0051] S4, model weight calculation and model update, the server receives the model parameters, threshold parameters and data features uploaded by each training client, as well as the data features uploaded by the target client. According to the data features of the training client and the target client, the MMD distance and Wasserstein distance are calculated.

[0052] Among U source domain training clients, each client provides data Upload the local threshold and model, then calculate the Wasserstein distance based on the data features to perform threshold aggregation, and then calculate the MMD distance to perform model aggregation. Aggregate the model and threshold to obtain the global model and global threshold, and deploy them to the target domain accordingly. For each source domain training client, use the formula:

[0053]

[0054] Calculate the data for each source domain training client and the target client's data D T The MMD distance between them is set as X = {x1, x2, ..., x n} is the data set The sample set in each client has n samples, Y = {y1, y2, ..., y m} is the dataset D T There are m samples in the sample set. Among them, k(x,y) uses the Gaussian kernel function σ is the bandwidth parameter of the kernel function.

[0055] The corresponding formula is:

[0056]

[0057] Calculate the data for each source domain training client and the target client's data D T Wasserstein distance between them. Where Γ: all possible To D T A collection of transportation plans, each of which is a matrix whose elements γ ij Indicates that from x i Transfer to y j The probability mass of .

[0058] According to the obtained distance, the weights corresponding to the model and the threshold are calculated. The specific expression for model weight calculation is:

[0059]

[0060] Among them, α k is the weight of each model, MMD k It measures the MMD distance between the training client and the target client data set, and obtains the weight of each model based on the different MMD distance calculations. The specific expression for threshold weight calculation is:

[0061]

[0062] Among them, β k is the weight of each threshold, EMD k It measures the Wasserstein distance between the training client and the target client data set, and obtains the weight of each threshold according to the different calculations of the Wasserstein distance.

[0063] The specific expression of global aggregation is as follows:

[0064]

[0065] Among them, α k is the weight of each model, is the model of each training client, β k is the weight of each threshold, is the threshold for each training client.

[0066] According to the weights of each model and threshold required above, the server performs weighted aggregation on the models and thresholds of all training clients to obtain the global model and global threshold. Using the weighted aggregation method, the model is fused according to the weights of each training client.

[0067] S5, Model deployment and inference, the target client receives the global model and global threshold sent by the server, and applies them to the data to be tested for industrial defect classification. The target client uses the updated global model to classify defects in the data in the actual production environment and obtains the classification results.

[0068] Therefore, the present invention adopts the above-mentioned open set identification method for industrial defect classification based on federated transfer learning. When faced with the problem scenario of multi-source domains and unknown category data classification in the test data, the classification ability of unknown categories is learned in multiple source domains and applied to the target domain users. Without sharing the original data, multiple participants are coordinated to jointly train the model and threshold, so as to improve the generalization ability and accuracy of the model under the condition of limited data amount and limited computing resources.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. An open set identification method for industrial defect classification based on federated transfer learning, characterized by: The following steps are involved: S1. Data preparation: collect defect data generated in the local industrial production process, train the client to use the negative guide to generate negative sample data from the defect data, and perform data processing simultaneously; S2, local model training, the training client inputs the processed data in S1 for training until the local model converges, solves the unknown categories in the data, and obtains the unknown category classification threshold; S3, model update and upload, the training client uploads the trained model parameters, unknown category classification thresholds and local data feature data sets to the server; S4, model weight calculation and aggregation, the server receives the model parameters, local data features, unknown category classification thresholds uploaded by the training client, and the second data features uploaded by the target client, measures the difference between the local data features and the second data features, and performs weighted aggregation on the model parameters of the training client and the unknown category classification threshold to obtain a global model and a global threshold; S5. Model deployment and inference. The target client receives the global model and global threshold sent by the server to classify the industrial defects of the data to be tested.

2. The method for industrial defect classification and open set identification based on federated transfer learning according to claim 1 is characterized by: In S1, the training client uses a negative guide to generate negative sample features for the collected defect data, cooperates with GAN to generate negative sample data, mixes the generated negative sample data and defect data for balanced training, combines the external data set background to generate a comprehensive negative sample data set, and adopts an open set recognition method for training.

3. The method for industrial defect classification and open set identification based on federated transfer learning according to claim 2 is characterized by: The negative sample data generation comprises the following steps: S101, extracting features of the defect data using a feature extractor, converting the features into feature vectors, and performing an inversion operation on some of the feature vectors; S102, the negative guide uses data enhancement and random selection strategy training to generate negative sample data with different characteristics.

4. The method for industrial defect classification and open set identification based on federated transfer learning according to claim 1 is characterized in that: In S4, the difference measurement includes calculating the model weight of a single training client and calculating each threshold weight.

5. The method for industrial defect classification and open set identification based on federated transfer learning according to claim 4 is characterized in that: The model weight calculation is specifically to measure the MMD distance between the local data feature data set and the second data feature data set, and calculate the model weight of a single training client.

6. The method for industrial defect classification and open set identification based on federated transfer learning according to claim 4 is characterized in that: The calculation of each threshold weight is specifically to measure the Wasserstein distance between the local data feature data set and the second data feature data set, and calculate each threshold weight.

Images