An industrial defect classification method based on federated transfer learning

Abstract

The present invention discloses an industrial defect classification method based on federated transfer learning, which belongs to the technical field of federated learning and industrial defect classification, including data preparation, a training client locally collects and preprocesses defect data generated in the industrial production process, and performs data processing; local model training, the training client inputs the data processed by step S1 through a high-low dual-channel multi-head attention mechanism module until the local model converges; model update and upload; model weight calculation and aggregation; model deployment and inference, the target client receives the global model sent by the server, and is used for industrial defect classification of the data to be tested. The present invention adopts the above-mentioned industrial defect classification method based on federated transfer learning, and exhibits strong transfer generalization performance in terms of protecting privacy and processing defect classification tasks with imbalanced small sample categories in multi-source domains, providing a new idea for defect classification in scenarios where data sets are scarce and privacy protection needs to be enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning and industrial defect classification, and in particular to an industrial defect classification method based on federated transfer learning. Background Art

[0002] With the advancement of Industry 4.0 and the development of intelligent manufacturing, the importance of industrial defect classification technology in fields such as manufacturing, automobiles, aerospace, etc. is becoming increasingly apparent. In the field of industrial defect classification, existing technologies face multiple challenges, especially in the case of small samples in multiple source domains and class imbalance. Due to the wide variety of defect types generated in the production process of industrial products and their difficulty in prediction, existing defect classification systems often have difficulty in accurately identifying and classifying various defects. In these systems, a major technical challenge is how to learn effective feature representations from a small number of samples collected from multiple source domains, especially when the defect categories in each source domain are unevenly distributed. In addition, since the number of defect samples is usually small and there is a significant class imbalance, that is, some defect categories are abundant while others are extremely scarce, traditional machine learning and deep learning methods in this case often cause the classifier to be biased towards the majority class, thereby ignoring the accurate classification of the minority class.

[0003] Existing methods in this context often rely on a large amount of annotated data to improve detection accuracy, which is impractical in practical applications because it is costly, time-consuming and labor-intensive to obtain a large amount of annotated defect data. Moreover, even if sufficient data is obtained, traditional methods are usually not effective in dealing with differences between different data domains because they cannot effectively adapt to changes in data distribution caused by changes in production lines, materials or environmental conditions.

[0004] In order to solve the above problems, federated learning and transfer learning technologies have gradually been introduced into the field of industrial defect detection. Federated learning protects data privacy by coordinating multiple participants to jointly train models without sharing original data. Transfer learning improves the performance of models on target tasks by drawing on the knowledge of existing models. However, in the scenario of class imbalance of small samples in multiple source domains, how to effectively combine federated learning and transfer learning technologies to achieve cross-domain knowledge transfer and model optimization is still a difficult problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide an industrial defect classification method based on federated transfer learning to solve the problems existing in the above-mentioned background technology. In the face of a multi-source domain small sample category imbalance scenario, the knowledge of clients trained in different source domains is migrated to the target client, so that the target client can realize data classification under the premise of ensuring data privacy in the current scenario.

[0006] To achieve the above object, the present invention provides an industrial defect classification method based on federated transfer learning, comprising the following steps:

[0007] S1. Data preparation: training the client to collect and preprocess the defect data generated in the industrial production process locally, and perform data processing, including standardizing the collected image data and applying data enhancement technology to increase the data processing process and divide the data into high and low frequency data;

[0008] S2, local model training, the training client inputs the data processed in step S1 through the high-low dual-channel multi-head attention mechanism module until the local model converges;

[0009] S3, model update and upload, the training client uploads the trained model parameters and local data features to the server;

[0010] S4, model weight calculation and aggregation. The server receives the model parameters and data features uploaded by the training client and the data features uploaded by the target client. According to the data features of the training client and the target client, multi-scale difference measurement is performed to measure the difference between the two data feature data sets in statistical consistency and directional consistency. The JS distance and cosine distance are calculated based on the difference. The measured difference distances of different scales are fused, and the model weight of each training client is calculated based on the multi-scale difference. The models of all training clients are weighted and aggregated to obtain a global model.

[0011] The specific expression for model weight calculation is:

[0012]

[0013] Among them, α k is the weight of each model, MD k It is the fusion distance of the statistical difference and directional difference distance of the multi-scale metric training client and the target client data set. The weight of each model is obtained according to the multi-scale metric distance calculation.

[0014] The specific expression of the global model aggregation is as follows:

[0015]

[0016] Among them, α k is the weight of each model, It is the model of each training client. Each model is weighted and aggregated according to different weights to obtain the global model.

[0017] S5. Model deployment and inference. The target client receives the global model sent by the server and uses it to classify industrial defects on the data to be tested.

[0018] Preferably, the data preparation in step S1 is to pre-process the data collected by the training client, and after data enhancement of the collected images, divide them into two data sets of high resolution and low resolution.

[0019] Preferably, the local model training of step S2 is specifically as follows: the training client locally uses the high-low dual-channel multi-head attention mechanism module to input the data processed by step S1 as a training data set for training; the obtained output is input to the classifier, and the loss function is obtained according to the result of the classifier, and the loss function is corrected to a form that combines cross entropy loss and dynamic center loss to optimize the model; the training data set is cyclically trained until the local model converges.

[0020] Preferably, the loss function is modified to a form combining cross entropy loss and dynamic center loss, specifically:

[0021] The cross entropy loss function is calculated from the prediction results obtained from the classifier to measure the difference between the model prediction results and the true labels;

[0022] Calculate the dynamic center loss function to measure the distance between sample features and category centers;

[0023] The cross entropy loss function and the dynamic center loss function are combined to obtain the corrected total loss function.

[0024] Preferably, the expression of the dynamic center loss function is:

[0025]

[0026] Among them, L DC is the dynamic center loss function; f i is a feature representation; It is a dynamic center class;

[0027] The expression of the total loss function is:

[0028] L total =L CE +λL DC

[0029] Among them, L CE is the cross entropy loss function; λ is the balancing hyperparameter.

[0030] Preferably, the high-low dual-channel multi-head attention mechanism module in step S2 includes: the high-resolution and low-resolution data sets are divided into data for feature extraction through a dual-channel feature extractor, the high-resolution data focuses on local features, and the low-resolution data focuses on global features, and then the two features are fused and input into the multi-head attention mechanism module.

[0031] Preferably, the model weight calculation and aggregation in step S4 are specifically as follows:

[0032] According to the data features of the training client and the target client, the difference between the data features of the two clients is measured at multiple scales. The difference in statistical consistency of the two data feature data sets is measured by calculating the JS distance of the two data feature data sets. The difference in directional consistency of the two data feature data sets is measured by calculating the cosine distance.

[0033] The multi-scale differences are aggregated, and the model weight of each training client is calculated according to the different difference distances of each training client. According to the calculated model weight of each training client, the models of each training client are weighted aggregated, and the weighted average model parameters are used as the global model parameters to update the global model.

[0034] Therefore, the present invention adopts the above-mentioned industrial defect classification method based on federated transfer learning, which has the following beneficial effects:

[0035] (1) Through federated learning, each training client only uploads model parameters and data features, not raw data, thereby protecting data privacy and innovation;

[0036] (2) The high-low dual-channel multi-head attention mechanism module is adopted to improve the expressiveness of data input. The modified loss function is used, combined with cross entropy loss and dynamic center loss to enhance the learning ability of the model. The input sequence format in model training is reconstructed, and the causal mask in the attention mechanism is corrected, so that the model can better complete the classification task;

[0037] (3) By calculating the JS distance and cosine distance, the differences between datasets are measured in multiple scales, the model weights are accurately calculated, and the accuracy and robustness of the global model are improved.

[0038] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is an overall framework diagram of an industrial defect classification method based on federated transfer learning of the present invention;

[0040] Figure 2 This is a flowchart of the steps of the industrial classification method of the present invention;

[0041] Figure 3 This is a training result diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Example

[0043] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0044] like Figure 1 As shown, an overall framework diagram of an industrial defect classification method based on federated transfer learning includes a training client, a target client and a server, wherein the training client and the server are connected in communication, and the target client transmits data features to the server.

[0045] like Figure 2 As shown, an industrial defect classification method based on federated transfer learning specifically includes the following steps:

[0046] S1. Data model initialization. In this embodiment, the federated transfer learning process in the scenario of unbalanced data of small samples in multiple source domains is defined, including the data used, the data category and data quantity set by each user, the different data sets for the training client and the target client, the hyperparameters of the model, etc. Regarding data preparation and preprocessing, each training client device collects defect data in the industrial production process locally, including image data, etc., performs preprocessing operations such as standardization and normalization, and generates a data set for model training.

[0047] S2, local model training, each training client device will obtain the fusion data features based on the preprocessed data set Input into the high-low dual-channel multi-head attention mechanism model, decompose the input features into multiple heads h, each head learns a different representation subspace, and for each head h, a different weight matrix is ​​used Perform linear transformation to obtain query, key and value. The specific formula is as follows:

[0048]

[0049] According to the query Q h and key K h Calculate the attention score and use the scaled dot product operation to get the score where d k Yes Key K h The softmax function is applied to the attention score of each head to obtain the standardized attention weight, and the value is weighted summed according to the weight, and all head outputs are combined into the final output feature O, O = Connect (O1, O2, ..., O h), and then linearly transform O through another learnable matrix to obtain the final output feature O f . Input the output into the classifier to get the prediction result The loss function is calculated based on the prediction results. The modified loss function is the cross entropy loss combined with the dynamic center loss function to optimize the model training. The training data set is trained repeatedly until the model converges. The specific formula for calculating the loss function is as follows:

[0050]

[0051] L total =L CE +λL DC

[0052] Among them, L CE Expressed as a cross entropy loss function, M is the total number of defect categories, y c is a single encoding of the true label, is the probability that the model predicts category c, L DC represents the dynamic center loss function, f i represents the feature representation of each sample, N is the total number of training samples, represents the dynamic center class of each category, i represents each sample, and λ is a hyperparameter to balance the two loss functions.

[0053] S3, model update and upload, each training client device uploads the locally trained model parameters and local data features to the server. The uploaded parameters are model parameter information, and the data features include data statistics and feature vectors.

[0054] S4, model weight calculation and model update, the server receives the model parameters and data features uploaded by each training client, as well as the data features uploaded by the target client. According to the data features of the training client and the target client, the difference in statistical consistency and directional consistency between the two data sets is measured, and the JS distance and cosine distance are calculated. The model weight of each training client is calculated based on the difference distance. The specific expressions for calculating the JS distance and cosine distance are as follows:

[0055]

[0056] The two distances are fused to obtain MD, and the weight of each model is calculated based on the multi-scale distance MD. The specific expression is as follows:

[0057]

[0058] According to the weight of each model obtained above, the server aggregates the models of all training clients to obtain the global model. Using the weighted aggregation method, the model is integrated according to the weight of each training client.

[0059] S5, Model deployment and inference, the target client receives the global model sent by the server and applies it to the data to be tested for industrial defect classification. The target client uses the updated global model to classify defects in the data in the actual production environment and obtains the classification results.

[0060] like Figure 3 As shown in Figure 2, the specific migration task is to use the public dataset NEU_CLS as the source domain data and the Severstal dataset as the target domain data. The experimental results obtained by comparing Fed-Avg, FedProx, DWF-A, FTL-ACGAN and BN-MMD methods are shown in Figure 2. Figure 3 The last one is the method proposed by the present invention. From this operation, it can be seen that the accuracy of the industrial defect classification method based on federated transfer learning of the present invention is significantly improved.

[0061] Therefore, the present invention adopts the above-mentioned industrial defect classification method based on federated transfer learning, and effectively solves the problems of scarce data, uneven data distribution, insufficient model generalization ability and limited computing resources in multiple source domain scenarios under the premise of ensuring data privacy. It provides an efficient and accurate industrial defect classification method, which can improve the accuracy and robustness of industrial defect classification under the premise of ensuring data privacy, and is suitable for various intelligent manufacturing and industrial defect classification scenarios.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. An industrial defect classification method based on federated transfer learning, characterized in that: The following steps are involved: S1. Data preparation: training the client to locally collect and pre-process defect data generated in the industrial production process, and perform data processing, including standardizing the collected image data and applying data enhancement technology to increase the data processing process; S2, local model training, the training client inputs the data processed in step S1 through the high-low dual-channel multi-head attention mechanism module until the local model converges; S3, model update and upload, the training client uploads the trained model parameters and local data features to the server; S4, model weight calculation and aggregation. The server receives the model parameters and data features uploaded by the training client and the data features uploaded by the target client. According to the data features of the training client and the target client, multi-scale difference measurement is performed to measure the difference between the two data feature data sets in statistical consistency and directional consistency. The JS distance and cosine distance are calculated based on the difference. The measured difference distances of different scales are fused, and the model weight of each training client is calculated based on the multi-scale difference. The models of all training clients are weighted and aggregated to obtain a global model. S5, model deployment and inference, the target client receives the global model sent by the server and uses it to classify industrial defects in the data to be tested; The local model training of step S2 is specifically as follows: the training client locally uses the high-low dual-channel multi-head attention mechanism module to input the data processed in step S1 as a training data set for training; the obtained output is input into the classifier, and the loss function is obtained according to the result of the classifier, and the loss function is corrected to a form combining cross entropy loss and dynamic center loss to optimize the model; the training data set is cyclically trained until the local model converges; The loss function is modified to combine the cross entropy loss and the dynamic center loss, specifically: The cross entropy loss function is calculated from the prediction results obtained from the classifier to measure the difference between the model prediction results and the true labels; Calculate the dynamic center loss function to measure the distance between sample features and category centers; The cross entropy loss function and the dynamic center loss function are combined to obtain the corrected total loss function; The expression of the dynamic center loss function is: ； in, is the dynamic center loss function; is a feature representation; It is a dynamic center class; The expression of the total loss function is: ； in, is the cross entropy loss function; To balance the hyperparameters.

2. The industrial defect classification method based on federated transfer learning according to claim 1 is characterized in that: The data preparation in step S1 is to pre-process the data collected by the training client, and after data enhancement of the collected images, divide them into two data sets: high-resolution and low-resolution.

3. The industrial defect classification method based on federated transfer learning according to claim 1 is characterized in that: The high-low dual-channel multi-head attention mechanism module in step S2 includes: the high-resolution and low-resolution data sets are divided into data features through a dual-channel feature extractor, the high-resolution data focuses on local features, and the low-resolution data focuses on global features, and then the two features are fused and input into the multi-head attention mechanism module.

4. The industrial defect classification method based on federated transfer learning according to claim 1 is characterized in that: The model weight calculation and aggregation in step S4 are specifically as follows: According to the data features of the training client and the target client, the difference between the data features of the two clients is measured at multiple scales. The difference in statistical consistency of the two data feature data sets is measured by calculating the JS distance of the two data feature data sets. The difference in directional consistency of the two data feature data sets is measured by calculating the cosine distance. The multi-scale differences are aggregated, and the model weight of each training client is calculated according to the different difference distances of each training client. According to the calculated model weight of each training client, the models of each training client are weighted aggregated, and the weighted average model parameters are used as the global model parameters to update the global model.

Images