Federal learning optimization method based on industrial small sample defect detection

Abstract

The invention discloses a federated learning optimization method based on industrial small sample defect detection, and the method comprises the steps: measuring the knowledge reliability of a global industrial small sample defect detection model and a local industrial small sample defect detection model in different classes through extracting the distribution characteristics of sample soft labels in local training, and carrying out the detection of the industrial small sample defects. Then, a distributed feature distillation method is adopted to supplement weak knowledge of a local model on a specific category from the global model in a targeted manner, and collaborative learning and fusion of local knowledge and non-local knowledge are realized; meanwhile, according to the relative reliability of knowledge of the client industrial small sample defect detection model, contribution weights of all client models in the server aggregation stage are reasonably distributed, so that a global industrial small sample defect detection model with higher representativeness and generalization ability is obtained, and the overall prediction performance of the global model is improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud-based collaborative distributed machine learning and is applicable to cross-production line and cross-enterprise collaborative computing scenarios in intelligent manufacturing, such as cloud-based collaborative modeling in industrial small-sample defect detection. Specifically, it relates to a federated learning optimization method based on industrial small-sample defect detection. Background Art

[0002] In recent years, with the rapid development of mobile internet, the Internet of Things, and various smart devices, massive amounts of data have been generated across all industries. This data has gradually become the core driving force for technological innovation and industrial upgrading. In the field of industrial manufacturing, product quality control and defect detection are key links to ensure production efficiency and product reliability. However, traditional industrial defect detection technology faces many challenges: on the one hand, the number of defect samples in industrial scenarios is scarce and the labeling cost is high, resulting in the generalization ability of deep learning-based detection models being limited due to insufficient training data; on the other hand, equipment differences, process parameter fluctuations, and environmental changes between different production lines or factories cause defect characteristics to exhibit significant distribution shifts in cross-domain cloud collaboration scenarios, further exacerbating the difficulty of adapting detection algorithms. In addition, industrial data often involves sensitive production parameters and equipment information. The centralized data sharing model may lead to privacy leakage risks, restricting the feasibility of cross-enterprise collaboration.

[0003] Traditional centralized machine learning models typically upload dispersed data to cloud servers for unified storage, processing, and model training. Although this approach can alleviate heterogeneity issues through centralized data cleaning, its limitations are particularly prominent in industrial scenarios: First, the data generated by industrial equipment is massive and requires high real-time performance, and centralized transmission results in high communication costs and storage pressures. Second, data privacy barriers between production lines make cross-domain data integration difficult, hindering the optimization of global models. Finally, the value of small sample defect data is difficult to fully exploit through a single data source, resulting in insufficient detection model recognition capabilities for rare defects. Therefore, how to achieve cross-domain knowledge sharing while ensuring data privacy and improve defect detection accuracy in small sample scenarios has become a bottleneck that urgently needs to be overcome in the transformation to industrial intelligence.

[0004] Federated learning, an emerging distributed learning paradigm, offers a new approach to addressing these challenges. Its core concept is to complete model training on the user's local device, avoiding the centralized transmission of raw data and sharing only model parameters or gradient information. This approach protects data privacy while reducing communication overhead. Within this framework, each factory or device can independently train a model using local data (including a small number of defect samples), uploading only the encrypted model parameters to a central server for aggregation. A global detection model is generated and synchronized with all participants. This "data-never-exit" model effectively mitigates the risk of sensitive industrial information leakage and provides a technical foundation for the joint mining of cross-domain data value.

[0005] However, due to differences in industrial equipment performance and the complexity of application scenarios, large-scale distributed data often exhibit significant heterogeneity, including variations in data size, distribution, and category composition. This heterogeneity exacerbates the complexity of data fusion and cross-domain modeling, hindering the deep mining of data value. In the federated learning framework, this phenomenon is collectively referred to as the non-IID problem. Specifically, the data held by individual clients (e.g., different production lines) often differ significantly due to collection sources, application contexts, and user behavior, resulting in uneven sample distribution. This non-IID problem has significant adverse effects in federated learning. Because local model training relies heavily on client-side data, which can be skewed in distribution, the training direction can deviate from the overall optimal goal, resulting in client-side drift. This drift not only slows down the convergence of the global model but also makes it difficult for the server to obtain an effective global representation when aggregating local models, ultimately impacting overall model performance.

[0006] Mitigating the adverse effects of data heterogeneity on model performance has become a key research issue in federated learning. To this end, researchers both domestically and internationally have proposed various improvement strategies to address this challenge. First, within the client-side local algorithm space, research has primarily focused on improving training strategies. For example, methods based on loss function regularization introduce regularization terms to constrain the update direction of local model parameters, aiming to prevent the model from overfitting to the local data and thus mitigate client-side drift caused by heterogeneity. While these methods can improve model stability to a certain extent, their learning efficiency is often limited by the regularization mechanism itself and relies heavily on the algorithm's design quality, potentially hindering the full learning of local data knowledge. Another approach is optimization strategies based on knowledge distillation. Compared to traditional regularization terms, these methods introduce a distillation loss to use global model knowledge to guide local model training, thereby preserving the overall knowledge structure during training and reducing global information loss. However, these methods also have limitations: local models rely heavily on the global model during learning, and poor performance of the global model on specific categories can mislead local training. Furthermore, the efficiency of local knowledge extraction also limits overall training effectiveness. Furthermore, because training still relies on the data distribution of each client, the model struggles to learn generalized knowledge beyond local data in highly heterogeneous scenarios, resulting in slower overall model convergence. Secondly, research on server-side aggregation mechanisms has focused on improving global model update strategies. For example, by designing more rational aggregation algorithms, the ability to integrate client models from different distributions is enhanced, thereby improving the performance of the final global model under heterogeneous data. These strategies attempt to construct a more representative global model in the presence of significant client differences, reducing information loss during the aggregation process and improving accuracy and generalization. However, the performance of these approaches is highly dependent on the quality of the client models themselves, and their ability to mitigate severe client drift remains limited. Finally, data-based optimization methods have also been widely explored. These efforts attempt to bridge the gap between data distributions by sharing small amounts of common data between clients or synthesizing virtual samples using generative models. While these approaches mitigate the data heterogeneity issue to some extent, they also pose privacy and performance challenges. On the one hand, shared data may leak user sensitive information; on the other hand, generating high-quality virtual data often requires high capabilities of the generator, which not only requires a large amount of computing resources but may also cause a large communication burden, thus limiting the practicality of this method in resource-constrained scenarios. Summary of the Invention

[0007] To better address the cloud-based collaborative modeling of industrial small-sample defect detection, and address the issues faced by federated learning in heterogeneous environments, such as insufficient accuracy and slow convergence of the global industrial small-sample defect detection model, this paper proposes a federated learning optimization method based on industrial small-sample defect detection from the perspectives of local training and model aggregation. This paper mainly consists of two parts:

[0008] In the first part, this paper proposes a local training method based on distributed feature distillation. The core idea of ​​this method is to extract the distribution characteristics of sample soft labels, measure the knowledge reliability of the global and local industrial small-sample defect detection models in different categories, and then use distributed feature distillation to specifically supplement the weak knowledge of the local model in specific categories from the global model, thereby achieving the collaborative learning and fusion of local and non-local knowledge.

[0009] In the second part, the present invention proposes an aggregation method based on model knowledge reliability. The core idea of ​​this method is to evaluate the relative knowledge reliability of each client's industrial small-sample defect detection model and rationally allocate the contribution weights of each client model during the server-side aggregation phase. This results in a more representative and generalizable global industrial small-sample defect detection model, improving the overall prediction performance of the global model.

[0010] Combined with the typical federated learning training process, the specific implementation of the federated learning optimization method based on industrial small sample defect detection includes the following steps, among which the local training method based on distribution feature distillation mainly involves steps (2.1) to (2.4), and the aggregation method based on model knowledge reliability involves step (4):

[0011] Step (1). In each communication round, the server side uses the sampling ratio Across all clients Random sampling clients, building a client collection , Indicates the current communication round number. The server will use the global industrial small sample defect detection model Send it to the selected client. For each client , based on the global model weight of the current communication round Initialize the local industrial small sample defect detection model as .

[0012] Step (2). Perform local training on the client based on the client's local data. The client's local training program includes Each training cycle includes the following steps.

[0013] Step (2.1). The client is in the local dataset The above calculation obtains the category knowledge reliability of the global industrial small sample defect detection model and reliability of category knowledge of local industrial small sample defect detection models , and determine the unreliable category set based on the two .in, represents the feature set of the data sample, Represents a set of labels for data samples.

[0014] Step (2.2). Based on the unreliable category set , get the modified weight of the soft label distribution of the teacher model (i.e., the global model) ; For each training sample , Represents the characteristics of the sample, is the true category to which the sample belongs, and the correction weight of the soft label distribution of the student model (i.e., the client model) is defined as .

[0015] Step (2.3). Construct a distillation loss term based on distribution features During the local training process, for any training sample , using the modified weight The teacher model prediction probability distribution is modified to obtain , using the modified weight The student model prediction probability distribution is modified to obtain Then, the distillation loss term based on the distribution characteristics Prediction probability distribution after weighted correction and The KL divergence of is composed of:

[0016]

[0017] in, Indicates that the teacher model predicts that the sample belongs to the category The probability of Indicates that the student model predicts that the sample belongs to the category The probability of is the total number of categories.

[0018] Step (2.4). Construct the total loss function for training the local industrial small sample defect detection model The total loss function for local model training The distillation loss term based on distribution features proposed in step (2.3) and the traditional federated training cross entropy loss term The client local model is composed of the total loss function , in the local dataset Minimize the training loss on the local client side, thereby optimizing the local training process of the client in federated learning. The total loss function for local model training is The calculation formula is:

[0019]

[0020]

[0021] in, In the tag category and the true label category of the sample The value is 1 if the match is correct, otherwise it is 0.

[0022] Step (3). After the client iterative training is completed, the client uploads the local industrial small sample defect detection model parameters To the server.

[0023] Step (4). After receiving the industrial small sample defect detection model of each client, the server uses a small amount of unshared data sets to calculate the relative reliability of the knowledge of each client model. , then perform weighted aggregation based on the reliability of model knowledge to generate an updated global industrial small sample defect detection model. Global model weight The aggregation formula is:

[0024]

[0025] in, Indicates The global model obtained by the aggregation of rounds, Represents the client In the Model parameters uploaded after a round of local training, is the total number of clients selected in the current round.

[0026] Beneficial effects of the present invention:

[0027] The federated learning optimization method proposed in this invention consists of two parts: a local training method based on distribution feature distillation and an aggregation method based on model knowledge reliability.

[0028] (1) The local training method based on distribution feature distillation measures the knowledge reliability of the global industrial small sample defect detection model and the local industrial small sample defect detection model in each category by analyzing the distribution characteristics of the sample soft labels, and then uses the distribution feature distillation method to specifically supplement the weak knowledge of the local model in specific categories from the global model, thereby realizing the collaborative learning and fusion of local knowledge and non-local knowledge.

[0029] (2) The aggregation method based on model knowledge reliability evaluates the relative reliability of the knowledge of each client's industrial small sample defect detection model and reasonably allocates the contribution weight of each client model in the server-side aggregation stage, thereby obtaining a more representative and generalizable global industrial small sample defect detection model, thereby improving the overall prediction performance of the global model. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are included to provide a further understanding of the invention, and constitute a part of this specification. They illustrate preferred embodiments of the invention and together with the description serve to explain the principles of the invention. The same elements are denoted by the same reference numerals throughout.

[0031] In the attached figure:

[0032] Figure 1 This is a framework flow chart of the federated learning optimization method based on industrial small sample defect detection in the present invention. DETAILED DESCRIPTION

[0033] In order to enable people skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments.

[0034] This paper proposes a federated learning optimization method based on industrial small sample defect detection. Figure 1 This is a framework flow chart of the federated learning optimization method based on industrial small sample defect detection in the present invention. Figure 1 As shown, the overall framework of the present invention consists of two parts: a local training method based on distribution feature distillation and an aggregation method based on model knowledge reliability. First, in the training stage of the federated learning client, the present invention measures the knowledge reliability of the global industrial small sample defect detection model and the local industrial small sample defect detection model in different categories by extracting the distribution characteristics of the sample soft labels, and then uses the distribution feature distillation method to supplement the weak knowledge of the local model in specific categories from the global model in a targeted manner, thereby realizing the collaborative learning and fusion of local knowledge and non-local knowledge. Secondly, in the server-side aggregation stage, the present invention reasonably allocates the contribution weight of each client model in the server-side aggregation stage by evaluating the relative reliability of the knowledge of the industrial small sample defect detection model of each client, thereby obtaining a more representative and generalized global industrial small sample defect detection model, thereby improving the overall prediction performance of the global model.

[0035] The implementation steps are described in detail below.

[0036] Step (1). In each communication round, the server side uses the sampling ratio Across all clients Random sampling clients, building a client collection , Indicates the current communication round number. The server will use the global industrial small sample defect detection model Send it to the selected client. For each client , based on the global model weight of the current communication round Initialize the local industrial small sample defect detection model as .

[0037] Step (2). Perform local training on the client based on the client's local data. The client's local training program includes Each training cycle includes the following steps.

[0038] Step (2.1). The client is in the local dataset The above calculation obtains the category knowledge reliability of the global industrial small sample defect detection model and reliability of category knowledge of local industrial small sample defect detection models , and determine the unreliable category set based on the two .in, represents the feature set of the data sample, Represents a set of labels for data samples.

[0039] First, for any training sample , the soft label output of the model is expressed as ,in is the total number of categories. The sum of all components in is 1. Assume that the soft label distribution except the true category item is recorded as ,Right now:

[0040]

[0041] Defining the predictive reliability of a sample for The reciprocal of the sum of the mean and standard deviation of , that is:

[0042]

[0043] in, represents the mean of the soft label distribution excluding the true category items, represents the standard deviation of the soft label distribution excluding the true category items.

[0044] Secondly, the category knowledge reliability of the model is further defined. Before each local iteration, this method counts the prediction reliability of each category of samples in the local dataset and takes the average value to obtain the category knowledge reliability of the model. Specifically, the category knowledge reliability of the model is expressed as , where any category The knowledge reliability of is defined as follows:

[0045]

[0046] in, The true category is Based on this, the category knowledge reliability of the global industrial small sample defect detection model can be calculated separately. and reliability of category knowledge of local industrial small sample defect detection models .

[0047] Again, the unreliable category set is defined based on the reliability of the model's category knowledge. For any category, if the knowledge reliability of the local model on this category is lower than the knowledge reliability of the global model on this category, then this category is added to the unreliable category set. In addition, for categories not included in the local dataset, they are also included in the unreliable category set. Unreliable category set The construction rules can be expressed as:

[0048]

[0049] in Represents the client The set of categories contained in the local dataset, Represents the set of all categories.

[0050] Step (2.2). Based on the unreliable category set , get the modified weight of the soft label distribution of the teacher model (i.e., the global model) ; For each training sample , Represents the characteristics of the sample, is the true category to which the sample belongs, and the correction weight of the soft label distribution of the student model (i.e., the client model) is defined as .

[0051] For any category The correction value is calculated as follows:

[0052]

[0053] For any category The correction value is calculated as follows:

[0054]

[0055] Step (2.3). Construct a distillation loss term based on distribution features During the local training process, for any training sample , using the modified weight The teacher model prediction probability distribution is modified to obtain , using the modified weight The student model prediction probability distribution is modified to obtain .

[0056] The teacher model predicts that the sample belongs to the category The probability of , calculated as follows:

[0057]

[0058] The student model predicts that the sample belongs to the category The probability of , calculated as follows:

[0059]

[0060] in, Represents the teacher model for the category The logits output, Represents the student model for category The logits output, is the temperature coefficient.

[0061] Then, the distillation loss term based on the distribution characteristics Prediction probability distribution after weighted correction and The KL divergence of is composed of:

[0062]

[0063] in, is the total number of categories.

[0064] Step (2.4). Construct the total loss function for training the local industrial small sample defect detection model The total loss function for local model training The distillation loss term based on distribution features proposed in step (2.3) and the traditional federated training cross entropy loss term The client local model is composed of the total loss function , in the local dataset Minimize the training loss on the local client side, thereby optimizing the local training process of the client in federated learning. The total loss function for local model training is The calculation formula is:

[0065]

[0066]

[0067] in, In the tag category and the true label category of the sample The value is 1 if the match is correct, otherwise it is 0.

[0068] Step (3). After the client iterative training is completed, the client uploads the local industrial small sample defect detection model parameters To the server.

[0069] Step (4). After receiving the industrial small sample defect detection model of each client, the server uses a small amount of unshared data sets to calculate the relative reliability of the knowledge of each client model. ,

[0070] First, for any training sample , its prediction is unreliable is defined as:

[0071]

[0072] Secondly, by counting the prediction unreliability of samples in each category in the statistical data set and calculating the mean, the model's knowledge unreliability for that category can be obtained, thereby measuring the model's mastery of that category. On this basis, this method sums the model's knowledge unreliability for all categories as the overall knowledge unreliability of the model, reflecting its overall unreliability. Therefore, the model's knowledge unreliability is defined as:

[0073]

[0074] in, Indicates a small amount of unshared data held by the server. Indicates the total number of samples in the dataset. In order to fairly measure the importance of each category of knowledge, the dataset is required to The number of samples in each category is roughly equal. Based on this, the knowledge unreliability of the global model can be calculated separately and the knowledge unreliability of each client model .

[0075] Again, the knowledge based on the model is unreliable, for any client , this method defines the relative reliability of the knowledge of its client model for:

[0076]

[0077] Then, based on the reliability of model knowledge, weighted aggregation is performed to generate an updated global industrial small sample defect detection model. Global model weight The aggregation formula is:

[0078]

[0079] in, Indicates The global model obtained by the aggregation of rounds, Represents the client In the Model parameters uploaded after a round of local training, is the total number of clients selected in the current round.

[0080] The experimental results of this paper are shown in Tables 1, 2, and 3. The methods compared in this paper primarily include the traditional FedAvg algorithm and algorithms improved to address data heterogeneity (FedProx, MOON, FedNTD, FedLMD, and FedSOL). All methods were replicated using the PyTorch framework (1.9.0 + cu111) and tested on an RTX 3080 and an Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz.

[0081] In our experiments, we employed several datasets widely used in federated learning research, including MNIST, CIFAR-10, and CINIC-10. To simulate the heterogeneous data environments of real-world scenarios, we partitioned the dataset using a non-IID partitioning strategy based on a latent Dirichlet distribution. By sampling the dataset using a Dirichlet distribution, we allocated an unbalanced data sample to each client based on different label distributions. The degree of data heterogeneity between different clients was determined by controlling 𝛼. In our experiments, we set 𝛼 to MNIST (α = 0.1), CIFAR-10 (α = 0.05, 0.1, 0.3, 0.5), and CINIC-10 (α = 0.05, 0.1).

[0082] To ensure fair comparison, all methods used a network model consisting of two convolutional layers, two max pooling layers, and two fully connected layers. The model was trained using the SGD optimizer. The learning rate was set to 0.01, the SGD momentum was set to 0.9, and the weight decay was set to 1e-5. The batch size was set to 50.

[0083] For the federated learning task, the experiment set some additional hyperparameters. Referring to the settings of previous studies, the total number of clients was set to 100, the client sampling ratio was 0.1, the number of communication rounds was 70, and the number of local iterations was 3 for the MNIST dataset; the total number of clients was set to 50, the client sampling ratio was 0.2, the number of communication rounds was 150, and the number of local iterations was 10 for the CIFAR-100 dataset; the total number of clients was set to 100, the client sampling ratio was 0.1, the number of communication rounds was 150, and the number of local iterations was 5 for the CIFAR-100 dataset; and the total number of clients was set to 100, the client sampling ratio was 0.1, the number of communication rounds was 200, and the number of local iterations was 5 for the CINIC-10 dataset.

[0084] After each round of communication, the global model is evaluated on the test dataset, and the average test accuracy is displayed as the result (a "-" indicates that the algorithm has not converged). FedDRA is a federated learning optimization method based on distributed feature distillation and reliable aggregation. FedDRA_simple removes the reliable aggregation module from FedDRA (that is, it only includes the distributed feature distillation optimization) to verify the effectiveness of both modules.

[0085] Table 1 Accuracy (%) on MNIST, CIFAR-10, and CINIC-10

[0086]

[0087] Table 1 shows the experimental results of the test accuracy of various algorithms. FedDRA outperforms baseline methods across different datasets and levels of label heterogeneity. In particular, on CINIC-10 (α = 0.1), FedDRA achieves a 2.71% improvement over the best baselines (FedLMD and FedSOL). FedDRA-simple also achieves results similar to the best baseline, FedSOL. These results highlight the stable performance of our method even in non-IID scenarios. Furthermore, on the CIFAR-10 dataset, as the value of α decreases, the role of reliable aggregation increases, and the effectiveness of FedDRA becomes increasingly pronounced. This further demonstrates the effectiveness of FedDRA's targeted optimization for heterogeneous data.

[0088] Table 2 Total communication overhead of each algorithm on the CIFAR-10 dataset

[0089]

[0090] Table 2 shows the communication overhead of each algorithm on the CIFAR-10 dataset when achieving a specified accuracy or above. When the specified accuracy is 60%, the total communication overhead of the algorithms FedAvg, FedProx, MOON, FedNTD, FedLMD, and FedSOL is 707.44MB, 773.24MB, 707.44MB, 723.89MB, 526.46MB, and 707.44MB, respectively. Compared to the baseline methods, the FedDRA method does not incur any additional communication overhead per communication round. Its total communication overhead decreases with faster convergence, reaching 394.85MB. This represents a 25.0% reduction compared to FedLMD and a 44.2% reduction compared to FedAvg. When the specified accuracy is 65%, the total communication overhead of FedDRA is 690.98MB, still significantly lower than the baseline methods.

[0091] Table 3. Convergence communication rounds on CINIC-10 and CIFAR-10 datasets

[0092] Table 3 shows the number of communication rounds required for each algorithm to reach above the target accuracy for three consecutive rounds on the CIFAR-10 and CINIC-10 datasets (where "-" indicates that the requirement was not met within the total number of communication rounds). It can be seen that FedDRA requires fewer rounds to converge than other methods in two different heterogeneous scenarios on CIFAR-10. For example, when α = 0.1, FedDRA reduces the number of communication rounds by 68.5%, 55.3%, 50.0%, and 46.5% compared to FedProx, FedNTD, FedLMD, and FedSOL, respectively, to reach 65% accuracy. When α = 0.3, FedDRA reduces the number of communication rounds by 22.4% and 15.1% compared to FedNTD and FedLMD, respectively, to reach 75% accuracy. On the CINIC-10 dataset, when α = 0.1, FedDRA reduces the number of communication rounds required to reach 50% accuracy by 35.2% and 35.2% compared to FedLMD and FedSOL, respectively, and reduces the number of communication rounds required to reach 55% accuracy by 34.2% and 43.9%, respectively. This shows that FedDRA's convergence speed on the CINIC-10 dataset has been improved.

[0093] In summary, the experimental results show that compared with previous methods, the method of the present invention has certain improvements and optimizations in terms of accuracy and cost.

[0094] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings. Many features and advantages of these embodiments will be apparent from this detailed description. Furthermore, since many modifications and variations will readily occur to those skilled in the art, the foregoing description is merely illustrative and is not intended to limit the present description. Various modifications and variations are readily apparent to those skilled in the art.

Claims

1. A federated learning optimization method based on industrial small sample defect detection, characterized by: The following steps are involved: Step 1. The server side uses the global industrial small sample defect detection model Sent to the selected client, the client is based on the global model weight of the current communication round Initialize the local industrial small sample defect detection model; Step 2. Perform local training on the client based on the client's local data. Step 3. After the client iterative training is completed, the client uploads the local industrial small sample defect detection model parameters To the server; Step 4. After receiving the industrial small sample defect detection model of each client, the server uses a small amount of unshared data sets to calculate the relative reliability of the knowledge of each client model. ,Then weighted aggregation is performed based on the reliability of the model knowledge to generate an updated ,global industrial small sample defect detection model.

2. The federated learning optimization method based on industrial small sample defect detection according to claim 1 is characterized in that: The specific process of step 1 is as follows: in each communication round, the server side calculates the sampling ratio according to the sampling ratio. Across all clients Random sampling clients, building a client collection , Indicates the current communication round number; the server side will global industrial small sample defect detection model Send to the selected client; for each client , based on the global model weight of the current communication round Initialize the local industrial small sample defect detection model as .

3. The federated learning optimization method based on industrial small sample defect detection according to claim 1 is characterized in that: In step 2, the client performs local training including E rounds of training, and each round of training includes the following steps: Step 2.

1. Client in local dataset The above calculation obtains the category knowledge reliability of the global industrial small sample defect detection model and reliability of category knowledge of local industrial small sample defect detection models , and determine the unreliable category set based on the two ,in, represents the feature set of the data sample, A set of labels representing data samples; Step 2.

2. Based on the unreliable category set , get the modified weight of the teacher model soft label distribution ; For each training sample , Represents the characteristics of the sample, is the true category to which the sample belongs, and the correction weight of the soft label distribution of the student model is defined as ; Step 2.

3. Construct a distillation loss term based on distribution features ; During the local training process, for any training sample , using the modified weight The teacher model prediction probability distribution is modified to obtain , using the modified weight The student model prediction probability distribution is modified to obtain ; Then, the distillation loss term based on the distribution characteristics Prediction probability distribution after weighted correction and The KL divergence of is composed of: in, Indicates that the teacher model predicts that the sample belongs to the category The probability of Indicates that the student model predicts that the sample belongs to the category The probability of is the total number of categories; Step 2.

4. Construct the total loss function for training the local industrial small sample defect detection model ; Total loss function for local model training , the distillation loss term based on the distribution characteristics proposed in step (2.3) and the traditional federated training cross entropy loss term Together; the client local model is based on the total loss function , in the local dataset Minimize the training loss, thereby optimizing the local training process of the client in federated learning; the total loss function of local model training The calculation formula is: in, In the tag category and the true label category of the sample The value is 1 if the match is correct, otherwise it is 0.

4. The federated learning optimization method based on industrial small sample defect detection according to claim 3 is characterized in that: The server generates an updated global industrial small sample defect detection model, and the global model weight The aggregation formula is: in, Indicates The global model obtained by the aggregation of rounds, Represents the client In the Model parameters uploaded after a round of local training, is the total number of clients selected in the current round.

Images