Federated learning method and device based on dual adversarial training in industrial Internet of Things

Abstract

The present invention belongs to the technical field of feature extraction, and more specifically, relates to a federated learning method and device based on dual adversarial training under the industrial Internet of Things. The method includes: the server initializes the global model and sends it to the client; each client updates the local model based on the received global information; after completing the local model update, each client enters the local training stage; then each client recalculates the local feature prototype and feature extractor upload round; then uploads the following local information to the server, wherein the local feature extractor is uploaded only if the current round is in the upload round set; finally, the server aggregates the local feature prototype and the local classifier, and the received local feature extractor, and then sends the aggregated information to each client. The present invention solves the problem that the existing federated learning method has insufficient design of the balance mechanism between personalization and shared knowledge, and is difficult to adapt to the multi-source heterogeneous and dynamically changing environment of the industrial site.

Description

Technical Field

[0001] The present invention belongs to the technical field of feature extraction, and more specifically, relates to a federated learning method and device based on dual adversarial training in the industrial Internet of Things. Background Art

[0002] In the Industrial Internet of Things (IIoT) landscape, widely deployed edge devices continuously collect high-frequency sensor data from diverse work conditions, production workflows, and environmental environments. This data is often high-dimensional, noisy, and has significant distribution variations, directly impacting key business indicators such as production efficiency, equipment health, and safety alerts. Due to stringent data privacy requirements across enterprises and devices, traditional centralized machine learning approaches cannot centrally aggregate and uniformly model data. Furthermore, limited network bandwidth, communication latency, and edge computing resources make uploading data to a central server for processing impractical. Therefore, federated learning, a distributed learning framework that supports local model training across devices and edge nodes and enables collaborative modeling through parameter aggregation, has demonstrated broad applicability in the IIoT. It not only mitigates the risk of raw data leakage but also improves model training efficiency, providing a secure and feasible solution for industrial intelligence.

[0003] For example, Chinese patent document CN119670916A discloses a federated learning method and device based on feature contrast optimization and dynamic integration of classifiers. The method includes: initializing a global model through a server and sending it to each client participating in federated learning; the client updates the local model based on the received global model and trains the updated local model; then uses the trained local feature extractor to extract features from its local private data set to construct a local feature prototype set; then uploads the local model, local feature prototype set, and the total client sample size to the server; during global aggregation, the server uses contrastive learning technology to improve the quality of the global feature prototype; finally, the aggregated global prototype and global feature prototype set are sent to each client to perform the next round of learning until the local model converges or reaches the set communication round. However, the feature extractor and feature prototype of this method need to be transmitted in each communication round, which results in high communication overhead. Moreover, the patent achieves a balance between model generalization ability and personalized performance by integrating the global classifier and the local classifier at the parameter level. However, this parameter-level fusion method has certain limitations and may not fully integrate the advantages of the two types of classifiers.

[0004] Although federated learning offers a new paradigm for intelligent modeling in the Industrial Internet of Things (IIoT), current personalized federated learning approaches still face significant technical bottlenecks. Specifically, existing methods lack a well-designed balance between personalization and shared knowledge. This results in models either overly relying on local data training, neglecting the common knowledge contained in global information and reducing their generalization capabilities; or emphasizing the uniformity of global aggregate parameters, ignoring the non-independent and identically distributed nature of data distribution across terminal devices, making them unable to effectively adapt to personalized task requirements. This insufficient and inappropriate use of global aggregate information limits the model's ability to simultaneously account for both "individuality" and "commonality," making it difficult to adapt to the multi-source, heterogeneous, and dynamically changing environments of industrial sites. In the pursuit of more efficient and robust personalized federated learning frameworks, an optimization mechanism is urgently needed that can both enhance the adaptability of personalized models and effectively utilize global shared knowledge. Summary of the Invention

[0005] The present invention aims to overcome at least one of the above-mentioned shortcomings of the prior art and provide a federated learning method based on dual adversarial training under the industrial Internet of Things to solve the problems of insufficient design of the balance mechanism between personalization and shared knowledge in existing federated learning methods, difficulty in adapting to the multi-source heterogeneous and dynamically changing environment of industrial sites, and high communication overhead.

[0006] The present invention also discloses a device loaded with a federated learning method based on dual adversarial training under the industrial Internet of Things.

[0007] The detailed technical solutions of the present invention are as follows:

[0008] A federated learning method based on dual adversarial training in the industrial Internet of Things, the method comprising:

[0009] S1. The server initializes the global model and sends it to each client participating in federated learning, where the client refers to an industrial device or terminal.

[0010] S2. Each client updates its local model based on the received global information:

[0011] When the local model is updated for the first time, each client initializes the local model to the received global model;

[0012] In subsequent rounds, when the client receives a new round of global feature extractors from the server, it updates the local feature extractor. However, in order to better adapt the local model to personalized tasks, the local classifier remains unchanged and is not updated, ensuring that the local classifier can focus on processing the specific needs of the local task.

[0013] S3. After completing the local model update, each client enters the local training phase. The feature extractor and classifier in the local model are trained independently in stages. The training order is to train the local classifier first, and then train the local feature extractor. In the first round of local training, the local feature extractor and the local classifier are trained only based on the data-driven cross entropy loss; in the non-first local training, the feature extractor is constrained by both the cross entropy loss and the global feature prototype loss, and the classifier is optimized by superimposing the adversarial training loss on the cross entropy loss. Through the complementary fusion of global generalization ability and local personalization ability at the knowledge level, the model can better adapt to personalized needs while maintaining generalization.

[0014] S4. After completing the first round of local training, each client calculates the upload round set of the feature extractor before uploading local information to the server. Subsequent rounds do not need to be recalculated. Each client then recalculates the local feature prototype and uploads the following local information to the server: the recalculated local feature prototype, the local feature extractor, the local classifier, the total number of training samples owned by the client, and the number of training samples in each category. The sample number information is uploaded only after the first round of training is completed. If the current round is in the upload round set, the local feature extractor is uploaded.

[0015] S5. The server uses the category sample size and total sample size as the basis for setting aggregation weights, aggregates the local feature prototypes and local classifiers uploaded by the client, and generates a global feature prototype and global classifier. If a local feature extractor is uploaded in the current round, it performs weighted aggregation based on the total sample size to obtain a global feature extractor. The aggregated information is then sent to the clients participating in federated learning.

[0016] S6. Repeat steps S2 to S5 until the local model converges or the preset number of communication rounds is reached.

[0017] According to the preferred embodiment of the present invention, step S2 specifically includes:

[0018] For the first round of local model update, both the local feature extractor and the local classifier are updated using the global feature extractor and the global classifier:

[0019] (1)

[0020] in, Indicates the local feature extractor after the first round of update, represents the global feature extractor, Indicates the local classifier after the first round of update, denotes a global classifier, the superscript Indicates the current state after updating and before training;

[0021] For the When the local model is updated, , client The judgment is completed Does the latest content downloaded by the server after the aggregation contain the global feature extractor?

[0022] If so, the local feature extractor is updated using this global feature extractor:

[0023] (2)

[0024] in, Indicates the After the local update, sub-local pre-trained feature extractor, Indicates the The global feature extractor obtained after aggregation on the secondary server side;

[0025] If not, the feature extractor continues to use the Sub-local update of the trained feature extractor :

[0026] (3)

[0027] For the local classifier, continue to use the Local classifier after local update training:

[0028] (4)

[0029] in, Indicates the After the local update, The local classifier before local training, Indicates the Sub-local update of the trained local classifier.

[0030] According to the preferred embodiment of the present invention, step S3 specifically includes:

[0031] For the first round of local training, both the local classifier and feature extractor are trained using only data-driven cross-entropy loss:

[0032] (5)

[0033] (6)

[0034] In formula (5) and formula (6), Indicates the local classifier trained in the first round, Indicates the local classifier after the first round of update, represents the gradient of the loss associated with the local classifier, represents the samples in the local dataset, Represents the client's local dataset, represents the cross entropy loss, Represents the input feature information of the image, Represents the label corresponding to the input feature information of the image, represents the local feature extractor trained in the first round, Indicates the local feature extractor after the first round of update, Represents the gradient of the loss associated with the local feature extractor.

[0035] According to a preferred embodiment of the present invention, step S3 specifically further includes:

[0036] For the During local training, When , the local classifier is trained using a combination of adversarial training loss and data-driven cross entropy loss:

[0037] Because local feature prototypes are needed in adversarial loss, the current local feature prototypes need to be calculated before this operation:

[0038] (7)

[0039] (8)

[0040] in, Indicates that the local feature prototype set is obtained before the local classifier and local feature extractor are trained. Contains The local feature prototype of each class, class, represented by , Indicates the Client local dataset Belong to the Each sample data set of each class, express A single sample in represents the local feature extraction function, Indicates the The client local data set belongs to The size of the sample dataset for each class, Indicates the Local feature extractor before training after round of local updates;

[0041] Then download the global feature prototype set of the previous round from the server , and the global classifier , to construct two local personalized discriminator losses, namely the prototype discriminator The loss and classifier discriminator Losses:

[0042] For the prototype discriminator The loss function is used for training :

[0043] (9)

[0044] In formula (9), Indicates the The client in The loss when training the prototype discriminator, Indicates belonging No. Class, Therefore The logarithmic function with base , is the prototype discriminant function, is the local classification function, Indicates the The local classifier after the local update, Indicates that it belongs to The global feature prototype of each class, Indicates that the first Local feature prototypes of each class;

[0045] For the classifier discriminator The loss function is used for training :

[0046] (10)

[0047] In formula (10), Indicates the The client in The loss during the round of training of the classifier discriminator, Indicates belonging No. Class, Therefore The logarithmic function with base , is the classifier discriminant function, represents the global classification function, Indicates the The global classifier obtained after aggregation, Indicates that it belongs to The global feature prototype of each class, is the local classification function, Indicates the The local classifier after the local update;

[0048] Based on the above two discriminator losses, the first-order adversarial training loss of the local classifier as the generator is obtained , used for training :

[0049] (11)

[0050] In formula (11), Indicates the Client No. The first-order adversarial training loss during round training, is the prototype discriminant function, is the local classification function, Indicates the The local classifier after the local update, Indicates that the first The local feature prototype of each class, is the classifier discriminant function, Indicates that it belongs to The global feature prototype of a class.

[0051] At the same time, the cross entropy loss formed by the local prototype is added to obtain the second-order adversarial training loss (This can be referred to as adversarial training loss), which effectively alleviates the local classifier's forgetting of local knowledge when adapting to local tasks:

[0052] (12)

[0053] So the total loss of training the local classifier is for:

[0054] (13)

[0055] The local classifier is trained using stochastic gradient descent to obtain the trained local classifier :

[0056] (14)

[0057] For the feature extractor, the global feature prototype is used to guide the training of the local feature extractor, helping the local feature extractor improve its generalization ability:

[0058] (15)

[0059] (16)

[0060] in, represents the loss using the global feature prototype, represents the Euclidean norm, Indicates that it belongs to The global trait prototype of the class, represents the local feature extractor trained in round t, Represents the local classifier after the t-th round of training.

[0061] Preferably, according to the present invention, step S4 specifically includes:

[0062] After the client completes the first round of local training and before uploading local information to the server, it calculates the ratio of the local feature extractor to the local feature prototype parameters. ,pass To achieve selective uploading of local feature extractors, this method can effectively reduce the communication cost of the entire system. The local feature extractor is uploaded only when necessary, thereby reducing unnecessary communication overhead. If the training of the local feature extractor is guided by the global feature prototype alone, it may not be possible to ensure that the local feature extractor obtains sufficient generalization ability. At the same time, if the feature extractor is uploaded in each round, on the one hand, it will bring too high a communication cost, and on the other hand, it may also cause the personalization ability of the local feature extractor to be excessively reduced. The ratio of the local feature extractor to the local feature prototype parameters is The calculation formula is as follows:

[0063] (17)

[0064] in Indicates the amount of calculation parameters;

[0065] Then Multiples of are eliminated from the communication rounds. In these rounds, the local feature extractor does not upload, and the feature extractor upload round set is obtained;

[0066] Recalculate the local feature prototype to obtain the local feature prototype set to be uploaded :

[0067] (18)

[0068] (19)

[0069] In formulas (18) and (19), Indicates that the local feature prototype set is obtained after the local feature extractor is trained. Contains The local feature prototype of each class, class, represented by , Indicates the Client local dataset Belong to the Each sample data set of each class, express A single sample in represents the local feature extraction function, Indicates the The client local data set belongs to The size of the sample dataset for each class, Indicates the Local feature extractor after one round of local training.

[0070] According to a preferred embodiment of the present invention, step S5 specifically includes:

[0071] If the feature extractor is received in the current aggregation round, the aggregation weight is set based on the total number of client samples to obtain the global feature extractor :

[0072] (20)

[0073] (twenty one)

[0074] In formula (20), Indicates that based on The weight set for the total number of client samples;

[0075] For the classifier, we also set the aggregation weight based on the total number of client samples to obtain the global classifier. :

[0076] (twenty two)

[0077] For feature prototype aggregation, the sample size of each class is used as the aggregation weight to obtain the global feature prototype:

[0078] (twenty three)

[0079] (twenty four)

[0080] (25);

[0081] In formula (23)-formula (25), Indicates that based on Client No. The weight set by the sample size of each class, Indicates the The client local data set belongs to The size of the sample dataset for each class ( Similarly), Indicates in After the global aggregation, we get the global feature prototype set. To include The global feature prototype of each class, class, represented by ,and Indicates that after The local feature prototype belonging to the kth class obtained after a round of local training.

[0082] Preferably, according to the present invention, the global information in step S2 contains a global feature extractor and a global classifier in the initial round, and these two parts constitute the global model; in the non-initial round, it must contain a global feature prototype and a global classifier, and if the server has a newly aggregated global feature extractor, it contains a global feature extractor.

[0083] In another aspect of the present invention, a device for implementing a federated learning method based on dual adversarial training in an industrial Internet of Things is provided, comprising:

[0084] at least one processor; and

[0085] A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to execute the above-mentioned federated learning method based on dual adversarial training under the industrial Internet of Things.

[0086] In another aspect of the present invention, a machine-readable storage medium is provided, which stores executable instructions. When the instructions are executed, the machine executes the above-mentioned federated learning method based on dual adversarial training under the industrial Internet of Things.

[0087] Compared with the prior art, the present invention has the following beneficial effects:

[0088] (1) Based on the characteristics of devices and data in the industrial Internet of Things scenario, the present invention selectively transmits feature extractors according to the dimensions of feature extractors and prototypes, thereby improving the performance of the model in a multi-device environment while ensuring that the system transmission cost is comparable to that of traditional federated learning methods.

[0089] (2) The present invention extracts and processes the effective knowledge in the global classifier, while ensuring the personalized characteristics of the local classifier of each industrial equipment, supplementing the local classifier's knowledge on the missing categories, thereby improving its generalization ability on the existing categories and optimizing the knowledge sharing effect in multi-device collaboration.

[0090] (3) The present invention has achieved, to a certain extent, an effective balance between the generalization capability and personalized characteristics of the model and the system communication cost in the industrial Internet of Things environment, ensuring efficient collaboration between devices while reducing communication overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 This is a flow chart of the federated learning method based on dual adversarial training under the industrial Internet of Things described in the present invention.

[0092] Figure 2 Schematic diagram of the implementation process of the federated learning method described in Example 1 of the present invention.

[0093] Figure 3 This is a graph showing the experimental results of the generalization performance test of the present invention conducted on the acceleration sensor-action recognition dataset.

[0094] Figure 4 This is a graph showing the experimental results of a personalized performance test conducted on an acceleration sensor-action recognition dataset.

[0095] Figure 5 This is a graph showing the experimental results of the convergence test of the present invention on the acceleration sensor-action recognition dataset. DETAILED DESCRIPTION

[0096] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0097] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0098] In the absence of conflict, the embodiments of the present disclosure and the features thereof may be combined with each other.

[0099] Example 1

[0100] like Figure 1 and Figure 2As shown, this embodiment provides a federated learning method based on dual adversarial training for the Industrial Internet of Things (IIoT). This method, designed for IIoT environments, addresses the personalized modeling challenges presented by edge device heterogeneity and data distribution discrepancies. In practical applications, data collected by sensors deployed across factories, production lines, or industrial equipment is influenced by multiple factors, such as geographic location, manufacturing process, equipment brand, and operational load, exhibiting significant heterogeneity and non-IID characteristics. Traditional federated learning methods struggle to balance model consistency and personalization across multiple nodes. To address this issue, the present invention proposes a personalized federated learning method based on dual adversarial training, specifically designed for the distributed intelligent modeling needs of the IIoT. This method divides the model into two parts: a feature extractor and a classifier, which respectively perform feature extraction and state classification on industrial equipment operating data. By introducing a prototype discriminator and a classifier discriminator to construct a dual adversarial structure, the local classifier acts as a generator in training, continuously optimizing the model to adapt to local operating conditions while retaining common knowledge from other industrial nodes. This method is suitable for industrial equipment state classification tasks and can effectively improve the model's adaptability to data from different industrial nodes, enhancing accuracy and stability in key scenarios such as worker behavior detection, equipment fault analysis, and production line optimization.

[0101] The following introduces the terms and system settings used in the present invention.

[0102] The present invention is directed to a The federated learning system framework is modeled with edge nodes and a central server. In this system, each client and server can exchange parameters to complete the distributed collaborative training of the model. Clients (subscript ), The stage after the round of local update training (superscript ), at this time the client's local model is (The same applies to the other stages.) The client holds a private dataset. , the sample has a classification label, and the category set is recorded as , the total number of categories is Each sample is recorded as ( ),in Represents the client The total number of training samples in the dataset.

[0103] The following describes the module division and function description of the present invention, starting with Client, After the local update training, Take the local update before training as an example, and the same applies to other clients and rounds.

[0104] In this invention, the model consists of a feature extraction module and a classification module, which respectively process the feature encoding of the input data and the final classification prediction task. Assume that the local feature extractor parameters are , the local classifier parameters are ,Right now Depend on and And the local feature extraction function is , the local classification function is Specifically, for any input sample ( ), first the feature extractor generates a feature vector (for example, if a local feature extractor is used, then ), and then the classification function processes the feature vector and outputs the final classification result. For example, if a local classifier is used, it is The above process is also applicable to the global feature extractor and global classifier. Global information has no client subscript. .

[0105] The method is specifically as follows:

[0106] S1. The server initializes the global model and sends it to each client participating in federated learning, where the client refers to an industrial device or terminal.

[0107] S2. Each client updates its local model based on the received global information:

[0108] When the local model is updated for the first time, each client initializes the local model to the received global model. By providing appropriate initial values ​​for the local model, the stability of the model is ensured during the initial training phase, thereby accelerating convergence and improving training results.

[0109] In subsequent rounds, when the client receives a new round of global feature extractors from the server, it updates the local feature extractor. To better adapt the local model to personalized tasks, the local classifier remains unchanged and is not updated. This ensures that the local classifier can focus on processing the specific needs of the local task, as follows:

[0110] For the first round of local model update, both the local feature extractor and the local classifier are updated using the global feature extractor and the global classifier:

[0111] (1)

[0112] in, Indicates the local feature extractor after the first round of update, represents the global feature extractor, Indicates the local classifier after the first round of update, denotes a global classifier, the superscript Indicates the current state after updating and before training;

[0113] For the When the local model is updated, , client The judgment is completed Does the latest content downloaded by the server after the aggregation contain the global feature extractor?

[0114] If so, the local feature extractor is updated using this global feature extractor:

[0115] (2)

[0116] in, Indicates the After the local update, sub-local pre-trained feature extractor, Indicates the The global feature extractor obtained after aggregation on the secondary server side;

[0117] If not, the feature extractor continues to use the Sub-local update of the trained feature extractor :

[0118] (3)

[0119] For the local classifier, continue to use the Local classifier after local update training:

[0120] (4)

[0121] in, Indicates the After the local update, The local classifier before local training, Indicates the Sub-local update of the trained local classifier;

[0122] In the initial round, the global information contains a global feature extractor and a global classifier, and these two parts constitute the global model; in the non-initial round, it must contain a global feature prototype and a global classifier. If the server has a newly aggregated global feature extractor, it contains a global feature extractor.

[0123] S3. After completing the local model update, each client enters the local training phase. The feature extractor and classifier in the local model are trained independently in stages. The training order is to train the local classifier first, and then train the local feature extractor. This method refines the training objectives, improves the flexibility of the training process, and enables the model to better adapt to the needs of different tasks. In the first round of local training, the local feature extractor and the local classifier are trained only based on the data-driven cross entropy loss; in the non-first local training, the feature extractor is constrained by both the cross entropy loss and the global feature prototype loss. By giving full play to the powerful migration ability of the feature extractor, a feature extractor with strong generalization is learned, thereby effectively alleviating the negative impact of heterogeneous data on model performance. The classifier is optimized by superimposing the adversarial training loss on the cross entropy loss. Through the complementary fusion of global generalization ability and local personalization ability at the knowledge level, the model can better adapt to personalized needs while maintaining generalization, as follows:

[0124] For the first round of local training, both the local classifier and feature extractor are trained using only data-driven cross-entropy loss:

[0125] (5)

[0126] (6)

[0127] In formula (5) and formula (6), Indicates the local classifier trained in the first round, Indicates the local classifier after the first round of update, represents the gradient of the loss associated with the local classifier, represents the samples in the local dataset, Represents the client's local dataset, represents the cross entropy loss, Represents the input feature information of the image, Represents the label corresponding to the input feature information of the image, represents the local feature extractor trained in the first round, Indicates the local feature extractor after the first round of update, Represents the gradient of the loss associated with the local feature extractor.

[0128] For the During local training, When , the local classifier is trained using a combination of adversarial training loss and data-driven cross entropy loss:

[0129] Because local feature prototypes are needed in the adversarial loss, the current local feature prototypes need to be calculated before this operation. The feature prototype is the centroid of the feature vectors of each category extracted by the feature extractor:

[0130] (7)

[0131] (8)

[0132] in, Indicates that the local feature prototype set is obtained before the local classifier and local feature extractor are trained. Contains The local feature prototype of each class, class, represented by , Indicates the Client local dataset Belong to the Each sample data set of each class, express A single sample in represents the local feature extraction function, Indicates the The client local data set belongs to The size of the sample dataset for each class, Indicates the Local feature extractor before training after round of local updates;

[0133] Then download the global feature prototype set of the previous round from the server , and the global classifier , to construct the loss of two local personalized discriminators, namely the prototype discriminator The loss and classifier discriminator Losses:

[0134] For the prototype discriminator The loss function is used for training :

[0135] (9)

[0136] In formula (9), Indicates the The client in The loss when training the prototype discriminator, Indicates belonging No. Class, Therefore The logarithmic function with base , is the prototype discriminant function, is the local classification function, Indicates the The local classifier after the local update, Indicates that it belongs to The global feature prototype of a class, Indicates that the first Local feature prototypes of each class;

[0137] For the classifier discriminator The loss function is used for training :

[0138] (10)

[0139] In formula (10), Indicates the The client in The loss during the round of training of the classifier discriminator, Indicates belonging No. Class, Therefore The logarithmic function with base , is the classifier discriminant function, represents the global classification function, Indicates the The global classifier obtained after aggregation, Indicates that it belongs to The global feature prototype of a class, is the local classification function, Indicates the Local classifier after local update;

[0140] Based on the above two discriminator losses, the first-order adversarial training loss of the local classifier as the generator is obtained , used for training Through this process, the local classifier is helped to obtain the generalization knowledge in the global classifier, thereby enhancing the generalization ability of the local personalized classifier. The first-order adversarial training loss as follows:

[0141] (11)

[0142] In formula (11), Indicates the Client No. The first-order adversarial training loss during round training, is the prototype discriminant function, is the local classification function, Indicates the The local classifier after the local update, Indicates that the first The local feature prototype of each class, is the classifier discriminant function, Indicates that it belongs to The global feature prototype of a class.

[0143] At the same time, the cross entropy loss formed by the local prototype is added to obtain the second-order adversarial training loss (This can be referred to as adversarial training loss), which effectively alleviates the local classifier's forgetting of local knowledge when adapting to local tasks:

[0144] (12)

[0145] So the total loss of training the local classifier is for:

[0146] (13)

[0147] The local classifier is trained using stochastic gradient descent to obtain the trained local classifier :

[0148] (14)

[0149] For the feature extractor, the global feature prototype is used to guide the training of the local feature extractor, helping the local feature extractor improve its generalization ability:

[0150] (15)

[0151] (16)

[0152] in, represents the loss formed using the global feature prototype, represents the Euclidean norm, Indicates that it belongs to The global trait prototype of the class, represents the local feature extractor trained in round t, Represents the local classifier after the t-th round of training.

[0153] S4. After completing the first round of local training, each client calculates the upload round set of the feature extractor before uploading local information to the server. Subsequent rounds do not need to be recalculated. Then each client recalculates the local feature prototype and uploads the following local information to the server: the recalculated local feature prototype, local feature extractor, local classifier, the total number of training samples owned by the client, and the number of training samples in each category. The sample number information is uploaded only after the first round of training is completed. If the current round is in the upload round set, the local feature extractor is uploaded. The details are as follows:

[0154] After the client completes the first round of local training and before uploading local information to the server, it calculates the ratio of the local feature extractor to the local feature prototype parameters. ,pass To achieve selective uploading of local feature extractors, this method can effectively reduce the communication cost of the entire system. The local feature extractor is uploaded only when necessary, thereby reducing unnecessary communication overhead. If the training of the local feature extractor is guided by the global feature prototype alone, it may not be possible to ensure that the local feature extractor obtains sufficient generalization ability. At the same time, if the feature extractor is uploaded in each round, on the one hand, it will bring too high a communication cost, and on the other hand, it may also cause the personalization ability of the local feature extractor to be excessively reduced. The ratio of the local feature extractor to the local feature prototype parameters is The calculation formula is as follows:

[0155] (17)

[0156] in Indicates the amount of calculation parameters;

[0157] Then Multiples of are removed from the communication rounds. In these rounds, the local feature extractor does not upload (the rest of the rounds upload), and the feature extractor upload round set is obtained.

[0158] Recalculate the local feature prototype to obtain the local feature prototype set to be uploaded :

[0159] (18)

[0160] (19)

[0161] In formulas (18) and (19), Indicates that the local feature prototype set is obtained after the local feature extractor is trained. Contains The local feature prototype of each class, class, represented by , Indicates the Client local dataset Belong to the Each sample data set of each class, express A single sample in represents the local feature extraction function, Indicates the The client local data set belongs to The size of the sample dataset for each class, Indicates the Local feature extractor after one round of local training.

[0162] S5. The server uses the category sample size and the total sample size as the basis for setting aggregation weights, aggregates the local feature prototypes and local classifiers uploaded by the client, and generates a global feature prototype and a global classifier. If a local feature extractor is uploaded in the current round, it performs weighted aggregation based on the total sample size to obtain a global feature extractor. The aggregated information is then sent to the clients participating in federated learning, as follows:

[0163] If the feature extractor is received in the current aggregation round, the aggregation weight is set based on the total number of client samples ( Indicates that based on The weight set by the total number of client samples) is used to obtain the global feature extractor :

[0164] (20)

[0165] (twenty one)

[0166] For the classifier, we also set the aggregation weight based on the total number of client samples to obtain the global classifier. :

[0167] (twenty two)

[0168] For feature prototype aggregation, the sample size of each class is used as the aggregation weight to obtain the global feature prototype:

[0169] (twenty three)

[0170] (twenty four)

[0171] (25);

[0172] In formula (23)-formula (25), Indicates that based on Client No. The weight set by the sample size of each class, Indicates the The client local data set belongs to The size of the sample dataset for each class ( Similarly), Indicates in After the global aggregation, we get the global feature prototype set. To include The global feature prototype of each class, class, represented by ,and Indicates that after The local feature prototype belonging to the kth class obtained after a round of local training.

[0173] S6. Repeat steps S2 to S5 until the local model converges or the preset number of communication rounds is reached.

[0174] The effectiveness of the present invention is verified by the following specific experiments:

[0175] The experimental configuration and experimental results of the present invention include the following:

[0176] (1) Dataset and model settings

[0177] The present invention conducts experiments on data collected by acceleration sensors to complete the task of motion recognition and classification. The dataset contains acceleration signal data of multiple motion categories, and each sample records the time series signals of different motions. The training set contains a large amount of acceleration data, and the test set is used to evaluate the performance of the model. In the experiment, the model used is a three-layer convolutional neural network. The number of convolution filters in each layer of the network is 16, 32, and 64, respectively, and the sizes of the convolution kernels are 5x5, 5x5, and 3x3, respectively. The activation function uses the rectified linear unit, and is combined with the maximum pooling operation for feature dimensionality reduction. Finally, through a fully connected layer containing 128 neurons, it is connected to an output layer corresponding to the number of motion categories for classification prediction and completion of classification tasks related to the industrial Internet of Things.

[0178] (2) Training settings and data segmentation

[0179] During local training, the feature extractor and classifier are trained separately, with each component trained independently for three rounds. The global communication round count is set to 100. Each client is trained with 300 samples, 20% of which is evenly distributed across all classes, and the remaining 80% is randomly assigned to three specific classes. This data distribution effectively simulates the heterogeneous nature of industrial IoT data.

[0180] (3) Comparison method

[0181] The compared FedAvg, FedPer, FedProto, MOON, and FedRep are all federated learning methods, among which FedAvg and MOON are traditional federated learning methods, and FedPer, MOON, and FedRep are personalized federated learning methods.

[0182] (4) Experimental results

[0183] In order to verify the generalization ability and personalization effect of the present invention under industrial Internet of Things data, experiments were conducted under different distribution conditions and the same distribution conditions of the training set and test set. The experimental results are as follows: Figure 3 and Figure 4 As shown. In both scenarios, the method of the present invention shows an average accuracy rate that is better than the comparison method (i.e., the average of the accuracy rates of each client). In addition, in order to evaluate the convergence of the method, a convergence test was conducted to calculate the loss value before and after model training. Figure 5 As shown, after about 80 rounds of training, the loss value of the method of the present invention tends to be stable and the model gradually converges.

[0184] Example 2

[0185] This embodiment also provides a device for implementing a federated learning method based on dual adversarial training in an industrial Internet of Things, including:

[0186] at least one processor; and

[0187] A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to execute the above-mentioned federated learning method based on dual adversarial training under the industrial Internet of Things.

[0188] In this embodiment, the device may include but is not limited to: a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile computing device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable computing device, a consumer electronic device, and the like.

[0189] Example 3

[0190] This embodiment also provides a machine-readable storage medium storing executable instructions, which, when executed, enable the machine to execute the above-described federated learning method based on dual adversarial training under the industrial Internet of Things.

[0191] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program codes that implement the functions of any of the above-mentioned embodiments are stored, and a computer or processor of the system or device can read and execute instructions stored in the readable storage medium.

[0192] In this case, the program code itself read from the machine-readable medium can implement the functions of any one of the above embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute part of this specification.

[0193] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-Rs, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, DVD-RWs), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.

[0194] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A federated learning method based on dual adversarial training in the industrial Internet of Things, characterized by: The method comprises: S1. The server initializes the global model and sends it to each client participating in federated learning, where the client refers to an industrial device or terminal. S2. Each client updates its local model based on the received global information: When the local model is updated for the first time, each client initializes the local model to the received global model; In subsequent rounds, when the client receives a new round of global feature extractors from the server, the client updates the local feature extractor, while the local classifier remains unchanged and is not updated; S3. After completing the local model update, each client enters the local training phase; The feature extractor and classifier in the local model are trained independently in stages, with the local classifier trained first and then the local feature extractor. In the first round of local training, the local feature extractor and classifier are trained solely based on the data-driven cross-entropy loss. In non-first rounds of local training, the feature extractor is constrained by both the cross-entropy loss and the global feature prototype loss, while the classifier is optimized using the cross-entropy loss and the adversarial training loss. S4. After completing the first round of local training, each client calculates the upload round set of the feature extractor before uploading local information to the server. Subsequent rounds do not need to be calculated repeatedly. Then each client recalculates the local feature prototype and uploads the following local information to the server: the recalculated local feature prototype, local feature extractor, local classifier, the total number of training samples owned by the client, and the number of training samples in each category. The sample number information is uploaded only after the first round of training is completed. If the current round is in the upload round set, the local feature extractor is uploaded. Calculating the upload round set of the feature extractor means: calculating the ratio of the local feature extractor to the local feature prototype parameters. , then Multiples of are eliminated from the communication rounds. In these rounds, the local feature extractor does not upload, and the feature extractor upload round set is obtained; S5. The server uses the category sample size and the total sample size as the basis for setting aggregation weights, aggregates the local feature prototypes and local classifiers uploaded by the client, and generates a global feature prototype and a global classifier. If a local feature extractor is uploaded in the current round, it performs weighted aggregation based on the total sample size to obtain a global feature extractor. The aggregated information is then sent to the clients participating in federated learning. S6. Repeat steps S2 to S5 until the local model converges or the preset number of communication rounds is reached.

2. The federated learning method based on dual adversarial training under the industrial Internet of Things according to claim 1 is characterized in that: Step S2 specifically includes: For the first round of local model update, both the local feature extractor and the local classifier are updated using the global feature extractor and the global classifier: （1） in, Indicates the local feature extractor after the first round of update, represents the global feature extractor, Indicates the local classifier after the first round of update, denotes a global classifier, the superscript Indicates the current state after updating and before training; For the When the local model is updated, , client The judgment is completed Does the latest content downloaded by the server after the aggregation contain the global feature extractor? If so, the local feature extractor is updated using this global feature extractor: （2） in, Indicates the After the local update, sub-local pre-trained feature extractor, Indicates the The global feature extractor obtained after aggregation on the secondary server side; If not, the feature extractor continues to use the Sub-local update of the trained feature extractor : （3） For the local classifier, continue to use the Local classifier after local update training: （4） in, Indicates the After the local update, The local classifier before local training, Indicates the Sub-local update of the trained local classifier.

3. The federated learning method based on dual adversarial training under the industrial Internet of Things according to claim 2 is characterized in that: Step S3 specifically includes: For the first round of local training, both the local classifier and feature extractor are trained using only data-driven cross-entropy loss: （5） （6） In formula (5) and formula (6), Indicates the local classifier trained in the first round, Indicates the local classifier after the first round of update, represents the gradient of the loss associated with the local classifier, represents the samples in the local dataset, Represents the client's local dataset, represents the cross entropy loss, Represents the input feature information of the image, Represents the label corresponding to the input feature information of the image, represents the local feature extractor trained in the first round, Indicates the local feature extractor after the first round of update, Represents the gradient of the loss associated with the local feature extractor.

4. The federated learning method based on dual adversarial training under the industrial Internet of Things according to claim 3 is characterized in that: Step S3 specifically also includes: For the During local training, When , the local classifier is trained using a combination of adversarial training loss and data-driven cross entropy loss: Because local feature prototypes are needed in adversarial loss, the current local feature prototypes need to be calculated before this operation: （7） （8） in, Indicates that the local feature prototype set is obtained before the local classifier and local feature extractor are trained. Contains The local feature prototype of each class, class, represented by , Indicates the Client local dataset Belong to the Each sample data set of each class, express A single sample in represents the local feature extraction function, Indicates the The client local data set belongs to The size of the sample dataset for each class, Indicates the Local feature extractor before training after round of local updates; Then download the global feature prototype set of the previous round from the server , and the global classifier , to construct two local personalized discriminator losses, namely the prototype discriminator The loss and classifier discriminator Losses: For the prototype discriminator The loss function is used for training : （9） In formula (9), Indicates the The client in The loss when training the prototype discriminator, Indicates belonging No. Class, Therefore The logarithmic function with base , is the prototype discriminant function, is the local classification function, Indicates the The local classifier after the local update, Indicates that it belongs to The global feature prototype of a class, Indicates that the first Local feature prototypes of each class; For the classifier discriminator The loss function is used for training : （10） In formula (10), Indicates the The client in The loss during the round of training of the classifier discriminator, Indicates belonging No. Class, Therefore The logarithmic function with base , is the classifier discriminant function, represents the global classification function, Indicates the The global classifier obtained after aggregation, Indicates that it belongs to The global feature prototype of a class, is the local classification function, Indicates the Local classifier after local update; Based on the above two discriminator losses, the first-order adversarial training loss of the local classifier as the generator is obtained , used for training : （11） In formula (11), Indicates the Client No. The first-order adversarial training loss during round training, is the prototype discriminant function, is the local classification function, Indicates the The local classifier after the local update, Indicates that the first The local feature prototype of each class, is the classifier discriminant function, Indicates that it belongs to The global feature prototype of each class; At the same time, the cross entropy loss formed by the local prototype is added to obtain the second-order adversarial training loss : （12） So the total loss of training the local classifier is for: （13） The local classifier is trained using stochastic gradient descent to obtain the trained local classifier : （14） For the feature extractor, the global feature prototype is used to guide the training of the local feature extractor: （15） （16） in, represents the loss formed using the global feature prototype, represents the Euclidean norm, Indicates that it belongs to The global trait prototype of the class, represents the local feature extractor trained in round t, Represents the local classifier after the t-th round of training.

5. The federated learning method based on dual adversarial training under the industrial Internet of Things according to claim 4 is characterized in that: Step S4 specifically includes: After the client completes the first round of local training and before uploading local information to the server, it calculates the ratio of the local feature extractor to the local feature prototype parameters. : （17） in Indicates the amount of calculation parameters; Then Multiples of are eliminated from the communication rounds. In these rounds, the local feature extractor does not upload, and the feature extractor upload round set is obtained; Recalculate the local feature prototype to obtain the local feature prototype set to be uploaded : （18） （19） In formulas (18) and (19), Indicates that the local feature prototype set is obtained after the local feature extractor is trained. Contains The local feature prototype of each class, class, represented by , Indicates the Client local dataset Belong to the Each sample data set of each class, express A single sample in represents the local feature extraction function, Indicates the The client local data set belongs to The size of the sample dataset for each class, Indicates the Local feature extractor after one round of local training.

6. The federated learning method based on dual adversarial training under the industrial Internet of Things according to claim 5 is characterized in that: Step S5 specifically includes: If the feature extractor is received in the current aggregation round, the aggregation weight is set based on the total number of client samples to obtain the global feature extractor : （20） （21） In formula (20), Indicates that based on The weight set for the total number of client samples; For the classifier, we also set the aggregation weight based on the total number of client samples to obtain the global classifier. : （22） For feature prototype aggregation, the sample size of each class is used as the aggregation weight to obtain the global feature prototype: （23） （24） （25）； In formula (23)-formula (25), Indicates that based on Client No. The weight set by the sample size of each class, Indicates the The client local data set belongs to The size of the sample dataset for each class, Similarly, Indicates in After the second global aggregation, we get the global feature prototype set. To include The global feature prototype of each class, class, represented by ,and Indicates that after The local feature prototype belonging to the kth class obtained after a round of local training.

7. The federated learning method based on dual adversarial training under the industrial Internet of Things according to claim 1 is characterized in that: The global information described in S2 contains a global feature extractor and a global classifier in the initial round, and these two parts constitute the global model; in non-initial rounds, it must contain a global feature prototype and a global classifier. If the server has a newly aggregated global feature extractor, it contains a global feature extractor.

8. A device for implementing a federated learning method based on dual adversarial training in the industrial Internet of Things, characterized in that: The device comprises: at least one processor; and A memory storing instructions, which, when executed by the at least one processor, causes the at least one processor to perform the federated learning method based on dual adversarial training under the industrial Internet of Things as described in any one of claims 1 to 7.

9. A machine-readable storage medium, characterized in that The machine-readable storage medium stores executable instructions, which, when executed, enable the machine to execute the federated learning method based on dual adversarial training under the industrial Internet of Things according to any one of claims 1 to 7.

Images