Industrial Internet of Things federal learning method based on federal increment decision tree

Abstract

The invention discloses an industrial Internet of Things federated learning method based on a federated increment decision tree, and the method comprises the steps: a cloud server deploys and initializes a federated learning global model and the federated increment decision tree, and sets a statistical histogram bucket boundary set of each feature value; in each federated learning iteration, the cloud server broadcasts a federated learning global model parameter, each industrial device adopts a local data set to train and count to obtain a local gradient histogram parameter, the edge server performs local aggregation on the local gradient histogram parameter and the federated learning model parameter, and the edge server performs local aggregation on the local gradient histogram parameter and the federated learning global model parameter; and the cloud server globally aggregates the local aggregation parameters of the gradient histogram and then incrementally trains the federated increment decision tree, simultaneously aggregates the parameters of a federated learning global model, and adaptively calculates an aggregation weight based on a second-order gradient value during local aggregation and global aggregation of the parameters of the federated learning model. The federal learning overhead can be effectively reduced, and the performance of the federal learning model is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of industrial internet of things federated learning, and more specifically relates to an industrial internet of things federated learning method based on federated incremental decision tree. BACKGROUND

[0002] Industrial internet of things (IIoT) is evolved from the integration of internet of things (IoT) and advanced industrial production system (AIPS), and a large amount of sensing and monitoring data is generated by multi-source heterogeneous sensors at the production line and device end in real time, which provides data support for training machine learning model to optimize production decision, and has great potential in intelligent applications such as device state monitoring, fault prediction and diagnosis, production scheduling optimization.

[0003] As a distributed machine learning framework, federated learning allows each heterogeneous device to keep data locally and collaboratively train a global model, and has been widely applied in industrial internet of things scenarios. Traditional federated learning schemes usually rely on a central server to aggregate complete gradients or model parameters, which has large communication overhead and energy consumption; under the conditions of multi-source non-independent and identically distributed data and dynamic network, the model convergence speed and accuracy are further reduced. Especially for decision tree models, high-dimensional continuous features need to construct fine histogram, and the distributed statistics and synchronization process will significantly increase the communication burden. The existing methods generally lack a model updating method that takes into account real-time performance, energy consumption and accuracy, and it is difficult to meet the dual needs of fast model evolution and low resource consumption in industrial internet of things scenarios. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art and provide an industrial internet of things federated learning method based on federated incremental decision tree, which adopts distributed feature bucketing and gradient histogram statistics to construct federated incremental decision tree, and realizes collaborative training across industrial devices through federated aggregation, thereby reducing the overhead of federated learning and improving the performance of federated learning model.

[0005] In order to achieve the above-mentioned purpose of the application, the industrial internet of things federated learning method based on federated incremental decision tree comprises the following steps:

[0006] S1: Construct an industrial internet of things federated learning network, including a cloud layer, a federated layer and an industrial device layer, wherein:

[0007] The cloud layer includes a cloud server for hosting a federated learning global model and a federated incremental decision tree;

[0008] The federated layer is a communication relay between the cloud layer and the industrial device layer, including One edge server, each edge server The data uploaded by industrial equipment within its coverage area is partially aggregated and then uploaded to the cloud server. ;

[0009] The industrial equipment layer includes all industrial equipment in the Industrial Internet of Things (IIoT), and each industrial device... This is used to mount a local federated learning model and collect local datasets to train the local federated learning model. , Indicates the quantity of industrial equipment;

[0010] S2: Initialize the parameters of the federated learning global model on the cloud server in the cloud layer. The federated incremental decision tree is initialized according to the federated learning model, with the input being the feature vector of the industrial equipment and the output being the predicted label value.

[0011] Cloud server settings for statistical histogram bucket boundary sets of each feature value in the feature vector of industrial equipment And broadcast to various industrial devices through the federal layer. ,in Representation of features The minimum value, Representation of features The The boundary of each bucket, , This indicates the number of features in the eigenvector. , Indicates the total number of buckets;

[0012] S3: Increase the number of federated learning iterations ;

[0013] S4: Cloud servers connect to various industrial devices in the Industrial Internet of Things (IIoT) through the federation layer. Broadcast Federated Learning Global Model Parameters Each industrial device Received global model parameters for federated learning Then, initialize its federated learning local model parameters. ;

[0014] S5: Each industrial device Collect local datasets ,in Indicates the first in the local dataset The feature vector of each sample Represents the features in the eigenvector The value, Representing the eigenvector The corresponding tags denotes the number of individuals in the set; each industrial device adopts the local dataset collected by itself federated learning local model of itself performs local training to obtain federated learning local model parameters and local loss ;

[0015] For each feature , the industrial device performs bucket index mapping on each feature value in the local dataset to obtain the corresponding bucket number ; calculates the gradient , the second-order gradient of the federated learning local model loss for each sample , and then calculates the gradient , the second-order gradient accumulation value , and the second-order gradient expectation value of the local gradient histogram within each feature bucket using the following formula :

[0016] ,

[0017] ,

[0018]

[0019] wherein, denotes the indicator function;

[0020] Then, the number of samples in the local dataset of the industrial device that fall within the feature bucket is counted ;

[0021] Each industrial device uploads the calculated gradient , second-order gradient accumulation value , second-order gradient expectation value , sample number , local dataset size , local model parameters , and local loss to the corresponding edge server;

[0022] S6: Each edge server receives the gradients , second-order gradient accumulation values second order gradient expected value , sample number , local dataset size , local model parameter and local loss , local gradient histogram aggregation, sample count aggregation, local model parameter aggregation and local loss aggregation are performed respectively to obtain each edge server For each feature , gradient local aggregation value , second order gradient local aggregation value , second order gradient expected local aggregation value , sample count local aggregation value , local model parameter and local loss , and upload to the cloud server; wherein the specific method of local model parameter aggregation is as follows:

[0023] The normalized weight of the industrial equipment is calculated by the following formula:

[0024] ,

[0025] wherein, represents the initial weight of the industrial equipment , and represents the set of industrial equipment covered by the edge server .

[0026] The maximum weight of the industrial equipment is calculated by the following formula:

[0027] ,

[0028] The local model parameter is calculated by the following formula:

[0029] ,

[0030] The local loss is calculated by the following formula:

[0031] ;

[0032] S7: When the cloud server receives the gradient local aggregation value , second order gradient local aggregation value , second order gradient expected local aggregation value of each feature uploaded by each edge server ​​, sample count local aggregation value , local model parameter , and local loss , the global gradient histogram aggregation is first performed, and the obtained gradient global aggregation value and the second-order gradient local aggregation value The federated incremental decision tree is incrementally trained, and then the global model parameter aggregation and global loss aggregation are performed to obtain the federated learning global model parameter , global loss The specific method of global model parameter aggregation is as follows:

[0033] The normalized weight of the edge server is calculated using the following formula:

[0034] ,

[0035] Wherein, the initial weight of the edge server ; The maximum weight of the edge server

[0036] is calculated using the following formula:

[0037] ,

[0038] The global model parameter is calculated using the following formula:

[0039] ,

[0040] The global loss is calculated using the following formula:

[0041] ;

[0042] S8: Determine whether the federated learning end condition is reached, if yes, the federated learning ends, otherwise go to step S9; The specific determination method is as follows:

[0043] First, the redundancy coefficient of each leaf node of the current federated incremental decision tree is calculated:

[0044] ,

[0045] Wherein, , Indicates the leaf node set of the current federated incremental decision tree, Indicates the leaf node set of the current federated incremental decision tree,​​​​ and the second order gradient local aggregation value the derived leaf node corresponding data set, representing the data set samples in the data set, , representing the gradient and the second order gradient of the sample representing the preset regularization parameter;

[0046] Then the redundancy loss of the federated incremental decision tree is calculated by the following formula :

[0047] ,

[0048] wherein, representing the preset redundancy coefficient threshold value;

[0049] determining whether and , wherein representing the redundancy loss of the federated incremental decision tree obtained in the last iteration, representing the global loss of the federated learning global model obtained in the last iteration, , representing the preset threshold value, if yes, the federated learning end condition is reached, otherwise not;

[0050] S9: let the iteration number , return to step S4.

[0051] The industrial internet of things federated learning method based on the federated incremental decision tree sets the federated learning global model and the federated incremental decision tree in the cloud server, initializes the federated learning global model and the federated incremental decision tree in the cloud server, and sets the statistical histogram bucket boundary set of each feature value. In each federated learning iteration, the cloud server broadcasts the federated learning global model parameters to the industrial devices through the federated layer, each industrial device trains the local data set, and the local gradient histogram parameters are obtained by statistics, the edge server in the federated layer performs local aggregation on the local gradient histogram parameters of the industrial devices and the federated learning model parameters, the cloud server performs global aggregation on the gradient histogram local aggregation parameters, and then performs incremental training on the federated incremental decision tree, and performs global parameter aggregation on the federated learning model, when the federated learning model parameters are locally aggregated and globally aggregated, the second order gradient value is adaptively calculated to calculate the aggregation weight.

[0052] The present application has the following beneficial effects:

[0053] ​1) The present application combines federal incremental decision tree training and federal learning model training, and adaptively calculates the aggregation weight of the federal learning model parameter through the second-order gradient value, so that the federal learning model parameter aggregation is more in line with the data distribution characteristics;

[0054] 2) The present application realizes dynamic collaborative optimization of energy consumption and network bandwidth by greatly compressing the uplink communication data volume and reducing the local computing load through distributed feature bucketing and gradient histogram aggregation;

[0055] 3) The present application can also introduce a dynamic pruning strategy of federal incremental decision tree in each global iteration, taking into account model accuracy and convergence speed, and further reducing redundant operations;

[0056] 4) The present application can also dynamically adjust the statistical histogram bucket boundary set and the redundancy coefficient threshold of the leaf node in each global iteration, further improving the convergence speed. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is the specific implementation flowchart of the industrial internet of things federal learning method of the present application based on federal incremental decision tree;

[0058] Figure 2 is the training loss comparison chart of the present application and three baseline methods on different data sets in the present embodiment;

[0059] Figure 3 is the test set accuracy comparison chart of the present application and three baseline methods on different data sets in the present embodiment;

[0060] Figure 4 is the comparison chart of the average aggregation times of different channel states on different data sets in the present embodiment;

[0061] Figure 5 is the comparison chart of the time consumption of different clustering conditions on different data sets in the present embodiment;

[0062] Figure 6 is the convergence rate comparison chart of Fed-DGBDT and three baseline methods on different data sets in the present embodiment;

[0063] Figure 7 is the comparison chart of the energy consumption of different channel states on different data sets in the present embodiment. DETAILED DESCRIPTION

[0064] The specific embodiments of the present application will be described below with reference to the accompanying drawings, so that those skilled in the art can better understand the present application. It should be particularly noted that in the following description, when the detailed description of known functions and designs may obscure the main content of the present application, these descriptions will be omitted here.

[0065] Example

[0066] Figure 1 This is a flowchart illustrating a specific implementation of the federated learning method for the Industrial Internet of Things based on federated incremental decision trees, as described in this invention. Figure 1 As shown, the specific steps of the federated learning method for industrial IoT based on federated incremental decision trees in this invention include:

[0067] S101: Building a Federated Learning Network for the Industrial Internet of Things

[0068] First, an industrial IoT federated learning network is constructed, comprising a cloud layer, a federated layer, and an industrial device layer, wherein:

[0069] The cloud layer includes cloud servers, which are used to host federated learning global models and federated incremental learning models.

[0070] The federal layer is a communication relay between the cloud layer and the industrial equipment layer, including One edge server, each edge server The system aggregates industrial equipment within its coverage area and uploads it to a cloud server. .

[0071] The industrial equipment layer includes all industrial devices in the Industrial Internet of Things (IIoT), each industrial device Used to host local models and collect local datasets to train local models. , Indicates the number of industrial equipment.

[0072] S102: Initialize model parameters:

[0073] Initialize the parameters of the federated learning global model on the cloud server in the cloud layer. The federated incremental decision tree is initialized based on the federated learning model. Its input is the feature vector of the industrial equipment, and its output is the predicted label value. The features and label values ​​in the feature vector can be set according to the types of industrial equipment included in the Industrial Internet of Things and the specific task of the federated learning model. Generally, the features include mechanical motion state features (speed, vibration, displacement, torque, etc.), electrical signal state features (operating current, operating voltage, power, load, etc.), and other features (operating temperature, etc.). Assuming that the federated learning model is a classification task (judging the fault risk level of industrial equipment), then the label value is the category.

[0074] In addition, to enable the learning of the federated incremental decision tree, the cloud server also needs to set the statistical histogram bucket boundary set for each feature value in the feature vector of industrial equipment. And broadcast to various industrial devices through the federal layer. ,in Representation of features The minimum value, Representation of features The The boundary of each bucket, , This indicates the number of features in the eigenvector. , This indicates the total number of buckets.

[0075] S103: Increase the number of federated learning iterations .

[0076] S104: Broadcast the current federated learning model parameters:

[0077] Cloud servers connect to various industrial devices in the Industrial Internet of Things (IIoT) through the federated layer. Broadcast Federated Learning Global Model Parameters Each industrial device Received global model parameters for federated learning Then, initialize its federated learning local model parameters. .

[0078] S105: Device-level local training and data statistics:

[0079] Each industrial equipment Collect local datasets ,in Indicates the first in the local dataset. The feature vector of each sample Represents the features in the eigenvector The value, Representing the eigenvector The corresponding tags This indicates the number of individuals in a set. Each industrial device... Using its collected local dataset The parameters of the federated learning local model are obtained by training the local model locally. and local losses .

[0080] Each industrial equipment The local dataset collected from it Discretization is performed so that subsequent histogram aggregation can complete statistical analysis without exposing the original data. Specifically, for any feature... Industrial equipment Local dataset Each eigenvalue Perform bucket index mapping to obtain the corresponding bucket number. The specific process of bucket index mapping can be defined by the following formula:

[0081] ,

[0082] After the completion of the bucketing, the following formula is used to calculate each sample For the gradient of the local model loss of the federated learning And the second-order gradient :

[0083] ,

[0084] ,

[0085] Wherein, Indicates the loss function of the sample , indicates the predicted label of the feature vector of the local model of the industrial equipment . In order to summarize the statistical information without leaking the original gradient vector, the industrial equipment

[0086] in the present application accumulates the sample gradient according to the bucket number to form a local gradient histogram, which can be regarded as a private summary of the feature distribution, which not only compresses the communication amount, but also avoids the leakage of the original feature. The calculation formula of the gradient and the second-order gradient accumulation value of the local gradient histogram is as follows:

[0087] ,

[0088] ,

[0089] Wherein, Indicates the indicator function.

[0090] At the same time, the expected value of the second-order gradient is calculated:

[0091] ,

[0092] Then the number of samples in the local data set of the industrial equipment falling into the feature bucket is counted:

[0093] ,

[0094] After the above statistics, each industrial equipment will calculate the gradient of each feature, the second-order gradient accumulation value ​​second order gradient expectation value sample number local dataset size local model parameter and local loss to the corresponding edge server.

[0095] S106: Federated layer local aggregation:

[0096] Each edge server receives the uploaded gradient second order gradient accumulation value second order gradient expectation value sample number local dataset size local model parameter and local loss from the industrial devices within its coverage range, and then performs local gradient histogram aggregation, sample count aggregation, local model parameter aggregation, and local loss aggregation, respectively, to obtain the gradient local aggregation value for each feature second order gradient local aggregation value second order gradient expectation local aggregation value sample count local aggregation value local model parameter and local loss of the edge server and uploads them to the cloud server.

[0097] The local gradient histogram aggregation aims to reduce communication overhead and improve privacy protection level before uploading data to the cloud server. In this embodiment, for any feature in the global feature space and any bucket, the gradient local aggregation value of the edge server is represented as the sum of the gradient sums of all devices in the coverage range on the feature bucket, i.e.:

[0098] ,

[0099] wherein denotes the set of industrial devices covered by the edge server .

[0100] Correspondingly, the second order gradient local aggregation value of the edge server is represented as the sum of the second order gradient sums of all devices in the coverage range on the feature bucket, i.e.:

[0101] ,

[0102] Second-order gradient expected local aggregation value The aggregation formula is:

[0103] .

[0104] Sample count aggregation value is expressed as the sum of the number of samples of all devices in the coverage falling into the feature bucket, that is:

[0105] ,

[0106] The specific method of local model parameter aggregation is as follows:

[0107] The normalized weight of the industrial equipment is calculated by the following formula:

[0108] ,

[0109] wherein, represents the initial weight of the industrial equipment , represents the set of industrial equipment covered by the edge server . In this embodiment, the initial weight is calculated by the following formula:

[0110] ,

[0111] The maximum weight of the industrial equipment is calculated by the following formula:

[0112] , The local model parameter

[0113] is calculated by the following formula:

[0114] .

[0115] Correspondingly, the local loss is calculated by the following formula:

[0116] .

[0117] S107: Cloud global aggregation:

[0118] When the cloud server receives the gradient local aggregation value , the second-order gradient local aggregation value , and the second-order gradient expected local aggregation value of each feature uploaded by each edge server,​​​​ , sample count local aggregation value , local model parameter and local loss After that, first perform global gradient histogram aggregation and adopt the obtained gradient global aggregation value and second-order gradient local aggregation value Incremental training is performed on the federated incremental decision tree, and then global model parameter aggregation and global loss aggregation are performed to obtain the federated learning global model parameter , global loss The specific method of global model parameter aggregation is as follows:

[0119] The normalized weight of the edge server is calculated as follows:

[0120] ,

[0121] wherein, the initial weight of the edge server is calculated as follows:

[0122] ,

[0123] The maximum weight of the edge server is calculated as follows:

[0124] ,

[0125] The global model parameter is calculated as follows:

[0126] ,

[0127] The global loss is calculated as follows:

[0128] .

[0129] In this embodiment, the local aggregated gradient histogram is aggregated by weighted merging to ensure that the global model fully represents the data distribution of each party. Therefore, the specific calculation formula of the global gradient histogram aggregation in this embodiment is as follows:

[0130]

[0131]

[0132] wherein, the initial weight of the edge server is calculated as follows: ​​​​​The weights corresponding to the global gradient histogram aggregation can be set empirically, as in this embodiment. .

[0133] Incremental training of decision trees based on gradient histograms is a commonly used technique, and its specific process will not be elaborated here. However, as the depth of the tree increases, the marginal contribution of some leaf nodes to the global error gradually decreases. Without processing, this will lead to increased communication and storage redundancy. Therefore, this embodiment prunes the federated incremental decision tree based on the redundancy coefficient of the leaf nodes, as follows:

[0134] Each leaf node is calculated using the following formula. Redundancy coefficient :

[0135] ,

[0136] in, , This represents the set of leaf nodes in the current federated incremental decision tree. This indicates that the cloud server aggregates values ​​based on degree. and local aggregation value of second-order gradient The derived leaf nodes The corresponding dataset, Represents the dataset The samples in , Indicates sample The gradient and the second gradient, This represents the preset regularization parameters.

[0137] When leaf node Redundancy coefficient Less than the preset redundancy coefficient threshold When, the leaf node Its sibling leaf node The gradients and second-order gradients of the two nodes are merged, and the sum of their gradients and second-order gradients is used as the gradients and second-order gradients of the new leaf node, thus maintaining the consistency of gradients before and after merging.

[0138] During the global aggregation phase, to ensure that the feature discretization strategy can adapt to the dynamic changes in data distribution, the cloud server can also update the feature bucket boundaries based on the global gradient information obtained in this round of training. The specific method is as follows:

[0139] First, calculate each feature. The Gradient density of each bucket , defined as the product of the average sample gradient magnitude within the bucket and the number of samples, is used to measure the importance of the data in the interval where the bucket is located. Therefore, its calculation formula is as follows:

[0140] ,

[0141] wherein, denotes the sum of the sample number of the i-th bucket of the feature .

[0142] Subsequently, the bucket boundaries are re-divided according to the gradient density distribution, aiming to divide the data range into several intervals, so that the sum of the gradient density in each interval is approximately equal. The bucket boundaries are updated by optimizing the following objective function:

[0143]

[0144] wherein, and are the minimum and maximum values of the current value range of the feature , respectively.

[0145] S108: Determine whether the federated learning end condition is reached, i.e., whether the condition is met, if yes, the federated learning ends, otherwise, proceed to step S109. The specific method of the federated learning end condition in the present application is as follows:

[0146] First, the redundancy coefficient of each leaf node of the current federated incremental decision tree is calculated:

[0147] ,

[0148] wherein, , denotes the leaf node set of the current federated incremental decision tree, denotes the data set corresponding to the leaf node derived by the cloud server according to the gradient global aggregation value and the second-order gradient local aggregation value , denotes the sample in the data set , , denotes the gradient and second-order gradient of the sample .

[0149] Then, the redundancy loss of the federated incremental decision tree is calculated by the following formula:

[0150] ,

[0151] ​​​​The present application needs to confirm whether the current federated learning model has met the dual requirements of precision and compression, that is, to determine whether and wherein represents the redundancy loss of the federated incremental decision tree obtained in the last iteration, represents the global loss of the federated learning global model obtained in the last iteration, , represents a preset threshold, if yes, the federated learning end condition is reached, otherwise it is not reached.

[0152] In order to make the pruning strategy of the federated incremental decision tree adaptive to the change of data distribution, and make the federated learning end condition more reasonable, the redundancy coefficient threshold is dynamically adjusted, and the dynamic adjustment formula is as follows:

[0153] ,

[0154] wherein, represents the learning rate.

[0155] S109: Let the iteration number , return to step S104.

[0156] In order to better illustrate the technical effect of the present application, specific examples are used to experimentally verify the present application. Since image data can be analogized as state feature maps collected by various sensors in industrial Internet of Things, MNIST, Fashion-MNIST and CIFAR10 data sets are used for verification and evaluation in this embodiment. Through experimental verification on different types of data sets, the adaptability and stability of the present application in multi-source heterogeneous features and complex scenes can be effectively tested. Table 1 is the data set information table in this embodiment.

[0157]

[0158] Table 1

[0159] In order to better verify the effect of the present application, the present application (denoted as Fed-DGBDT) is compared with three baseline methods, including FedAvg, FedMA and FedProx. FedAvg is a classic algorithm in federated learning, mainly solving the problem of distributed training; FedMA considers the model structure matching between clients when aggregating; FedProx introduces a regularization term to enhance the robustness under data heterogeneity. In this embodiment, three experiments are set.

[0160] Experiment 1: Comparison of overall learning accuracy of Fed-DGBDT and three baseline methods. Figure 2is a training loss comparison chart of the present application and three baseline methods on different data sets in the embodiment. Figure 3 is a test set accuracy comparison chart of the present application and three baseline methods on different data sets in the embodiment. From the training loss, the Fed-DGBDT training loss on MNIST is reduced by 35.2%, 32.1%, and 28.5%, respectively; on Fashion-MNIST is reduced by 28.3%, 24.5%, and 21.8%, respectively; and on CIFAR10 is reduced by 23.4%, 17.6%, and 16.2%, respectively. From the test set accuracy, the Fed-DGBDT accuracy on MNIST is increased by 25.3%, 22.1%, and 18.6%, respectively; on Fashion-MNIST is increased by 20.5%, 17.2%, and 15.3%, respectively; and on CIFAR10 is increased by 18.4%, 12.6%, and 11.7%, respectively, which shows the performance advantages of Fed-DGBDT in training and testing on different data sets.

[0161] Table 2 is a performance comparison table of the present application and three baseline methods on different data sets in the embodiment. As shown in Table 2, the MAE, RMSE, R², and ACC four indicators reveal the significant differences of each model in prediction accuracy, error distribution, and fitting ability. From the results, Fed-DGBDT performs best in most cases, with lower MAE and RMSE values and higher R² and ACC values, indicating its stronger robustness and generalization ability in handling distributed data.

[0162]

[0163] Table 2

[0164] Experiment 2: Comparison of convergence speed of Fed-DGBDT and three baseline methods of the present application. Figure 4 is a comparison chart of the average aggregation times of different channel states on different data sets in the embodiment. Figure 5 is a comparison chart of time consumption of different clustering conditions on different data sets in the embodiment. Each cluster corresponds to an edge server. As shown in Figure 4 and Figure 5 , the present embodiment can significantly reduce the average aggregation time under different channel states, improving the communication efficiency of federated learning; and still maintains low time consumption under diversified cluster conditions, verifying its robustness and adaptability advantages.

[0165] Figure 6 is a comparison chart of the convergence rate of Fed-DGBDT and three baseline methods on different data sets in the embodiment. As shown in Figure 6As shown, Fed-DGBDT reaches stable accuracy faster than other models on MNIST (improving convergence efficiency by about 30%-40%); on Fashion-MNIST, its convergence curve is smoother and optimization is completed 20-30 rounds earlier; on CIFAR10, although the overall convergence is slower, Fed-DGBDT is still significantly better than the baseline model (the final accuracy gap is widened by 10%-15%), verifying its fast adaptation capability in complex data scenarios.

[0166] Experiment 3: Analysis of energy consumption of Fed-DGBDT under different channel conditions. Figure 7 is a comparison chart of energy consumption of different channel conditions of this embodiment on different data sets. As shown, Figure 7 with the increase of training rounds, the energy consumption of Fed-DGBDT under good channel conditions is significantly lower than that under medium channel conditions, especially on complex data sets (such as CIFAR10), the difference is more obvious, and the energy consumption can be reduced by 40%-60% at most.

[0167] Although the above describes the specific embodiments of the present application for the purpose of facilitating the understanding of the present application by those skilled in the art, it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, as long as various changes are within the spirit and scope of the present application defined and determined by the appended claims, all the inventions utilizing the concept of the present application are included in the protection.

Claims

1. An industrial Internet of Things federated learning method based on federated incremental decision tree, characterized in that, The method comprises the following steps: S1: constructing an industrial internet of things federated learning network, comprising a cloud layer, a federated layer and an industrial device layer, wherein: The cloud layer comprises a cloud server, used to carry a federated learning global model and a federated incremental decision tree; The federal layer is a communication relay between the cloud layer and the industrial device layer, including one edge server, each edge server locally aggregates data uploaded by the industrial devices within its coverage range and uploads the data to the cloud server, ; The industrial device layer includes all industrial devices in the industrial Internet of Things, each industrial device The local model of the federated learning is loaded, and the local data set is collected to train the local model of the federated learning, , represents the number of industrial devices; S2: the cloud server of the cloud layer initializes parameters of a federated learning global model and initializes a federated incremental decision tree according to the federated learning model, an input of which is a feature vector of the industrial equipment and an output of which is a predicted label value; The cloud server sets a statistical histogram bucket boundary set of each feature value in the industrial equipment feature vector and broadcast to each industrial equipment through the federal layer wherein denotes the minimum value of the feature , denotes the first bucket boundary of the feature , , , denotes the number of features in the feature vector , denotes the total number of buckets; S3: Let the number of federated learning iterations be ; S4: the cloud server transmits the federated learning global model parameters to each industrial device in the industrial Internet of Things through a federation layer broadcasts the federated learning global model parameters each industrial device receives the federated learning global model parameters after receiving the federated learning global model parameters, initializes its federated learning local model parameters ; S5: Each industrial device Collect local datasets ,in Indicates the first in the local dataset The feature vector of each sample Represents the features in the eigenvector The value, Representing the eigenvector The corresponding tags This indicates calculating the number of individuals in a set; each industrial device Using its collected local dataset For its federated learning local model Perform local training to obtain the parameters of the federated learning local model. and local losses ; For each feature , industrial equipment Map the local data set Each feature value Bucket index mapping, get the corresponding bucket number ; Calculate the gradient of each sample The local model loss of federated learning And the second order gradient Then calculate the gradient of the local gradient histogram in each feature bucket , the second order gradient accumulation value And the second order gradient expectation value : ， ， ， wherein denotes an indicator function; Then the number of samples falling within the feature buckets in the local dataset of industrial equipment is counted ;​​​ Each industrial device The gradient of each feature is calculated The second-order gradient accumulation value The second-order gradient expectation value The number of samples The local dataset size The local model parameters And the local loss are uploaded to the corresponding edge server; S6: each edge server the gradient uploaded by each industrial device in its coverage range the second-order gradient accumulated value the second-order gradient expected value the sample number the local dataset size the local model parameters and the local loss , respectively, the local gradient histogram aggregation, the sample count aggregation, the local model parameter aggregation and the local loss aggregation are performed to obtain the gradient local aggregation value of each edge server for each feature the second-order gradient local aggregation value the second-order gradient local aggregation value the second-order gradient expected local aggregation value the sample count local aggregation value the local model parameters and the local loss and upload to the cloud server; wherein the specific method of local model parameter aggregation is as follows: The normalized weight of the industrial equipment is calculated using the following formula :​ ， wherein, represents an initial weight of the industrial device , represents a set of industrial devices covered by the edge server ; The maximum weight of the industrial equipment is calculated using the following formula :​ ， The local model parameters are calculated using the following equation : ， The local loss is calculated using the following equation : ； S7: When the cloud server receives data from various edge servers Each feature uploaded Local aggregation value of gradient Second-order gradient local aggregation value Second-order gradient expectation local aggregation value Local aggregated values ​​of sample counts Local model parameters and local loss Then, firstly, global gradient histogram aggregation is performed, and the obtained global gradient aggregation value is used. and local aggregation value of second-order gradient Incremental training is performed on the federated incremental decision tree, followed by global model parameter aggregation and global loss aggregation to obtain the global model parameters for federated learning. Global loss The specific method for global model parameter aggregation is as follows: The normalized weight of the edge server is calculated using the following formula :​ ， wherein, represents the initial weight of the edge server . The edge server is calculated using the following formula : w = 1 - e : ， The global model parameters are calculated using the following equation : ， The global loss is calculated using the following equation : ； S8: determining whether a federated learning end condition is reached, if yes, the federated learning ends, otherwise, entering step S9; the specific determination method is as follows: First, the redundancy coefficient of each leaf node of the current federation incremental decision tree is calculated :​ ， wherein, , denotes a set of leaf nodes of the current federal incremental decision tree, denotes a leaf node derived by the cloud server according to a gradient global aggregation value and a second-order gradient local aggregation value corresponding to a data set, denotes the data set denotes a sample in the data set , , denotes a gradient and a second-order gradient of the sample , denotes a preset regularization parameter; The redundancy loss of the federal incremental decision tree is then calculated using the following formula : ， wherein, represents a predetermined redundancy coefficient threshold value; determining whether and wherein represents the redundancy loss of the federated incremental decision tree obtained in the last iteration, represents the global loss of the federated learning global model obtained in the last iteration, , represents a preset threshold, if yes, the federated learning end condition is reached, otherwise not. S9: Let the number of iterations , return to step S4.

2. The industrial IoT federated learning method of claim 1, wherein, The edge server in the step S5 The specific method of performing local gradient histogram aggregation is as follows: The gradient local aggregation value is calculated using the following equation : ， The second order gradient local aggregation value is calculated using the following equation : ， The following formula is used to calculate the staircase gradient expected local aggregation value : 。 3. The industrial IoT federated learning method of claim 1, wherein, The edge server in the step S5 The specific formula for performing sample count aggregation is: 。 4. The industrial IoT federated learning method of claim 1, wherein, The initial weights of the industrial equipment in the step S5 are calculated using the following formula: are calculated using the following formula: 。 5. The industrial IoT federated learning method of claim 1, wherein, The calculation formula of the global gradient histogram aggregation in the step S6 is as follows: ， ， wherein, indicates an edge server corresponding weight.

6. The industrial IoT federated learning method of claim 1, wherein, The step S6 edge server of the initial weight The calculation formula is as follows: 。 7. The industrial IoT federated learning method of claim 1, wherein, The step S6 further comprises pruning processing of the federated incremental decision tree based on a leaf node redundancy coefficient, and the specific method is as follows: Each leaf node is calculated using the following formula. Redundancy coefficient : ， wherein, , denotes a set of leaf nodes of the current federal incremental decision tree, denotes a leaf node derived by the cloud server according to the degree global aggregation value and the second-order gradient local aggregation value corresponding data set, denotes a sample in the data set , , denotes the gradient and the second-order gradient of the sample ;​ When leaf node Redundancy coefficient Less than the preset redundancy coefficient threshold When, the leaf node Its sibling leaf node The gradients and second-order gradients of the two nodes are merged, and the sum of their gradients and second-order gradients is used as the gradients and second-order gradients of the new leaf node, thus maintaining the consistency of gradients before and after merging.

8. The industrial IoT federated learning method of claim 1, wherein, The step S6 further comprises updating a feature bucket boundary based on global gradient information obtained in the current round of training, and the specific method is as follows: First, calculate each feature. The Gradient density of each bucket : ， The bucket boundaries are then updated by optimizing the following objective function Update: ， wherein, with are the minimum and maximum values of the current range of values, respectively. the minimum and maximum values of the current range of values, respectively.

9. The industrial IoT federated learning method of claim 1, wherein, the redundancy coefficient threshold With dynamic adjustment, the dynamic adjustment formula is as follows: ， wherein, denotes the learning rate.

Images