Federal incremental modeling method based on dynamic probability optimization in heterogeneous edge environment

Abstract

The invention provides a federal incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment. The method comprises the following steps that equipment in each factory counts class density vectors after noise adding and uploads the class density vectors to a server; the server selects a plurality of candidate sets, calculates the distribution offset coefficient of each candidate device set, and selects the minimum set to participate in federal training; equipment training is selected, and optimal parameters are uploaded; the server aggregates and distributes global model parameters; the device calculates a newly-added hidden layer output according to the global parameters and uploads the newly-added hidden layer output to the server; the server performs multi-index evaluation according to the performance of the equipment, optimizes the selected probability of the equipment through an Exp3 algorithm based on multi-index evaluation and fairness constraint, and selects the equipment to participate in training based on the probability; and when the number of hidden layer nodes exceeds the limit or the residual error satisfies the expected tolerance, finishing the training, and distributing to the equipment as a soft measurement model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of soft measurement of industrial process product quality indicators, and particularly relates to a federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment. BACKGROUND

[0002] Federated learning is a privacy-preserving distributed machine learning framework that enables multi-party collaborative modeling without transmitting raw data. This technology has been widely used in data-sensitive scenarios such as medical, financial, and industrial in recent years, and is particularly suitable for joint training tasks of enterprise devices that cannot directly share data.

[0003] In complex industrial process control, accurate prediction of product quality indicators is crucial to improve production efficiency and stability. However, due to the serious data silos between enterprise devices, and the significant differences in computing power, storage capacity, network conditions, etc. of industrial devices, single-party data is often insufficient to support high-quality model training. Federated learning provides an effective way for industrial devices to share models while protecting privacy.

[0004] Due to the large number of terminal devices, strong data heterogeneity, and limited communication resources, federated learning usually adopts a client selection mechanism. Existing methods prefer to select clients with strong computing power, ignoring devices with limited resources but representative data distribution, resulting in training data bias and decreased model generalization performance. In addition, existing client selection strategies rely heavily on static information, making it difficult to adapt to dynamic changes such as frequent online and offline of clients, significant performance differences, and training interruptions in industrial Internet of Things environments. Currently, federated learning is mostly combined with deep neural networks, but its training process is prone to problems such as gradient vanishing, slow convergence, and uneven resource consumption, limiting its application effect in industrial scenarios. In recent years, random configuration networks with simpler structure, more efficient training, and stronger adaptability to industrial devices have gradually become an alternative solution, showing strong generalization ability and practical value. SUMMARY

[0005] The application aims to address the problem of limited number of industrial process data samples and strong distribution heterogeneity, as well as unstable online state fluctuations of devices. The application proposes a federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment.

[0006] Technical solution, in order to solve the above technical problems, the application proposes a federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment, which includes the following steps:

[0007] S1, the equipment in each factory corresponds the industrial process sample data and the corresponding sample quality category label it has, obtains a category vector according to category counting, adds noise to the vector, uploads the category density vector after adding noise to the server end, the server determines the required number of equipment participating, initializes the exponential weight and the selected probability distribution of each equipment;

[0008] S2, the server selects a plurality of candidate equipment sets according to the selected probability, predicts a device subset that can successfully upload in combination with the drop-out rate of each equipment, calculates the category density distribution of each subset, quantifies the uneven degree of the distribution offset coefficient, and selects the equipment set with the smallest distribution offset coefficient for federated training;

[0009] S3, select the hidden layer nodes of the equipment, and put the hidden layer nodes meeting the local data constraint into the candidate pool, evaluate the data quality of the equipment, upload the best candidate node and data quality index to the server end;

[0010] S4, the server aggregates the uploaded hidden layer nodes to obtain global parameters, and obtains the fluctuation of the equipment, and distributes the global parameters to all equipment as the hidden layer parameters of the local incremental network;

[0011] S5, each equipment calculates the new hidden layer output after receiving the global parameters, and calculates the loss of the equipment in the current round with the global parameters, uploads the output weight and loss value to the server end, and the server aggregates the output weight of the selected equipment, and evaluates the local model of the equipment;

[0012] S6, the contribution of the equipment is obtained by weighting a plurality of score indicators of the equipment, and the selected probability distribution of each equipment is optimized according to the contribution according to the Exp3 algorithm based on multi-index evaluation and fairness constraint, so as to ensure that the probability meets the requirements;

[0013] S7, repeat steps S2-S6 until the current node number exceeds the limit or the residual error in the current iteration meets the expected tolerance, no new node is added, and a trained global model is obtained and distributed to each equipment as a soft measurement model.

[0014] Further, in step S1, the equipment in each factory corresponds the industrial process sample data and the corresponding sample quality category label it has, obtains a category vector according to category counting, and adds noise to the vector, and uploads the category density vector after adding noise to the server end. The specific method is as follows:

[0015] S11, in the heterogeneous edge environment of data heterogeneity and state fluctuation drop-out, that is, the amount of data each equipment has and the number of each category are different, set K common equipment to participate in federated training, equipment i statistics its historical industrial process sample data X i and the corresponding product quality category label Ti , denoted as {X i , T i}, where, represents the feature matrix of the vth industrial process sample data of device i;

[0016] S12, the number of the jth type of data owned by device i is denoted as , and for the J types of data, the vector represents the local data distribution of device i;

[0017] S13, each device follows the principle of differential privacy, and adds noise n i to the data distribution vector h i , where each noise component obeys Laplace distribution, i.e. is a privacy operator, let h′ i = [h′ i,1 , h′ i,2 , …, h′ i,J ], h′ i,j is the number of the jth type of data of device i after adding noise, and the formula for adding noise to the vector h i is as follows:

[0018] h′ i = h i + n i (1)

[0019] where h′ i is the category vector after adding noise;

[0020] S14, calculate the category density vector p i of each device according to h′ i , upload the vector p i to the server, and the calculation formula of p i is as follows:

[0021]

[0022] where the superscript T is the transpose of the matrix, and 1 T is a full column vector of 1.

[0023] Further, in step S1, the server determines the required number of devices to participate, and the specific method of initializing the exponential weight of each device and the selected probability distribution is as follows:

[0024] S15, the device selection ratio is d, then the number of selected devices k = K * d;

[0025] S16, ws L,i denotes the exponential weight of device i in training the Lth node, and the exponential weight ws is initialized 1,i = 1, and each device has the same probability of being selected at the beginning

[0026] Further, in step S2, the server selects a plurality of candidate device sets according to the selected probability, predicts a device subset that can successfully upload in combination with the drop-out rate of each device, calculates the category density distribution of each subset and quantifies the unevenness of the distribution offset coefficient, and selects the device set with the smallest distribution offset coefficient for federated training. The specific method is as follows:

[0027] S21, the server selects n client sets with k devices as candidate sets based on the selected probability denotes the device set selected in training the Lth node in the uth set in the candidate set. For these candidate sets, based on the drop-out rate of each device, the device set that can successfully upload is predicted denotes the device set that can successfully upload in step S20, and the category density vector p i of the device set is obtained wherein,

[0028] S22, the coefficient of variation in statistics is applied to federated learning, which is called distribution offset coefficient, denoted as DSC, and the formula is as follows:

[0029]

[0030] wherein, is the average value of the total category distribution vector, and u = 1, 2, …, n;

[0031] S23, the distribution offset coefficient of the total category distribution vector of all predicted sets is calculated, and the set with the smallest distribution offset coefficient is selected as the device set for training the Lth node participate in federated training:

[0032]

[0033] Further, in step S3, the selected device trains the hidden layer node, and the hidden layer node that meets the local data constraint is put into the candidate pool, and the data quality of the device is evaluated. The specific method of uploading the best candidate node and data quality index to the server side is as follows:

[0034] S31, the selected device i obtains the data quality score D_s according to the data it has i ​, formula as follows:

[0035]

[0036] Wherein, N i is the number of data categories owned by device i, represents the sum of the total number of J data categories of device i, max(h i ) and min(h i ) represent the maximum and minimum values in the orientation vector h i , H HEL (Q i , Q u ) is the Hellinger distance, which is a distance measure formula for measuring the similarity of two probability distributions Q i and Q u :

[0037]

[0038] Wherein, represents the class probability distribution of device i data, Q u is the probability distribution when the data is uniformly distributed;

[0039] S32, the selected device i initializes the parameters required for local incremental random configuration network learning: the maximum number of hidden layer nodes L max , the maximum number of random configurations γ max , the expected tolerance ε, the random configuration range of hidden layer parameters θ min is the minimum value of the configuration range, θ max is the maximum value of the configuration range, Δθ is the step size, and r is the learning parameter;

[0040] During the construction of the local incremental network of each device, the device hidden layer parameters w L,i and b L,i are randomly generated in the adjustable symmetric interval , and the hidden layer output H L,i is calculated:

[0041]

[0042] Wherein, g(·) is an activation function, and 1 is a full 1 vector of the same dimension as X i , and a sequence of non-negative real numbers is set The hidden layer nodes that satisfy the following inequality constraint are candidate nodes:

[0043]

[0044]

[0045] where <·,·> denotes the inner product of vectors, represents the supervision value corresponding to the qth column label of the data set when training the Lth hidden layer node in device i, is the qth column of the data residual error matrix when training the L-1th node of the network model of device i, and the initial residual error is the sample category label;

[0046] S33, device i performs γ max times random configuration during network training to find the pair of hidden layer parameters that maximizes , that is, the best hidden layer parameters of device i that satisfy the supervision mechanism and Upload the best hidden layer parameters and data quality scores to the server side.

[0047] Further, in step S4, the server aggregates the uploaded hidden layer nodes to obtain global parameters and the fluctuation of the device, and the specific method of distributing the global parameters to all devices as the hidden layer parameters of the local incremental network is as follows:

[0048] S41, the server recombines the successfully uploaded devices into The server aggregates the best hidden layer parameters uploaded by the devices to obtain the global parameters of the Lth node and

[0049]

[0050] S42, according to the device offline situation, update the device offline rate P L,i , and then calculate the volatility score V_s i , P L,i and V_s i The calculation formula is as follows:

[0051]

[0052] where F L,i represents the number of times device i is selected in training the L nodes of the global model, is the number of times device i successfully uploads in federated training of L nodes, and when training the jth node, device i is indexed in the set 1, otherwise 0, is the number of times device i is selected in federated training of L nodes, and when training the jth node, device i is indexed in the set 1, otherwise 0;

[0053] S43, the server will add the global parameters of the Lth node and and the global parameters of the previous L-1 nodes to obtain the global parameters and The parameters are issued to all devices as the parameters of the local incremental network.

[0054] Further, in step S5, after each device receives the global parameters, the newly added hidden layer output is calculated, and the loss of the device in this round is calculated with the global parameters. The specific method of uploading the output weight and the loss value to the server side is as follows:

[0055] S51, after the device receives the global parameters, the local hidden layer output is calculated according to the current parameters

[0056]

[0057] Calculate the local hidden layer output matrix of the device Where + represents the pseudo-inverse of the matrix;

[0058] S52, according to the current network, the cross-entropy loss value of the device i is calculated Where L(·) is the cross-entropy loss function, is the sample quality class label predicted by the model, and the calculation formula is as follows:

[0059]

[0060] S53, each device uploads the output weight β L,i and the loss value Loss L,i of this round to the server side.

[0061] Further, in step S5, the server weights and aggregates the output weight of the selected device, and evaluates the local model of the device. The specific method is as follows:

[0062] S54, the server weights and averages the uploaded output weight to update the global β L :

[0063]

[0064] S55, evaluate the local model of the device by the loss value of the device i to obtain the model score M_s i :

[0065]

[0066] Where the global loss value

[0067] Furthermore, in step S6, the weighted average of multiple score indicators of the equipment to obtain the equipment contribution, and based on this contribution, the specific method for optimizing the selection probability distribution of each equipment according to the Exp3 algorithm based on multi-indicator evaluation and fairness constraints is as follows:

[0068] S61, the server will assign D_s to each device i V_s i and M_s i Normalize and weight them respectively to obtain their contribution C to the current node. L,i :

[0069] C L,i =a·D_s i +b·V_s i +c·M_s i (17)

[0070] Where a, b, and c are the weights of each score, and their values ​​can be set according to the task requirements, and a + b + c = 1;

[0071] S62, based on the Exp3 algorithm which is based on multi-index evaluation and fairness constraints, and based on the true contribution C of device i. L,i The unbiased contribution estimate is obtained from the following formula.

[0072]

[0073] in, p′ L,i This represents the probability of device i being selected during the training of the Lth node. When L = 1,

[0074] S63, for cases where the device probability exceeds 1, the weights are not updated, and the update formula is as follows:

[0075]

[0076] Where η is the learning rate. It is the set of devices whose probability of occurrence exceeds 1 when training the Lth node;

[0077] S64, based on weight ws L,i The initial selection probability distribution is obtained. Wherein, the initial probability of device i being selected. The calculation formula is as follows:

[0078]

[0079] Furthermore, the specific method for ensuring that the probability meets the requirements in step S6 is as follows:

[0080] S65, for Ensure And to ensure that each device fair participation in training, set a selected probability lower limit sigma, that is, the required constraints: Where, i=1, 2, …, K;

[0081] S66, if Satisfy the constraints, the final selected probability Where, p ′ L =[p ′ L,1 ,p ′ L,2 ,…,p ′ L,K If not, set a proportionality coefficient That is To get the closest weight ratio p ′ L , Get the following objective function:

[0082]

[0083] S67, the above objective function is a quadratic programming problem, use Lagrange multiplier method to solve:

[0084]

[0085] Where, lambda L , mu L,i , v L,i Lagrange multiplier, construct the Lagrange function and combine KKT optimality condition to solve p' L .

[0086] Beneficial effects, compared with the prior art, the technical scheme of the present application has the following beneficial technical effects:

[0087] The technical scheme of the present application is based on the Exp3 algorithm of multi-index evaluation and fairness constraint to dynamically optimize the selected probability, so that each device participates in the training reasonably, and the diversity of the total data is ensured through the entropy of the selected device set. At the same time, use the random configuration network as the industrial process product quality soft measurement model, avoid the problems of slow training speed, falling into local minimum point and gradient explosion of deep learning network, and has good generalization and high accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0088] Figure 1 It is a flow chart of a federal incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment. DETAILED DESCRIPTION

[0089] The application provides a federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment, and a specific process is as shown in the figure Figure 1 , and specific steps are as follows:

[0090] In step S1, a soft measurement model is trained for 50 devices from different factories, that is, 50 devices participate in federated training, device i obtains its historical industrial process auxiliary data X i and corresponding product quality category data T i , denoted as {X i ,T i}, the data amount collected by the 50 devices is different, and the different categories collected by each device are also different, wherein wherein, represents the feature matrix of the vth industrial process sample data of device i; the number of the jth data possessed by device i is represented by , and for 10 categories of data, the vector is used to represent the local data distribution of device i.

[0091] Each device follows the principle of differential privacy, and adds noise n i to the data distribution vector h i . wherein each noise component obeys Laplace distribution, that is is a privacy operator, let h′ i =[h′ i,1 ,h′ i,2 ,…,h′ i,10 ], h′ i,j is the number of the jth data of device i after adding noise, and the noise formula for adding noise to the vector h i is as follows:

[0092] h′ i =h i +n i (1)

[0093] wherein h′ i is the category vector after adding noise;

[0094] According to h′ i , the category density vector p i of each device is calculated, the vector p i is uploaded to the server, and the calculation formula of p i is as follows:

[0095]

[0096] wherein the superscript T is the transpose of the matrix, and 1T is a full 1 column vector;

[0097] The device selection ratio is 0.4, and the number of selected devices is 20; ws L,i represents the exponential weight of device i in training the Lth node, and the exponential weight ws 1,i is initialized to 1, and each device has the same probability of being selected, which is 0.4.

[0098] In step S2, the server selects 10 client sets with 20 devices as candidate sets based on the selected probability represents the device set selected in training the Lth node in the uth set in the candidate set, and for these candidate sets, based on the device drop rate, the successfully uploaded device set on each set is predicted represents in the Lth node, the device set predicted to be successfully uploaded is determined according to the category density vector p i of the set category vector wherein,

[0099] The coefficient of variation in statistics is applied to federated learning, called distribution shift coefficient, denoted as DSC, and the formula is as follows:

[0100]

[0101] wherein, is the average value of the total category distribution vector, and u = 1, 2, …, 10;

[0102] The distribution shift coefficient of the total category distribution vector of all predicted sets is calculated, and the set with the smallest distribution shift coefficient is selected as the device set for training the Lth node Participate in federated training:

[0103]

[0104] In step S3, the selected device i obtains the data quality score D_s according to the data it owns i , and the formula is as follows:

[0105]

[0106] wherein, N i is the number of data categories owned by device i, represents the total sum of the total number of 10 categories of data owned by device i, max(h i ) and min(h i ) represent the maximum and minimum values in the vector h i , and H HEL(Q i ,Q u ) is the Hellinger distance, which measures the distance between two probability distributions Q i and Q u , and the formula is as follows:

[0107]

[0108] wherein, represents the category probability distribution of the data of the device i, Q u is the probability distribution when the data is uniformly distributed;

[0109] The selected device i initializes the parameters required for local incremental random configuration network learning: the maximum number of hidden layer nodes L max = 400, the maximum number of random configurations Y max = 15, the expected tolerance e = 0.06, and the random configuration range of the hidden layer parameter The learning parameter r = 0.97;

[0110] During the local incremental network construction process of each device, the device hidden layer parameters w L,i and b L,i are randomly generated in the adjustable symmetric interval , and the node hidden layer output H L,i is calculated:

[0111]

[0112] wherein g(·) is an activation function, 1 is a full 1 vector with the same dimension as X i , and the non-negative real sequence The hidden layer node satisfying the following inequality constraint is the candidate node:

[0113]

[0114] wherein <·,·> represents the inner product of vectors, represents the supervision value corresponding to the label of the qth column of the data set when the Lth hidden layer node of the device i is currently trained, is the qth column of the data residual error matrix when the L-1th node of the network model of the device i is trained, and the initial residual error is the sample category label;

[0115] The device i performs 15 random configurations during the network training process, and finds a pair of hidden layer parameters that maximizes , that is, the best hidden layer parameters of the device i that satisfy the supervision mechanism and The best hidden layer parameters and the data quality score are uploaded to the server side.

[0116] In step S4, the server recombines the devices that successfully upload The server weights and averages the best hidden layer parameters uploaded by the devices to obtain the global parameters of the Lth node And

[0117]

[0118] According to the device drop-off situation, update the device drop-off rate P L,i , and then calculate the volatility score V_s i , P L,i and V_s i The calculation formula is as follows:

[0119]

[0120] Where F L,i represents the number of times device i is selected in the L nodes of the training global model, is the number of times device i successfully uploads in the L nodes of federated training, that is, when training the jth node, if the index of device i is in the set , it is 1, otherwise it is 0; is the number of times device i is selected in the L nodes of federated training, that is, when training the jth node, if the index of device i is in the set , it is 1, otherwise it is 0;

[0121] The server splices the global parameters of the newly added Lth node and with the global parameters of the previous L-1 nodes to obtain the global parameters and The parameters are issued to all devices as the parameters of the local incremental network.

[0122] In step S5, after receiving the global parameters, the device calculates the local hidden layer output according to the current parameters

[0123]

[0124] Calculate the local hidden layer output matrix of the device Where + represents the pseudo-inverse of the matrix;

[0125] According to the current network, the cross-entropy loss value of device i is calculated Where L(·) is the cross-entropy loss function, is the sample quality class label predicted by the model, and the calculation formula is as follows:

[0126]

[0127] Each device will output the weight value β L,i and the loss value Loss L,i to the server side;

[0128] The server aggregates and updates the global β L by weighted average of the uploaded output weights;

[0129]

[0130] The model score M_s i is obtained by evaluating the device local model with the loss value of device i;

[0131]

[0132] where the global loss value

[0133] In step S6, the server normalizes and weights D_s i , V_s i and M_s i of each device to obtain its contribution C L,i to the current node:

[0134] C L,i = a·D_s i + b·V_s i + c·M_s i (17)

[0135] where a, b, c are the weights of each score, set

[0136] According to the Exp3 algorithm based on multi-index evaluation and fairness constraints, the unbiased estimated contribution L,i is obtained based on the real contribution C

[0137]

[0138] where p′ L,i represents the selected probability of device i when training the Lth node, when L = 1, p′ 1,i = 0.4;

[0139] The unbiased estimator is used to estimate the potential return of the client, and the exponential weight ws L,i is applied to reflect this estimate. For some devices whose probability exceeds 1, the weight is not updated, and its update formula is as follows:

[0140]

[0141] Where η = 0.2 is the learning rate. It is the set of devices whose probability of occurrence exceeds 1 when training the Lth node;

[0142] Based on weight Obtain the initial selection probability distribution Wherein the initial probability of device i being selected The calculation formula is as follows:

[0143]

[0144] for make sure Furthermore, to ensure fair participation of each device in training, a lower limit for the probability of selection is set, σ = 0.1, which requires the following constraints: Where i = 1, 2, ..., 50;

[0145] like If the constraints are satisfied, then the final probability of being selected is... Where, p ′ L =[p ′ L,1 ,p ′ L,2 ,…,p ′ L,50 If this condition is not met, assume there exists a proportionality coefficient. Right now To obtain p that is closest to the weight ratio ′ L The objective function is obtained as follows:

[0146]

[0147] The objective function described above is a quadratic programming problem, which can be solved using the Lagrange multiplier method:

[0148]

[0149] Where, λ L ,μ L,i ,ν L,i Assuming the Lagrange multipliers, construct the Lagrange function and solve for p′ using the KKT optimality conditions. L .

[0150] In step S7, steps S2-S6 are repeated until the model converges or the residual in the current iteration meets the expected tolerance. At this point, no new nodes are added, and the trained global model is obtained and distributed to each device as a soft measurement model.

[0151] It should be noted that, for those skilled in the art, without departing from the basic concept and principles of the present application, a number of modifications, substitutions or equivalent improvements can be made, such as adjusting the parameter settings of the federal incremental mechanism, introducing differential privacy and other data protection measures, etc. These technical means should be considered to fall within the protection scope of the present application. The technical details not explicitly described in the above embodiments can be implemented by using existing mature technologies, and the technical features described in the present application can be combined in any reasonable manner, as long as there is no technical conflict, which falls within the scope of the present application. Therefore, the protection scope of the present application should be limited by the contents defined in the appended claims, and should not be limited to the embodiments in the specification.

Claims

1. A federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment, characterized in that, The method comprises the following steps: S1, the equipment in each factory corresponds the industrial process sample data and the corresponding sample quality category label owned by it, obtains a category vector according to category counting, adds noise to the vector, uploads the noisy category density vector to the server end, the server determines the required number of equipment participating, initializes the exponential weight and the selected probability distribution of each equipment; S2, the server selects multiple candidate equipment sets according to the selected probability, predicts a device subset that can successfully upload in combination with the drop-out rate of each device, calculates the category density distribution of each subset, quantifies the imbalance degree of each subset by distribution offset coefficient, and selects the equipment set with the smallest distribution offset coefficient for federated training; S3, select the hidden layer nodes of the equipment, and put the hidden layer nodes meeting the local data constraints into the candidate pool, evaluate the data quality of the equipment, and upload the best candidate node and data quality index to the server end; S4, the server aggregates the uploaded hidden layer nodes to obtain global parameters, and obtains the fluctuation of the equipment, and distributes the global parameters to all equipment as the hidden layer parameters of the local incremental network; S5, each equipment calculates the new hidden layer output after receiving the global parameters, and calculates the loss of the equipment in the current round with the global parameters, uploads the output weight and loss value to the server end, and the server aggregates the output weight of the selected equipment, and evaluates the local model of the equipment; S6, the contribution of the equipment is obtained by weighting multiple score indicators of the equipment, and the selected probability distribution of each equipment is optimized according to the contribution based on the Exp3 algorithm based on multi-index evaluation and fairness constraint, so that the probability meets the requirements; S7, repeat steps S2-S6 until the current node number exceeds the limit or the residual error in the current iteration meets the expected tolerance, no new node is added, and a trained global model is obtained and distributed to each equipment as a soft measurement model.

2. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 1, characterized in that, In step S1, the equipment in each factory corresponds the industrial process sample data and the corresponding sample quality category label owned by it, obtains a category vector according to category counting, and adds noise to the vector, and uploads the noisy category density vector to the server end. The specific method is as follows: S11, in the heterogeneous edge environment of data heterogeneity and state fluctuation disconnection, that is, the amount of data owned by each device and the number of each category are different, set K common devices to participate in federated training, and device i counts its historical industrial process sample data X i and the corresponding product quality category label T i , denoted as {X i , T i}, wherein, represents the feature matrix of the vth industrial process sample data of device i; S12, the number of data of the jth type owned by device i is denoted by and the local data distribution of device i is denoted by the vector for the jth type of data. S13, each device follows the principle of differential privacy to the data distribution vector h i Add noise n i , where each noise component obeys Laplace distribution, i.e. is a privacy operator, let h′ i = [h′ i,1 , h′ i,2 , …, h′ i,J ], h′ i,j is the number of the jth type of data after adding noise to device i, and the vector h i Add noise formula as follows: h' i = h i + n i (1) where h' = h + n i is the class vector after adding noise; S14, according to h' i The category density vector p of each device is calculated i The vector p is uploaded to the server i The vector p is uploaded to the server i The calculation formula is as follows: where the superscript T denotes the transpose of a matrix, 1 T is an all-ones column vector.

3. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 2, characterized in that, In step S1, the server determines the required number of equipment participating, initializes the exponential weight and the selected probability distribution of each equipment. The specific method is as follows: S15, if the device selection ratio is d, the number of selected devices k = K * d; S16, ws L,i denotes the exponential weight of device i in training the Lth node, initialized as 1,i = 1, each device has the same probability of being selected at the beginning 4. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 3, characterized in that, In step S2, the server selects multiple candidate equipment sets according to the selected probability, predicts a device subset that can successfully upload in combination with the drop-out rate of each device, calculates the category density distribution of each subset, quantifies the imbalance degree of each subset by distribution offset coefficient, and selects the equipment set with the smallest distribution offset coefficient for federated training. The specific method is as follows: S21, the server selects, based on the selected probabilities, a set of n clients with k devices as a candidate set denotes the device set selected for training the Lth node of the u-th set in the candidate set, for these candidate sets, based on the drop-off rate of each device, the device set that will successfully upload on each set is predicted denotes the device set that will successfully upload in the i-th set is predicted according to the category density vector p i obtains the category vector of the set wherein, ​ S22, the coefficient of variation in statistics is applied to federated learning, which is called distribution offset coefficient, represented by DSC, and the formula is as follows: wherein, is the average of the total class distribution vector; S23, calculate the distribution offset coefficient of all prediction set total category distribution vectors, select the set that makes the distribution offset coefficient minimum as the device set for training the Lth node Participate in federated training:

5. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 4, characterized in that, In step S3, the hidden layer nodes of the equipment are selected, and the hidden layer nodes meeting the local data constraints are put into the candidate pool, and the data quality of the equipment is evaluated, and the best candidate node and data quality index are uploaded to the server end. The specific method is as follows: S31, the selected device i obtains a data quality score D_s from the data it has i , as follows: where N i is the number of data categories owned by device i, is the sum of the total number of J data categories owned by device i, max(h i ) and min(h i ) represent the maximum and minimum values in the orientation vector h i , H HEL (Q i , Q u ) is the Hellinger distance, a distance measure formula for measuring the similarity of two probability distributions Q i and Q u as follows: wherein, Q represents the class probability distribution of the data of device i u is the probability distribution when the data is uniformly distributed; S32, the selected device i initializes the parameters required for local incremental random configuration network learning: maximum hidden layer node number L max , maximum random configuration number γ max , expected tolerance ε, hidden layer parameter random configuration range θ min is the minimum value of the configuration range, θ max is the maximum value of the configuration range, Δθ is the step size, learning parameter r; In the local incremental network construction process of each device, the device hidden layer parameters w and b are randomly generated in the adjustable symmetric interval L,i L,i , and the node hidden layer output H is calculated L,i :​​ H L,i = g((w L,i ) T X i + b L,i 1) T (7) where g(·) is an activation function, 1 is the identity function, and X i a full vector of ones of the same dimension, a sequence of non-negative real numbers The hidden layer nodes that satisfy the following inequality constraint are candidate nodes: where <·,·> denotes the vector inner product, represents the supervision value corresponding to the label of the qth column of the data set when the Lth hidden layer node is currently trained in the device i, is the qth column of the data residual error matrix for the network model of device i when training the L-1th node, and the initial residual error is the sample category label; S33, the device i performs γ in the training network process max second random configuration, find out the best hidden layer parameters that make the maximum pair of hidden layer parameters, that is, the best hidden layer parameters that satisfy the supervision mechanism for device i and Upload the best hidden layer parameters and data quality scores to the server side.

6. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 5, characterized in that, In step S4, the server uploads the hidden layer node weighting aggregation to obtain the global parameter, and obtains the fluctuation of the device, and the specific method of issuing the global parameter to all devices as the hidden layer parameter of the local incremental network is as follows: S41, the server recombines the devices uploaded successfully into The server weights and averages the best hidden layer parameters uploaded by the devices to obtain the global parameters of the Lth node and S42, update the device offline rate P according to the device offline situation L,i , and the volatility score V_s is obtained i , P L,i and V_s i The calculation formula is as follows: where F L,i represents the number of times device i is selected in training the global model L nodes, is 1 when device i is indexed in the set when training the jth node, and 0 otherwise, is 1 when device i is indexed in the set when training the jth node, and 0 otherwise; S43, the server will add the global parameters of the Lth node and Splice with the global parameters of the previous L-1 nodes to obtain the global parameters and Distribute this parameter to all devices as the parameters of the local incremental network.

7. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 6, characterized in that, In step S5, each device receives the global parameter, calculates the new hidden layer output, and obtains the loss of the device in this round with the global parameter, and the specific method of uploading the output weight and loss value to the server end is as follows: S51, after the device receives the global parameters, the local hidden layer output is calculated according to the current parameters Computing device local hidden layer output matrix where + denotes the Moore-Penrose pseudo-inverse of a matrix; S52, obtain the cross-entropy loss value of the device i from the current network wherein L(·) is a cross-entropy loss function, is the sample quality category label predicted by the model, and the calculation formula is as follows: S53, each device will output weight value β L,i and the current round loss value Loss L,i upload to the server side.

8. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 7, characterized in that, In step S5, the server weights and aggregates the output weight of the selected device, and evaluates the local model of the device, and the specific method is as follows: S54, the server aggregates and updates the global β by weighted average of the uploaded output weight values L : S55, evaluate the model score M_s from the loss value of the device i's local model i : wherein the global loss value 9. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 8, characterized in that, In step S6, the device contribution is obtained by weighting the plurality of score indicators of the device, and according to the contribution, the selected probability distribution of each device is optimized according to the Exp3 algorithm based on the multi-index evaluation and fairness constraint, and the specific method is as follows: S61, the server normalizes and weights D_s i , V_s i and M_s i of each device respectively to obtain its contribution C L,i to the current node: C L,i = a · D_s i + b · V_s i + c · M_s i (17) Wherein, a, b, c is the weight of each score, which is set by itself according to the task requirement, and a+b+c=1; S62, based on the Exp3 algorithm based on multi-index evaluation and fairness constraints, based on the real contribution C of the device i L,i An unbiased estimate of the contribution is obtained from the following formula wherein, p′ L,i denotes the selected probability of device i when training the Lth node, when L = 1, S63, for the case that the probability of the device exceeds 1, the weight is not updated, and the updating formula is as follows: where η is the learning rate, is the set of devices whose probability of occurrence exceeds 1 when training the Lth node; S64, based on the weight ws L,i , to obtain an initial selected probability distribution where the initial selected probability of device i The calculation formula is as follows:

10. The federated incremental modeling method based on dynamic probability optimization in a heterogeneous edge environment according to claim 9, characterized in that, The specific method of ensuring that the probability meets the requirements in step S6 is: S65, for Ensuring And to ensure that each device participates in training fairly, set a selected probability lower limit σ, that is, the constraint: Where, i = 1, 2, …, K; S66, if is selected if the constraints are satisfied Let p ′ L = [p ′ L,1 , p ′ L,2 , …, p ′ L,K ], if not, let there be a proportionality coefficient i.e. to find the p ′ L that is closest to the weight ratio, the following objective function is obtained: S67, the above objective function is a quadratic programming problem, which is solved by Lagrange multiplier method: where λ L ,μ L,i , and v L,i are Lagrange multipliers. The Lagrangian function is constructed and combined with the KKT optimality conditions to solve for p′ L .

Images