Federal learning acceleration method and system for industry user water consumption prediction
By combining synchronous and asynchronous federated learning methods in water usage prediction for industry users, edge servers are clustered, which solves the problem of low training efficiency caused by differences in computing resources and data volume, and achieves efficient model training and resource utilization.
Patent Information
- Application Number
- CN202411452855.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-01-10
AI Technical Summary
Existing federated learning methods struggle to train efficiently in complex environments for water usage prediction by industry users, particularly due to low training efficiency and model inconsistency caused by differences in computing resources and data volume.
By combining synchronous and asynchronous training, multiple edge server clusters are formed by clustering edge servers. During global training, clustering is performed based on local training speed to achieve a combination of synchronous and asynchronous training. The edge server clusters are then used for model aggregation.
While ensuring model accuracy, it significantly reduces training time, makes full use of computing resources, improves training efficiency, reduces communication overhead, and adapts to edge servers with different computing capabilities.
Smart Images
Figure CN119294479A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine learning technology, and in particular to a federated learning acceleration method and system for water consumption prediction for industry users. Background Art
[0002] Water use prediction is conducive to coordinating water networks and ensuring the stability of urban water use. Neural network algorithms are suitable for predictions in various occasions, but different network models and training methods have a great impact on prediction accuracy.
[0003] To achieve water consumption forecasting for industry users using federated learning, more training data and larger neural networks with stronger performance are necessary to achieve higher prediction accuracy. However, the increase in the amount of training data and the expansion of the neural network also means that the training process will take more time. At the same time, the amount of data and computing resources owned by different users also vary greatly. If the users participating in the training cannot be coordinated during the training process, it will affect the efficiency of the entire training process and increase the time consumed by the training process.
[0004] Federated learning training methods can be divided into two types: synchronous training and asynchronous training. Both methods have their own advantages and disadvantages. Synchronous aggregation ensures the consistency and stability of the global model by collecting updates from all clients at fixed time intervals, but its efficiency may be limited by the slowest client and may cause communication bottlenecks in large-scale distributed environments. In contrast, asynchronous aggregation allows clients to update the global model as soon as they are ready, thereby improving the efficiency and flexibility of training, especially when there are large differences in client computing and network capabilities. However, this approach may introduce model inconsistencies, as clients may train based on different versions of the global model, and may affect the convergence and performance of the model when the update delay is large. Summary of the invention
[0005] In order to overcome the defect that existing federated learning methods are difficult to adapt to water consumption prediction for industrial users in complex environments, the present invention proposes a federated learning acceleration method for water consumption prediction for industrial users, which combines synchronous training and asynchronous training, clusters edge servers, and ensures the speed and performance of model training.
[0006] The present invention proposes a federated learning acceleration method for water consumption prediction for industrial users. During the global training process, multiple global training times are intervals, and edge servers are clustered according to their local training speeds to form multiple edge server clusters; the edge servers in the edge server clusters perform global aggregation to update the local model.
[0007] Prioritize, the edge server E in the first round of global iteration k The number of local training times zl,k for:
[0008]
[0009] Among them, z0 is the local training times of the fastest convergent edge server in the first global iteration process, z k For edge server E k The corresponding local training threshold, l old To set constants; Indicates rounding down.
[0010] The preferred steps include:
[0011] S1, the edge server receives the initialized global model broadcast by the cloud server and initializes the global iteration measurement value l to 1;
[0012] S2, each edge server performs local training, obtains the local training times of the fastest converged edge server as the initial iteration times z0, and broadcasts it to all edge servers;
[0013] S3, each edge server calculates the number of local training times z l,k ;
[0014] S4. Determine whether there is any z l,k If yes, then go to step S5; if no, then go to step S7;
[0015] S5. Obtain the number of clusters N and the corresponding edge server cluster;
[0016] S6: For each edge server cluster, instruct each edge server to perform ,k The local model is trained locally, and then uploaded to the cloud server for aggregation. The edge server loads the global model aggregated from the edge server cluster from the cloud server.
[0017] S7. Update l to l+1 and return to step S3, and repeat the training cycle until the local models of each edge server converge.
[0018] Preferably, the number of clusters N obtained in S5 is the maximum value that satisfies the clustering condition; the clustering condition is:
[0019]
[0020] in, Represents the edge server cluster S in the first round of global iteration i Time consumed in uploading local models; is the edge server cluster S in the first round of global iteration i Local training time, is the edge server cluster S in the first round of global iteration i Middle edge server E j Local training time; 1≤i≤N; max means taking the maximum value.
[0021] Priority:
[0022]
[0023] Among them, z l,j For edge server E j The number of local trainings in the lth global iteration; k D represents the computational weight brought by the neural network D, k D is the set value, data j Represents the edge server E j The amount of data on R j Represents the edge server E j The maximum computing resources on .
[0024] Priority:
[0025]
[0026] in, Represents the edge server cluster S in the first round of global iteration i The amount of parameter data of the model in the cluster transmitted, v all is the total bandwidth of the core network, which is the network composed of edge servers and cloud servers in the edge server cluster; i Assigned to S i ratio of the communication bandwidth.
[0027] Preferably, step S5 comprises the following sub-steps:
[0028] S5.1. Obtain the number of clusters N and the corresponding edge server clusters; when l=1, the initial value of N is 2, and the edge server is divided into 2 edge server clusters according to the local training time; when l>1, the existing edge server cluster is used;
[0029] S5.2, determine whether the number of clusters N meets the clustering condition; if yes, execute step S5.3; if no, execute step S5.4;
[0030] S5.3, update N to N+1, divide the edge server into N edge server clusters according to the local training time, and then return to step S5.2;
[0031] S5.4, update N to N-1, then divide the edge server into N edge server clusters according to the local training time, and execute step S6.
[0032] Preferably, clustering algorithms are used when clustering edge servers.
[0033] Preferably, it is applied in the field of industry and user water consumption prediction, and the edge server adopts the industry user water consumption monitoring device; the intelligent body set in the edge server is used to predict the subsequent water consumption data based on the historical water consumption data.
[0034] A federated learning acceleration system proposed in the present invention provides a carrier for the above-mentioned federated learning acceleration method for predicting water consumption by industry users. The system includes a cloud server and multiple edge servers. An intelligent agent is provided in the edge server, and the intelligent agent is trained using the above-mentioned federated learning acceleration method for predicting water consumption by industry users.
[0035] The advantages of the present invention are:
[0036] (1) The present invention proposes a federated learning acceleration method for water consumption prediction for industrial users. By clustering edge servers, clustering models within clusters based on edge server clusters, synchronous training and asynchronous training are combined. In addition, during the global training process, clustering is performed according to local training time intervals to achieve dynamic combination of edge server clusters, thereby ensuring the relevance of all edge servers during the global training process, ensuring model training performance, and greatly reducing training time while ensuring model accuracy; each cluster is independent of each other, and cluster aggregation greatly saves model training time compared to aggregation of all edge servers.
[0037] (2) In the present invention, setting a larger number of local iterations in the early stage of the training process can achieve a higher convergence speed. On the contrary, setting a smaller number of local iterations in the middle and late stages of the training process can reduce the time consumed by global iterations. The local training is accelerated by adjusting the number of local iterations.
[0038] (3) In the present invention, after clustering is completed, the difference in local training time of edge servers in the same cluster is very small. Edge servers in the same cluster can continue training, reducing the waiting time between edge servers and maximizing the use of computing resources in the network. In addition, the edge servers in the cluster upload model parameters to the cloud server only after completing the synchronous aggregation within the cluster, which greatly reduces the communication overhead during the training process.
[0039] (4) In the present invention, the number of local training times in each round of global iteration is adjusted to reduce the local training time as much as possible, so that the model can be trained faster.
[0040] (5) The present invention divides the devices into clusters and places the devices with similar training speeds into the same cluster, thereby reducing the impact of device heterogeneity and making full use of computing resources.
[0041] (6) Compared with the synchronous training method, the present invention asynchronously aggregates the model on the cloud server, does not need to wait for the slow training device, has higher training efficiency, and fully utilizes computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 A flowchart of a federated learning acceleration method for water consumption prediction for industry users;
[0043] Figure 2 It is the flow chart of clustering method;
[0044] Figure 3 It is the accuracy curve of multiple FedAvg algorithms when 15 clients participate in the training in the embodiment;
[0045] Figure 4 It is the accuracy curve of the method of the present invention when 60 clients participate in the training in the embodiment;
[0046] Figure 5 The loss curve of the method of the present invention when 60 clients participate in the training in the embodiment;
[0047] Figure 6 It is the accuracy-time mapping curve of different methods when 60 clients participate in training in the embodiment. DETAILED DESCRIPTION
[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0049] Reference Figure 1 , Figure 2 , this embodiment proposes a federated learning acceleration method for water consumption prediction of industrial users, including the following steps S1-S7.
[0050] S1. The edge server receives the initialized global model broadcasted by the cloud server and initializes the global iteration measurement value l to 1. The initialized global model is a model randomly initialized by the cloud server.
[0051] S2. The edge server performs local training, obtains the local training times of the fastest converging edge server as the initial iteration times z0, and broadcasts the initial iteration times z0 to all edge servers.
[0052] Specifically, when the edge server satisfies the condition that the loss change of the last two batches is less than the set floating threshold α, the edge server is judged to be converged; the formula is expressed as:
[0053] When S(loss p )<α, the edge server is judged to be converged;
[0054] S(loss p ) represents the degree of change of the loss of the edge server in the local training of the p-th batch (epoch), which is specifically expressed as the absolute value of the difference between the loss of the p-th batch (epoch) and the loss of the p-1-th batch (epoch).
[0055] S3, calculate the edge server E according to the local initial iteration number z0 k The number of local training times z in the lth global iteration l,k ;
[0056]
[0057] z k For edge server E k The corresponding local training threshold, l old To set constants; Indicates rounding down;
[0058] S4. Determine whether there is any z l,k If yes, then go to step S5; if no, then go to step S7;
[0059] That is, when l = 1 or l old The local iteration number of at least one edge server has not reached the corresponding local training threshold z k , then execute step S5 to cluster all edge servers.
[0060] S5. Execute steps S5.1-S5.4 to obtain the number of clusters N and the corresponding edge server clusters, and then execute step S6;
[0061] S5.1. Obtain the number of clusters N and the corresponding edge server clusters;
[0062] When l=1, the initial value of N is 2, and the edge servers are divided into two edge server clusters according to the local training time;
[0063] When l>1, the initial value of N is the number of existing edge server clusters, and the edge server cluster is the existing edge server cluster;
[0064] S5.2, determine whether the number of clusters N meets the following clustering conditions;
[0065]
[0066] in, Represents the edge server cluster S in the first round of global iteration i Time consumed in uploading local models; is the edge server cluster S in the first round of global iteration i Local training time, is the edge server cluster S in the first round of global iteration i Middle edge server E j Local training time; 1≤i≤N; max means taking the maximum value;
[0067] z l,j For edge server E j The number of local trainings in the lth global iteration; k D represents the computational weight brought by the neural network D, k D is a set value in the interval (0,1). In the following embodiments, let k D =0.44; data j Represents the edge server E j The amount of data on R j Represents the edge server E j The maximum computing resources on
[0068] Represents the edge server cluster S in the first round of global iteration i The amount of parameter data of the model in the cluster transmitted, v all is the total bandwidth of the core network, which is the network composed of edge servers and cloud servers in the edge server cluster; i Assigned to S i ratio of the communication bandwidth.
[0069] S5.3: If yes, update N to N+1, divide the edge server into N edge server clusters according to the local training time, and then return to step S5.2;
[0070] S5.4: No, then update N to N-1, divide the edge server into N edge server clusters according to the local training time, and then execute step S6.
[0071] Specifically, in this embodiment, the edge servers are divided into N edge server clusters according to the local training time, and the local training time of the edge servers in the same edge server cluster is similar; the edge server E k The local training time is the time it takes for the edge server to complete z l,k The time for local training.
[0072] In specific implementation, a clustering algorithm can be used to cluster edge servers;
[0073] S6. Perform the following steps for each edge server cluster:
[0074] Instruct each edge server to perform l,k The local model is trained locally, and then uploaded to the cloud server for aggregation. The edge server loads the global model aggregated from the edge server cluster from the cloud server.
[0075] S7. Update l to l+1 and return to step S3; repeat the training until the local model of each edge server converges.
[0076] In specific implementation, the local model convergence condition of the edge server can be set as follows: after the local training of the edge server is completed, the loss or accuracy of the local model converges. Or it can be set to when L global iterations are completed, then the local model of each edge server is judged to be converged; that is, in S8, if l before the update is equal to the set value L, then l is no longer updated in S8, but the training is terminated and the model on each edge server is fixed.
[0077] The above-mentioned federated learning acceleration method for water consumption prediction of industrial users is verified in combination with specific embodiments below.
[0078] In this embodiment, a total of 60 clients, namely edge servers, constitute the edge network, and all edge servers are connected to the core network through the same route. In this embodiment, the bandwidth is set to 20MB / s.
[0079] It should be noted that in the entire edge network, the data volume distribution between different edge servers is unbalanced.
[0080] This embodiment selects LSTM (Long Short-Term Memory Network) as the neural network for the experiment. The data set used in this embodiment is statistically obtained by integrating the water consumption data of users in a certain area based on the data provided by all local water supply companies.
[0081] In this embodiment, the method proposed in the present invention and the classic federated learning algorithm FedAvg are used to train the LSTM on each client. During the training process of the classic federated learning algorithm FedAvg, the number of local training times is fixed. The training parameters are set as follows: the learning rate lr is 0.0001, and the batch size batchsize is 256.
[0082] In this embodiment, 15 clients are first selected to participate in the federated training, and the accuracy index changes of the algorithm FedAvg and the method proposed in the present invention are compared. The accuracy index of the present invention selects the mean R2 score of the LSTM on each client.
[0083] In this embodiment, the local training times of the classic federated learning algorithm FedAvg are first fixed to 1, 3, 5, 7, and 9, and the prediction accuracy of the FedAvg algorithm under different local training times is statistically analyzed as the global iteration times change. Figure 3 When the model converges, the accuracy of all FedAvg algorithms is close, reaching about 0.7; and the more local training times, the easier it is for the accuracy index to converge.
[0084] In this embodiment, when the method proposed in the present invention is used to train the LSTM on 60 clients, the prediction accuracy changes with the change of the global iteration number and the loss, such as Figure 4 , Figure 5 As shown. It can be seen that when 60 clients participate in the training, when the global iteration number l reaches more than 500 times, the model accuracy index can reach more than 0.75, and the model accuracy index and loss tend to be stable; when the global iteration number reaches more than 1000 times, the model accuracy index is stable at about 0.85, which proves that the performance of the model trained by the method of the present invention is stable.
[0085] In this embodiment, when all 60 clients participate in the training, the FedAvg algorithm with fixed local training times of 5, 10, 15 and 20 times is used as a comparison, and the time required for each comparison algorithm and the algorithm of the present invention to achieve different accuracy indicators is counted. The results are shown in Tables 1 and Figure 6 shown.
[0086] Obviously, no matter what the target accuracy is, the method proposed in the present invention always takes the least time to reach the target accuracy. In addition, it is worth noting that the time required for the FedAvg algorithm to reach the target accuracy is different for different local training times, which shows that the number of local iterations affects the training speed.
[0087] Table 1 Training time required to achieve different R2 scores (seconds)
[0088]
[0089] Of course, it is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any reference numerals in the claims should not be regarded as limiting the claims involved.
[0090] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
[0091] The techniques, shapes, and structural parts not described in detail in the present invention are all well-known techniques.
Claims
1. A federated learning acceleration method for water consumption prediction of industrial users, characterized in that: During the global training process, multiple global training times are repeated and edge servers are clustered according to their local training speeds to form multiple edge server clusters. The edge servers in the edge server clusters perform global aggregation to update local models.
2. The federated learning acceleration method for water consumption prediction for industrial users according to claim 1, characterized in that: In the first round of global iteration, edge server E k The number of local training times z l，k for: Among them, z0 is the local training times of the fastest convergent edge server in the first global iteration process, z k For edge server E k The corresponding local training threshold, l old To set constants; Indicates rounding down.
3. The federated learning acceleration method for water consumption prediction for industrial users according to claim 2, characterized in that: The following steps are involved: S1, the edge server receives the initialized global model broadcast by the cloud server and initializes the global iteration measurement value l to 1; S2, each edge server performs local training, obtains the local training times of the fastest converged edge server as the initial iteration times z0, and broadcasts it to all edge servers; S3, each edge server calculates the number of local training times z l，k ; S4. Determine whether there is any z l，k Changes occur; if yes, execute step S5; If no, proceed to step S7; S5. Obtain the number of clusters N and the corresponding edge server cluster; S6: For each edge server cluster, instruct each edge server to perform l，k The local model is trained locally, and then uploaded to the cloud server for aggregation. The edge server loads the global model aggregated from the edge server cluster from the cloud server. S7. Update l to l+1 and return to step S3, and repeat the training cycle until the local models of each edge server converge.
4. The federated learning acceleration method for water consumption prediction for industrial users according to claim 3, characterized in that: The number of clusters N obtained in S5 is the maximum value that satisfies the clustering condition; the clustering condition is: in, Represents the edge server cluster S in the first round of global iteration i Time consumed in uploading local models; is the edge server cluster S in the first round of global iteration i Local training time, is the edge server cluster S in the first round of global iteration i Middle edge server E j Local training time; 1≤i≤N; max means taking the maximum value.
5. The federated learning acceleration method for water consumption prediction for industrial users according to claim 4, characterized in that: Among them, z l，j For edge server E j The number of local trainings in the lth global iteration; k D represents the computational weight brought by the neural network D, k D is the set value, data j Represents the edge server E j The amount of data on R j Represents the edge server E j The maximum computing resources on the .
6. The federated learning acceleration method for water consumption prediction for industrial users according to claim 4, characterized in that: in, Represents the edge server cluster S in the first round of global iteration i The amount of parameter data of the model in the cluster transmitted, v all is the total bandwidth of the core network, which is the network composed of edge servers and cloud servers in the edge server cluster; i Assigned to S i ratio of the communication bandwidth.
7. The federated learning acceleration method for water consumption prediction for industrial users according to claim 4, characterized in that: Step S5 The following steps are included: S5.
1. Obtain the number of clusters N and the corresponding edge server clusters; when l=1, the initial value of N is 2, and the edge server is divided into 2 edge server clusters according to the local training time; when l>1, the existing edge server cluster is used; S5.2, determine whether the number of clusters N meets the clustering condition; if yes, execute step S5.3; If no, proceed to step S5.4; S5.3, update N to N+1, divide the edge server into N edge server clusters according to the local training time, and then return to step S5.2; S5.4, update N to N-1, then divide the edge server into N edge server clusters according to the local training time, and execute step S6.
8. The federated learning acceleration method for water consumption prediction for industrial users according to claim 7, characterized in that: Clustering algorithm is used when clustering edge servers.
9. The federated learning acceleration method for water consumption prediction for industrial users according to claim 1, characterized in that: Applied to the field of industry and user water consumption prediction, the edge server adopts industry user water consumption monitoring device; the intelligent body set in the edge server is used to predict subsequent water consumption data based on historical water consumption data.
10. A federated learning acceleration system, characterized in that: It includes a cloud server and multiple edge servers; an intelligent agent is arranged in the edge server, and the intelligent agent is trained by the federated learning acceleration method for water consumption prediction for industrial users as described in any one of claims 1 to 9.