Layered federal learning training method and system based on compressed sensing communication

Abstract

The invention discloses a hierarchical federal learning training method and system based on compressed sensing communication. The method comprises the steps that a cloud server broadcasts a global model to an edge server and clients participating in training; the client uses a local private data set to train, uses a 1-bit compressed sensing technology to compress a local model, updates and uploads the local model to the edge server; the edge server performs edge aggregation, reconstructs the edge aggregation model by using a BIHT technology, compresses the reconstructed edge aggregation model by using a 1-bit compressed sensing technology, and uploads the edge aggregation model to the cloud server; and the cloud server carries out cloud aggregation and reconstructs by using a BIHT reconstruction technology, and broadcasts the updated global model to the edge server and the client for next round of training. According to the method, the problem that a large amount of communication overhead is generated in the hierarchical federal learning process is effectively solved through the 1-bit compressed sensing technology, noise introduced in model quantization is greatly reduced through the BIHT reconstruction method, and the overall training efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed machine learning, and more specifically, to a hierarchical federated learning training method and system based on compressed sensing communication. Background Art

[0002] With the rapid development of the Internet of Things and mobile device technologies, the Internet of Everything technology has penetrated into industrial manufacturing, health care, unmanned driving, smart life and other aspects. Driven by the development of cloud computing, various mobile devices, led by mobile phones, have very powerful computing capabilities, capable of processing large amounts of data and completing precise calculations. In recent years, the cloud-edge hierarchical federated learning framework proposed based on edge learning and federated learning frameworks has received widespread attention in the field of distributed learning. This technology adds edge servers as communication relays, reducing the large amount of communication overhead generated by direct communication between a large number of clients and central servers, and also ensuring that there are enough clients participating in training. Studies have shown that compared with traditional federated learning, the hierarchical federated learning framework has higher training efficiency.

[0003] One of the main challenges of this technology is the large amount of communication overhead generated during model transmission. Because the development of large models has become a development trend, if the complete model is transmitted from the client to the edge server and from the edge server to the cloud server during the hierarchical federated learning process, a large amount of communication overhead will be generated. This process may also lead to a decrease in the overall training efficiency due to insufficient communication capabilities.

[0004] In view of the above situation, it is necessary to effectively compress the transmission model to reduce the communication overhead of the whole process. At present, there are many methods for compressing the transmission model of traditional federated learning and hierarchical federated learning, such as Sign-SGD, FL-STC, Local-QSGD, etc. Sign-SGD proposed a method to use the majority voting method in traditional federated learning to send locally updated symbols to the server for fusion, which reduces the model size by 32 times compared with traditional federated learning. After Sign-SGD, many methods for compressing the transmission model to quantization of symbolic information were proposed.

[0005] Reducing the communication overhead during model transmission through quantization methods will also affect the efficiency of cloud-edge hierarchical federated learning. This is because some important model parameters are often lost during the compression process, so how to balance communication overhead and learning efficiency is a very critical issue. Summary of the invention

[0006] The purpose of the present invention is to significantly reduce the communication overhead generated during training while ensuring the efficiency of hierarchical federated learning training.

[0007] To achieve the above objectives, the present invention provides a hierarchical federated learning training method and system based on compressed sensing communication. The technical solution of the present invention is as follows:

[0008] A hierarchical federated learning training method based on compressed sensing communication includes the following steps:

[0009] Broadcast the global model of the cloud server to the edge servers and clients participating in the training;

[0010] The global model is used as a local model, and the local model is trained for several rounds using a local private data set of the client to obtain a model update value;

[0011] Using 1-bit compressed sensing technology to compress the model update value to obtain 1-bit symbol information, and uploading the 1-bit symbol information to the edge server to which each client belongs;

[0012] Performing edge aggregation on the 1-bit symbol information in each edge server to obtain a measurement value of an edge model update and reconstructing the measurement value by using a binary iterative hard threshold (BIHT) reconstruction method to obtain an edge model update, and obtaining a first local update model based on the edge model update;

[0013] The first local updated model is returned to the client as a local model for further training for several rounds, and the above steps are repeated several times to obtain a second local updated model after several rounds of edge aggregation;

[0014] Compress the second local update model in each edge server using 1-bit compressed sensing technology to obtain edge server model measurement values ​​and upload them to the cloud server;

[0015] The edge server model measurement values ​​are aggregated in the cloud and reconstructed using a binary iterative hard threshold (BIHT) reconstruction method, and an updated cloud server global model is obtained based on the reconstructed values, completing a complete hierarchical federated learning training iteration.

[0016] Preferably, the client participating in the training uses a stochastic gradient descent method to train the local model.

[0017] Preferably, the specific step of obtaining the model update value is that each client participating in the round of training performs E1 local training iterations on the local model according to the local private data set to obtain the local model update value after the training is completed. Will Compare with the local model before updating to get the model update value

[0018]

[0019] Among them, i represents the client, t represents the round of the current hierarchical federated learning training iteration, and E1 represents the round of the current edge aggregation.

[0020] Preferably, the method of compressing the model update value by using 1-bit compressed sensing technology to obtain 1-bit symbol information specifically includes: performing sparse processing on the model update value by topk sparse processing to obtain Then use the measurement matrix Φ t Compress the sparse model update value to obtain the client's local observation value

[0021]

[0022] Among them, Φ t is an independent zero-mean Gaussian matrix;

[0023] The local observation value is then quantized into 1-bit symbol information using a symbol function:

[0024]

[0025] Wherein, sign(·) represents the sign function.

[0026] Preferably, the edge aggregation method includes: taking the sample size of the local private data set of each client participating in the training under the edge server as a weight, multiplying it with the 1-bit symbol information uploaded by the corresponding client and then accumulating it, and then using the symbol function to obtain the measurement value of the edge model update:

[0027]

[0028] Among them, l represents the edge server, S t represents the clients participating in the training under each edge server, D i Indicates the sample size of the client's local private dataset.

[0029] Preferably, the measured value of the edge model update is obtained and reconstructed using a binary iterative hard threshold BIHT reconstruction method to obtain an edge model update, and based on the edge model update, a first local update model is obtained, specifically: using the BIHT reconstruction technology to reconstruct the 1-bit edge model update measured value Reconstruct and get edge model update

[0030]

[0031] based on Get the local update model of the edge server

[0032]

[0033] in, represents the local update model obtained after the previous round of edge aggregation, γ t Indicates the update factor of the edge server.

[0034] Preferably, the compressing the second local update model in each edge server using 1-bit compressed sensing technology to obtain the edge server model measurement value specifically includes:

[0035] The second local update model after several edge aggregations The edge model update value is obtained by comparing it with the global model initially received by the edge server, i.e., the initial edge model:

[0036]

[0037] The edge model update value is topk sparsely distributed to obtain the sparse model update value Then update the value of the sparse model Use 1-bit compressed sensing technology to compress and quantize the edge server model measurement value

[0038]

[0039] Preferably, the cloud aggregation method is specifically: accumulating and summing the edge server model measurement values, and then performing 1-bit quantization through a sign function sign(·):

[0040]

[0041] in, represents the edge server model measurement value after cloud aggregation, and s represents the cloud server.

[0042] Preferably, the edge server model measurement values ​​are aggregated in the cloud and reconstructed using a binary iterative hard threshold BIHT reconstruction method, and an updated cloud server global model is obtained based on the reconstructed values, specifically:

[0043] The edge server model measurements aggregated in the cloud are reconstructed to obtain a global model update:

[0044]

[0045] Based on global model update Get the updated cloud server global model:

[0046]

[0047] Among them, w t-1 represents the global model sent from the cloud server to the edge server and client in the initial step of the current t-th round of hierarchical federated learning training iteration, η t is the update factor of the cloud server.

[0048] To achieve the purpose of the invention, the present invention further provides a hierarchical federated learning training system based on compressed sensing communication, and a hierarchical federated learning training method based on compressed sensing communication using the above technical solution, the system comprising:

[0049] The cloud server unit is used to broadcast the global model of the cloud server to the edge servers and the clients participating in the training, and to perform cloud-side aggregation and binary iterative hard threshold BIHT reconstruction to obtain the updated cloud server global model.

[0050] The client unit is used to perform several rounds of training on the local model using the local private data set of the client to obtain a model update value and compress the model update value using 1-bit compressed sensing technology to obtain 1-bit symbol information, and upload the 1-bit symbol information to the edge server to which each client belongs;

[0051] The edge server unit is used to receive the 1-bit symbol information of all subordinate clients participating in the training, perform edge aggregation on the 1-bit symbol information in each edge server to obtain the measurement value of the edge model update, reconstruct the measurement value of the edge model update using the binary iterative hard threshold BIHT reconstruction method to obtain the edge model update, obtain the first local update model based on the edge model update, send the first local update model to the client, and use 1-bit compressed sensing technology to compress the second local update model obtained after several rounds of edge aggregation to obtain the edge server model measurement value, and upload the edge server model measurement value to the cloud server.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The present invention adopts 1-bit compressed sensing technology to reduce the amount of data in the model update value with ultra-low coding complexity, and only retains the symbol information in the observation value of the model update value, thereby achieving extreme compression of the transmission model, and uses the binary iterative hard threshold BIHT reconstruction method at the receiving end to reconstruct the model update value, thereby reducing the loss of important information in the quantization process to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of the steps of a hierarchical federated learning training method based on compressed sensing communication according to Example 1 of the present application;

[0055] Figure 2 This is a structural diagram of a hierarchical federated learning training system based on compressed sensing communication according to Example 2 of the present application;

[0056] Figure 3 This is a comparison chart of the accuracy of the method of the present invention and other algorithms under the conditions of E1=60 and E2=1 in Example 3 of the present application;

[0057] Figure 4 This is a comparison chart of the accuracy of the method of the present invention and other algorithms under the conditions of E1=20 and E2=3 in Example 3 of the present application;

[0058] Figure 5 This is a comparison chart of the accuracy of the method of the present invention and other algorithms under the conditions of E1=15 and E2=4 in Example 3 of the present application;

[0059] Figure 6 This is a comparison chart of the accuracy of the method of the present invention and other algorithms under the conditions of E1=10 and E2=6 in Example 3 of the present application. DETAILED DESCRIPTION

[0060] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0061] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0062] Example 1

[0063] See also Figure 1 Embodiment 1 of the present application provides a hierarchical federated learning training method based on compressed sensing communication, comprising the following steps:

[0064] S1: Broadcast the global model of the cloud server to the edge servers and clients participating in the training.

[0065] S2: Use the global model as the local model and use the client's local private dataset to train the local model for several rounds to obtain the model update value.

[0066] S3: Use 1-bit compressed sensing technology to compress the model update value to obtain 1-bit symbol information, and upload the 1-bit symbol information to the edge server to which each client belongs.

[0067] S4: Perform edge aggregation on the 1-bit symbol information in each edge server to obtain the measurement value of the edge model update and reconstruct it using the binary iterative hard threshold BIHT reconstruction method to obtain the edge model update, and obtain the first local update model based on the edge model update.

[0068] S5: The first local updated model is returned to the client as a local model for several rounds of training again. The above steps are repeated several times, and the second local updated model is obtained after several rounds of edge aggregation.

[0069] S6: Use 1-bit compressed sensing technology to compress the second local update model in each edge server, obtain the edge server model measurement value and upload it to the cloud server.

[0070] S7: After the edge server model measurement values ​​are aggregated in the cloud, they are reconstructed using the binary iterative hard threshold BIHT reconstruction method, and the updated cloud server global model is obtained based on the reconstructed values, completing a complete hierarchical federated learning training iteration.

[0071] Compressed sensing (CS) is a signal processing technology proposed by Candes et al. in 2006. It can reconstruct signals at a sampling frequency far lower than that required by the Nyquist sampling theorem. Compressed sensing technology compresses data while collecting it, thus saving the resources required for storage and transmission. In the CS framework, for an N-dimensional K-sparse signal The high-dimensional signal is projected onto the M-dimensional space through the measurement matrix, M<<N, that is, y=Φx, where is the measurement matrix. The whole process compresses the high-dimensional signal (N dimension) into a low-dimensional signal (M dimension), and y is transmitted as the observation signal. When it reaches the observation end, the sparse signal x can be accurately restored from y.

[0072] Different from the classic CS theory, 1-bit compressed sensing technology (1-bit CS) is committed to quantizing the measured value y by 1-bit, retaining only the sign information of the measured value to obtain a more extreme signal compression effect. However, after signal processing, only the sign information of the measured signal is available, which will cause irreversible information loss in signal reconstruction. Therefore, it is necessary to use the binary iterative hard threshold (BIHT) method to reconstruct the 1-bit CS signal. This reconstruction method has the advantages of high reconstruction performance, good consistency, and low complexity. The implementation process is as follows:

[0073]

[0074] x l =η k (a l )

[0075] After n iterations:

[0076]

[0077] Among them, α is the parameter that controls the step size of gradient descent, and the nonlinear operator η k (v) Keep the values ​​of the K elements with the largest magnitude in vector v unchanged and set the other elements to zero.

[0078] In this embodiment 1, a hierarchical federated learning training method based on compressed sensing communication is divided into two stages:

[0079] Phase 1: The cloud server broadcasts the global model to each edge server and each client participating in the training (i.e., the edge server or client downloads the global model from the cloud server). Each client participating in this round of training uses a local private dataset D i Perform E1 local training iterations, and the local model after training is updated to i∈N t , N t Represents the set of clients participating in this round of training. By comparing with the model before updating, we can get the model update value.

[0080]

[0081] The model update value is obtained by sparse topk sparsification Then use the measurement matrix Φ t Compress the sparse model update value to obtain the local observation value of the client model

[0082]

[0083] where Φ t is an independent zero-mean Gaussian matrix. The information of the local observation value of the client model is quantized into 1-bit symbol information using the symbol function:

[0084]

[0085] Each client transmits the compressed 1-bit local observation value of the client model to its own edge server, which performs edge aggregation processing to obtain the measurement value of the edge model update. The edge server performs edge aggregation by taking the data sample size of each client under the edge server as the weight, multiplying it with the 1-bit model observation value uploaded by the corresponding client, and then using the sign function sign(·) to obtain the measurement value of the edge model update:

[0086]

[0087] Among them, l represents the edge server, S t represents the clients participating in the training under each edge server, D i Indicates the sample size of the client's local private dataset.

[0088] The edge server uses the binary iterative hard threshold (BIHT) reconstruction technique to update the measurement value of the 1-bit edge model Reconstruct and get edge model update The reconstruction process is expressed as:

[0089]

[0090] based on The first local update model of the edge server can be obtained

[0091]

[0092] in, represents the local update model obtained after the previous round of edge aggregation, γ t Represents the update factor of the edge server. A complete round of edge aggregation iteration is completed.

[0093] The edge server returns the local updated model to the client (i.e., the client downloads the edge model) as the local model for another E1 local training iteration, and repeats the above steps for E2 rounds to obtain the second local updated model after E2 rounds of edge aggregation.

[0094] Phase 2: After completing the E2 round of edge aggregation, each edge server obtains the second local update model after the E2 round of edge aggregation update. The second local update model after E2 round edge aggregation The edge model update value is obtained by comparing it with the global model initially received by the edge server, i.e., the initial edge model:

[0095]

[0096] The edge model update value topk is sparsely distributed to obtain the sparse model update value get Then use 1-bit compressed sensing technology to compress and quantize it to obtain the edge server model measurement value Then, each edge server will Transmit to the cloud server for cloud aggregation. The cloud aggregation method is as follows: each edge server uploads 1-bit edge server model measurement value to the cloud server after multiple rounds of edge aggregation. After receiving, the cloud server accumulates and sums, and then quantizes the measurement value through the sign function sign(·), thereby achieving the average measurement value of each edge server and 1-bit quantization:

[0097]

[0098] in, represents the edge server model measurement value after cloud aggregation, and s represents the cloud server.

[0099] Then use the BIHT reconstruction method to reconstruct the edge server model measurement values ​​aggregated from the cloud to obtain the global model update:

[0100]

[0101] Therefore, the updated cloud server global model can be obtained:

[0102]

[0103] Among them, w t-1 represents the global model sent from the cloud server to the edge server and client in the initial step of the current t-th round of hierarchical federated learning training iteration, η t is the update factor of the cloud server.

[0104] The completion of the second stage represents the completion of a complete round of compressed sensing-based hierarchical federated learning (1-bit CS-HFL) iterations, and a total of t rounds need to be completed.

[0105] Example 2

[0106] See also Figure 2 This embodiment 2 is based on embodiment 1, and applies a hierarchical federated learning training method based on compressed sensing communication of embodiment 1 to provide a hierarchical federated learning training system based on compressed sensing communication, the system comprising:

[0107] The cloud server unit is used to broadcast the global model of the cloud server to the edge servers and the clients participating in the training, and to perform cloud-side aggregation and binary iterative hard threshold BIHT reconstruction to obtain the updated cloud server global model.

[0108] The client unit is used to train the local model for several rounds using the client's local private data set to obtain a model update value, compress the model update value using 1-bit compressed sensing technology to obtain 1-bit symbol information, and upload the 1-bit symbol information to the edge server to which each client belongs;

[0109] The edge server unit is used to receive 1-bit symbol information of all subordinate clients participating in the training, perform edge aggregation on the 1-bit symbol information in each edge server to obtain the measurement value of the edge model update, reconstruct the measurement value of the edge model update using the binary iterative hard threshold BIHT reconstruction method to obtain the edge model update, obtain the first local update model based on the edge model update, send the first local update model to the client, and use 1-bit compressed sensing technology to compress the second local update model obtained after several rounds of edge aggregation to obtain the edge server model measurement value, and upload the edge server model measurement value to the cloud server.

[0110] The other steps of this embodiment 2 are the same as those of embodiment 1 and will not be repeated here.

[0111] Example 3

[0112] This embodiment 3 is based on embodiment 1 and embodiment 2, and applies a hierarchical federated learning training method and system based on compressed sensing communication of the above embodiments 1 and embodiment 2. Multiple simulation experiments are carried out using the real data set MNIST to demonstrate the practicality and feasibility of the present invention. For comparison, we introduce the benchmark algorithm HFL and the improved algorithm QHFL algorithm as comparison algorithms.

[0113] Experimental setup:

[0114] The simulation experiments of 1-bit CS-HFL conducted in this Example 3 were all conducted and completed on the Pytorch platform. In the experimental setting, the framework has a cloud server, 5 edge servers, and 50 clients. The clients are evenly distributed under each edge server. The real data set MNIST is used to simulate the hierarchical federated learning experiment. The clients participating in the training use stochastic gradient descent when participating in the training, where the local training learning rate of the training task under the MNIST data set is 0.01, and the batch size is set to 20. The update factor γ of the edge server t and the update factor η of the cloud server t The participation rate C of each client under each edge server in a round of hierarchical federated learning is 0.3. The sparsity of the model in 1-bit compressed sensing is 0.005, and the measurement matrix Φ tThe compression ratio is 0.1. In this setting, the advantages of the proposed algorithm compared with the existing hierarchical federated learning algorithm are tested to prove the feasibility and performance of the proposed algorithm.

[0115] Comparison of experimental results:

[0116] See also Figure 3 , Figure 4 , Figure 5 , Figure 6 , which are the accuracy comparison diagrams of the method of the present invention and other algorithms under the conditions of E1=60, E2=1, E1=20, E2=3, E1=15, E2=4, E1=10, and E2=6, respectively. The experimental results comparison diagram shows that the model accuracy obtained by the present invention when the hierarchical federated learning reaches t=300 iterations is better than the existing hierarchical federated learning quantization algorithm QHFL as a whole, and is close to the accuracy convergence value of the benchmark algorithm HFL algorithm that has not undergone communication optimization, that is, compression processing. In QHFL, the quantization method reduces the number of parameters to 1 / 8 of the original when the client transmits the model to the edge server and the edge server transmits the model to the cloud server, while the algorithm proposed in this paper only needs to transmit the positive and negative signs of the model during transmission, that is, only 1 bit of data needs to be transmitted, and the observation value obtained by the compressed sensing algorithm is 1 / 10 of the original model size, which greatly reduces the model transmission amount from the client to the edge server and from the edge server to the cloud server in the hierarchical federated learning process. In the case of extreme compression of the model, the gap between the model accuracy convergence value and the existing benchmark algorithm HFL that has undergone communication compression and quantization processing can be basically ignored.

[0117] The other steps of this embodiment 3 are the same as those of embodiment 1 and will not be repeated here.

[0118] In summary, the present invention designs an efficient cloud-edge hierarchical federated learning communication framework, which utilizes 1-bit compressed sensing technology (1-bit CS) to effectively solve the problem of a large amount of communication overhead generated in the hierarchical federated learning communication process, and under the condition of extreme compression transmission of model parameters, through the BIHT reconstruction method, greatly reduces the noise introduced in the model quantization process, thereby improving the overall training efficiency.

[0119] Through the comparative test in Example 3, the present invention compares the obvious advantages of different combinations of hierarchical federated learning aggregation times over the prior art. Under the condition of extreme compression of the model, it can ensure that the convergence performance in the training process is basically negligible compared with the benchmark algorithm HFL before communication compression optimization.

[0120] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A hierarchical federated learning training method based on compressed sensing communication, characterized in that: The following steps are involved: Broadcast the global model of the cloud server to the edge servers and clients participating in the training; The global model is used as a local model, and the local model is trained for several rounds using a local private data set of the client to obtain a model update value; Using 1-bit compressed sensing technology to compress the model update value to obtain 1-bit symbol information, and uploading the 1-bit symbol information to the edge server to which each client belongs; Performing edge aggregation on the 1-bit symbol information in each edge server to obtain a measurement value of an edge model update and reconstructing the measurement value by using a binary iterative hard threshold (BIHT) reconstruction method to obtain an edge model update, and obtaining a first local update model based on the edge model update; The first local updated model is returned to the client as a local model for further training for several rounds, and the above steps are repeated several times to obtain a second local updated model after several rounds of edge aggregation; Compress the second local update model in each edge server using 1-bit compressed sensing technology to obtain edge server model measurement values ​​and upload them to the cloud server; The edge server model measurement values ​​are aggregated in the cloud and reconstructed using a binary iterative hard threshold (BIHT) reconstruction method, and an updated cloud server global model is obtained based on the reconstructed values, completing a complete hierarchical federated learning training iteration.

2. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that: The client participating in the training trains the local model using a stochastic gradient descent method.

3. The hierarchical federated learning training method based on compressed sensing communication according to claim 2, characterized in that: The specific steps of obtaining the model update value are: each client participating in this round of training performs E1 local training iterations on the local model according to the local private data set, and obtains the local model update value after the training is completed. Will Compare with the local model before updating to get the model update value Among them, i represents the client, t represents the round of the current hierarchical federated learning training iteration, and E1 represents the round of the current edge aggregation.

4. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that: The method of compressing the model update value by using 1-bit compressed sensing technology to obtain 1-bit symbol information specifically includes: sparsely performing topk sparse processing on the model update value to obtain Then use the measurement matrix Φ t Compress the sparse model update value to obtain the client's local observation value Among them, Φ t is an independent zero-mean Gaussian matrix; The local observation value is then quantized into 1-bit symbol information using a symbol function: Wherein, sign(·) represents the sign function.

5. A hierarchical federated learning training method based on compressed sensing communication according to claim 4, characterized in that: The edge aggregation method includes: taking the sample size of the local private data set of each client participating in the training under the edge server as a weight, multiplying it with the 1-bit symbol information uploaded by the corresponding client and then accumulating it, and then using the symbol function to obtain the measurement value of the edge model update: Among them, l represents the edge server, S t represents the clients participating in the training under each edge server, D i Indicates the sample size of the client's local private dataset.

6. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that: The measured value of the edge model update is obtained and reconstructed using the binary iterative hard threshold BIHT reconstruction method to obtain the edge model update, and based on the edge model update, a first local update model is obtained, specifically: using the BIHT reconstruction technology to reconstruct the 1-bit edge model update measured value Reconstruct and get edge model update based on Get the local update model of the edge server in, represents the first local update model obtained after the previous round of edge aggregation, γ t Indicates the update factor of the edge server.

7. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that: The compressing the second local update model in each edge server by using 1-bit compressed sensing technology to obtain the edge server model measurement value specifically includes: The second local update model after several edge aggregations The edge model update value is obtained by comparing it with the global model initially received by the edge server, i.e., the initial edge model: The edge model update value is topk sparsely distributed to obtain the sparse model update value Then update the value of the sparse model Use 1-bit compressed sensing technology to compress and quantize the edge server model measurement value 8. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that: The cloud aggregation method is specifically as follows: the edge server model measurement values ​​are accumulated and summed, and then quantized to 1-bit by a sign function sign(·): in, represents the edge server model measurement value after cloud aggregation, and s represents the cloud server.

9. A hierarchical federated learning training method based on compressed sensing communication according to claim 8, characterized in that: The edge server model measurement values ​​are aggregated in the cloud and reconstructed using a binary iterative hard threshold BIHT reconstruction method, and an updated cloud server global model is obtained based on the reconstructed values, specifically: The edge server model measurements aggregated in the cloud are reconstructed to obtain a global model update: Based on global model update Get the updated cloud server global model: Among them, w t-1 represents the global model sent from the cloud server to the edge server and client in the initial step of the current t-th round of hierarchical federated learning training iteration, η t is the update factor of the cloud server.

10. A hierarchical federated learning training system based on compressed sensing communication, applying a hierarchical federated learning training method based on compressed sensing communication as described in any one of claims 1 to 9, characterized in that: The system comprises: The cloud server unit is used to broadcast the global model of the cloud server to the edge servers and the clients participating in the training, and to perform cloud-side aggregation and binary iterative hard threshold BIHT reconstruction to obtain the updated cloud server global model. The client unit is used to perform several rounds of training on the local model using the local private data set of the client to obtain a model update value and compress the model update value using 1-bit compressed sensing technology to obtain 1-bit symbol information, and upload the 1-bit symbol information to the edge server to which each client belongs; The edge server unit is used to receive the 1-bit symbol information of all subordinate clients participating in the training, perform edge aggregation on the 1-bit symbol information in each edge server to obtain the measurement value of the edge model update, reconstruct the measurement value of the edge model update using the binary iterative hard threshold BIHT reconstruction method to obtain the edge model update, obtain the first local update model based on the edge model update, send the first local update model to the client, and use 1-bit compressed sensing technology to compress the second local update model obtained after several rounds of edge aggregation to obtain the edge server model measurement value, and upload the edge server model measurement value to the cloud server.

Images