A hierarchical federated learning training method and system based on compressed sensing communication

Abstract

The application discloses a layered federated learning training method and system based on compressed sensing communication, and the method comprises the following steps: a cloud server broadcasts a global model to an edge server and a client participating in training; the client trains using a local private data set, compresses a local model update using 1-bit compressed sensing technology, and uploads the local model update to the edge server; the edge server performs edge aggregation and reconstruction using BIHT technology, compresses the reconstructed edge aggregation model using 1-bit compressed sensing technology, and uploads the reconstructed edge aggregation model to the cloud server; the cloud server performs cloud aggregation and reconstruction using the BIHT reconstruction technology, and broadcasts the updated global model to the edge server and the client for the next round of training. The application effectively solves the problem of a large amount of communication overhead generated in the layered federated learning process through 1-bit compressed sensing technology, greatly reduces the noise introduced in model quantization through the BIHT reconstruction method, and improves the overall training efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed machine learning, and more particularly, to a hierarchical federated learning training method and system based on compressed sensing communication. BACKGROUND

[0002] With the rapid development of Internet of Things and mobile device technology, the Internet of Everything technology has penetrated into all aspects of industrial manufacturing, health care, unmanned driving, and smart life. Various mobile devices, led by mobile phones, have very strong computing power under the development of cloud computing, and can process a large amount of data and complete precise calculations. In recent years, the cloud-edge-end hierarchical federated learning framework based on edge learning and federated learning framework has attracted widespread attention in the field of distributed learning. This technology adds an edge server as a communication relay, reducing the large communication overhead generated by direct communication between a large number of clients and a central server, and ensuring that there are enough clients participating in training. Studies have shown that the hierarchical federated learning framework has higher training efficiency compared to traditional federated learning.

[0003] One of the main challenges of this technology is the large communication overhead generated during model transmission. Because the development of large models has become a development trend today, in the hierarchical federated learning process, if the complete model is transmitted from the client to the edge server and from the edge server to the cloud server, a large amount of communication overhead will be generated, and this process may also cause the training efficiency to decrease due to insufficient communication capacity.

[0004] In view of the above, it is necessary to effectively compress the transmitted model to reduce the communication overhead in the entire process. At present, there are many methods for transmitting model compression in traditional federated learning and hierarchical federated learning, such as Sign-SGD, FL-STC, Local-QSGD, etc. Sign-SGD proposes a method of using majority voting in traditional federated learning to send the symbol of local update to the server for fusion, which reduces the model size by 32 times compared to traditional federated learning. After Sign-SGD, a quantization method is proposed to compress the transmitted model to symbol information.

[0005] Reducing communication overhead in the model transmission process through the quantization method will also affect the efficiency of the cloud-edge-end hierarchical federated learning. This is because some important model parameters are often lost in the compression process, so how to balance the communication overhead and learning efficiency is a very key problem. SUMMARY

[0006] The purpose of the present application is to significantly reduce the communication overhead generated in training while ensuring the training efficiency of hierarchical federated learning.

[0007] To achieve the above object, the application provides a hierarchical federated learning training method and system based on compressed sensing communication, and the technical scheme of the application is as follows:

[0008] A hierarchical federated learning training method based on compressed sensing communication comprises the following steps:

[0009] The global model of the cloud server is broadcast to the edge server and the client participating in training;

[0010] The global model is used as a local model, and the local model is trained for several rounds of training using the local private data set of the client to obtain a model update value;

[0011] The 1-bit compressed sensing technology is used to compress the model update value to obtain 1-bit symbol information, and the 1-bit symbol information is uploaded to the edge server to which each client belongs;

[0012] The 1-bit symbol information in each edge server is edge aggregated to obtain an edge model update measurement value and is reconstructed by a binary iterative hard threshold (BIHT) reconstruction method to obtain an edge model update, and based on the edge model update, a first local update model is obtained;

[0013] The first local update model is returned to the client as a local model for several rounds of training again, and the above steps are repeated several times, and after several rounds of edge aggregation, a second local update model is obtained;

[0014] The 1-bit compressed sensing technology is used to compress the second local update model in each edge server to obtain an edge server model measurement value and upload it to the cloud server;

[0015] The edge server model measurement value is cloud aggregated and reconstructed by the binary iterative hard threshold (BIHT) reconstruction method, and based on the reconstructed value, an updated cloud server global model is obtained, and a complete hierarchical federated learning training iteration is completed.

[0016] Preferably, the client participating in training uses the stochastic gradient descent method to train the local model.

[0017] Preferably, the specific steps of obtaining the model update value are that each client participating in the round of training performs E1 local training iterations on the local model according to the local private data set to obtain a local model update after training The The model update value is obtained by comparing the local model before updating with the local model after updating

[0018]

[0019] wherein, i represents a client, t represents a round of a current hierarchical federated learning training iteration, and E1 represents a round of a current edge aggregation.

[0020] Preferably, the compression of the model update value by using the 1-bit compressed sensing technology to obtain 1-bit signed information specifically includes: sparsifying the model update value by topk sparsification to obtain Then, the measurement matrix Φ t After the sparsified model update value is compressed, the local observation value of the client is obtained

[0021]

[0022] wherein, Φ t is an independent zero-mean Gaussian matrix.

[0023] Then, the local observation value is quantized to 1-bit signed information by using a sign function:

[0024]

[0025] wherein, sign(·) represents a sign function.

[0026] Preferably, the method of edge aggregation includes: taking the sample size of the local private data set of each participating training client under the edge server as the weight, multiplying the 1-bit signed information uploaded by the corresponding client, and then accumulating, and then using a sign function to obtain the measurement value of the edge model update:

[0027]

[0028] wherein, l represents an edge server, S t represents a participating training client under each edge server, and D i represents the sample size of the local private data set of the client.

[0029] Preferably, the measurement value of the edge model update is obtained and reconstructed by using a binary iterative hard thresholding (BIHT) reconstruction method to obtain an edge model update, and based on the edge model update, a first local update model is obtained, specifically: the measurement value of the 1-bit edge model update is reconstructed by using a BIHT reconstruction technology to obtain an edge model update

[0030]

[0031] Based on an edge server local update model is obtained​

[0032]

[0033] wherein, denotes the local update model obtained after the edge aggregation of the last round, γ t denotes the update factor of the edge server.

[0034] Preferably, the second local update model in each edge server is compressed using a 1-bit compressive sensing technology to obtain edge server model measurements, specifically comprising:

[0035] The second local update model after several times of edge aggregation is compared with the global model initially received by the edge server, i.e., the initial edge model, to obtain edge model update values:

[0036]

[0037] The edge model update values are topk-sparse to obtain sparse model update values The sparse model update values are further compressed using a 1-bit compressive sensing technology to obtain edge server model measurements

[0038]

[0039] Preferably, the cloud aggregation method specifically comprises: accumulating and summing the edge server model measurements, and then performing 1-bit quantization through a sign function:

[0040]

[0041] wherein, denotes the edge server model measurements after cloud aggregation, and s denotes a cloud server.

[0042] Preferably, the edge server model measurements after cloud aggregation are reconstructed using a binary iterative hard thresholding (BIHT) reconstruction method, and an updated global model of the cloud server is obtained based on the reconstructed values, specifically comprising:

[0043] The edge server model measurements after cloud aggregation are reconstructed to obtain a global model update:

[0044]

[0045] Based on the global model update an updated global model of the cloud server is obtained:​​

[0046]

[0047] wherein, w t-1 represents the global model of the cloud server issued to the edge server and the client in the initial step of the current t round layered federated learning training iteration, η t is the update factor of the cloud server.

[0048] To achieve the purpose of the application, another aspect of the application also provides a layered federated learning training system based on compressed sensing communication, which applies the above technical solution to a layered federated learning training method based on compressed sensing communication. The system comprises:

[0049] A cloud server unit is configured to broadcast the global model of the cloud server to the edge server and the client participating in the training, and to obtain the updated cloud server global model by cloud aggregation and binary iterative hard threshold (BIHT) reconstruction.

[0050] A client unit is configured to train the local model using the local private data set of the client for several rounds to obtain the model update value, to compress the model update value using 1-bit compressed sensing technology to obtain 1-bit sign information, and to upload the 1-bit sign information to the edge server to which each client belongs.

[0051] An edge server unit is configured to receive the 1-bit sign information of all participating training clients under its jurisdiction, to perform edge aggregation on the 1-bit sign information in each edge server to obtain edge model update measurement values, to reconstruct the edge model update measurement values using a binary iterative hard threshold (BIHT) reconstruction method to obtain edge model updates, to obtain a first local update model based on the edge model updates, to issue the first local update model to the client, and to compress a second local update model obtained after several rounds of edge aggregation using 1-bit compressed sensing technology to obtain edge server model measurement values, and to upload the edge server model measurement values to the cloud server.

[0052] Compared with the prior art, the application has the following advantages:

[0053] The application uses 1-bit compressed sensing technology to reduce the amount of data in the model update value with ultra-low encoding complexity, and only retains the sign information in the observation value of the model update value, thereby achieving extreme compression of the transmitted model, and using a binary iterative hard threshold (BIHT) reconstruction method at the receiving end to reconstruct the model update value, thereby reducing the loss of important information in the quantization process to a certain extent. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 A step schematic diagram of a hierarchical federated learning training method based on compressed sensing communication according to Embodiment 1 of the present application;

[0055] Figure 2 A structure schematic diagram of a hierarchical federated learning training system based on compressed sensing communication according to Embodiment 2 of the present application;

[0056] Figure 3 A comparison diagram of accuracy of the method of the present application and other algorithms under the condition of E1=60, E2=1 according to Embodiment 3 of the present application;

[0057] Figure 4 A comparison diagram of accuracy of the method of the present application and other algorithms under the condition of E1=20, E2=3 according to Embodiment 3 of the present application;

[0058] Figure 5 A comparison diagram of accuracy of the method of the present application and other algorithms under the condition of E1=15, E2=4 according to Embodiment 3 of the present application;

[0059] Figure 6 A comparison diagram of accuracy of the method of the present application and other algorithms under the condition of E1=10, E2=6 according to Embodiment 3 of the present application. DETAILED DESCRIPTION

[0060] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0061] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below.

[0062] Embodiment 1

[0063] Please refer to Figure 1 Embodiment 1 of the present application provides a hierarchical federated learning training method based on compressed sensing communication, including the following steps:

[0064] S1: broadcast the global model of the cloud server to the edge server and the client participating in training.

[0065] S2: use the global model as the local model, train the local model using the local private data set of the client for several rounds to obtain the model update value.

[0066] S3: The 1-bit compressed sensing technology is used to compress the model update value to obtain 1-bit symbol information, and the 1-bit symbol information is uploaded to the edge server to which each client belongs.

[0067] S4: The 1-bit symbol information in each edge server is edge-aggregated to obtain the measurement value of the edge model update and is reconstructed by using a binary iterative hard thresholding (BIHT) reconstruction method to obtain the edge model update, and based on the edge model update, a first local update model is obtained.

[0068] S5: The first local update model is returned to the client as a local model for further training for several rounds, and the above steps are repeated several times, and after several rounds of edge aggregation, a second local update model is obtained.

[0069] S6: The 1-bit compressed sensing technology is used to compress the second local update model in each edge server to obtain the edge server model measurement value and upload it to the cloud server.

[0070] S7: After the edge server model measurement value is cloud-aggregated, it is reconstructed by using a binary iterative hard thresholding (BIHT) reconstruction method, and based on the reconstructed value, an updated cloud server global model is obtained, and a complete hierarchical federated learning training iteration is completed.

[0071] Compressed sensing (CS) is a signal processing technology proposed by Candes et al. in 2006, which can reconstruct a signal at a sampling frequency much lower than the Nyquist sampling law. Compressed sensing technology compresses data while collecting, thereby saving resources required for storage and transmission. In the CS framework, for a K-sparse signal of N dimensions A high-dimensional signal is projected onto an M-dimensional space by a measurement matrix, M << N, i.e. y = Φx, where is the measurement matrix. The entire process compresses a high-dimensional signal (N-dimensional) into a low-dimensional signal (M-dimensional), and y is transmitted as an observation signal, which can accurately recover the sparse signal x from y when it reaches the observation end.

[0072] Unlike classical CS theory, 1-bit compressed sensing technology (1-bit CS) aims to obtain more extreme signal compression effects from 1-bit quantization of the measurement value y, only retaining the sign information of the measurement value. However, only the sign information of the measurement signal is obtained after signal processing, which causes irreversible information loss for signal reconstruction, and therefore a binary iterative hard thresholding (BIHT) method is used to reconstruct the 1-bit CS signal, which has the advantages of high reconstruction performance, good consistency, and low complexity, and its implementation process is as follows:

[0073]

[0074] x l = η k (a l )

[0075] After n iterations:

[0076]

[0077] where α is a parameter to control the gradient descent step size, and the nonlinear operator η k (v) Keep the values of the K largest elements in the vector v unchanged, and set other elements to zero.

[0078] The training method of the hierarchical federated learning based on compressed sensing communication in this embodiment 1 is divided into two stages:

[0079] The first stage: the cloud server broadcasts the global model to each edge server and each client participating in training (i.e., the edge server or the client downloads the global model from the cloud server). Each client participating in this round of training updates the local model based on the local private data set D i performs E1 local training iterations, and the local model after training is updated to i∈N t , N t represents the set of clients participating in this round of training. The model update value is obtained by comparing with the model before updating

[0080]

[0081] The model update value is sparsified by topk sparsification to obtain Then, the measured matrix Φ t is used to compress the sparsified model update value to obtain the local observation value of the client model

[0082]

[0083] where Φ t is an independent zero-mean Gaussian matrix. The information of the local observation value of the client model is quantized to 1-bit signed information using the sign function:

[0084]

[0085] Each client transmits the compressed 1-bit client model local observation value to the edge server to which it belongs, and the edge server performs edge aggregation processing to obtain the measurement value of the edge model update, wherein the method of edge aggregation performed by the edge server is: taking the data sample size of each client under the edge server as the weight, multiplying the 1-bit model observation value uploaded by the corresponding client, and then accumulating to obtain the measurement value of the edge model update using the sign(·) function:

[0086]

[0087] Wherein, l represents the edge server, S t represents the client participating in training under each edge server, D i represents the sample size of the local private data set of the client.

[0088] The edge server uses the binary iterative hard thresholding (BIHT) reconstruction technology to reconstruct the 1-bit measurement value of the edge model update to obtain the edge model update The reconstruction process is represented as:

[0089]

[0090] Based on The first local update model of the edge server can be obtained

[0091]

[0092] Wherein, represents the local update model obtained after the last round of edge aggregation, γ t represents the update factor of the edge server, and a complete edge aggregation iteration is completed.

[0093] The edge server returns the local update model to the client (i.e., the client downloads the edge model) as the local model for E1 times of local training iteration, and the above steps are repeated for E2 rounds to obtain the second local update model after E2 rounds of edge aggregation.

[0094] The second phase: after completing E2 rounds of edge aggregation, each edge server obtains the second local update model after E2 rounds of edge aggregation update The second local update model after E2 rounds of edge aggregation is compared with the global model initially received by the edge server, i.e., the initial edge model, to obtain the edge model update value:

[0095]

[0096] After the edge model update value topk is sparse, the sparse model update value is obtained After the edge model update value topk is sparse, the sparse model update value is obtained After the edge model update value topk is sparse, the sparse model update value is obtained Subsequently, each edge server transmits the 1-bit edge server model measurement value to the cloud server for cloud aggregation. to the cloud server for cloud aggregation, and the cloud server receives and accumulates the sum, and then quantizes the measurement value through the sign function, thereby realizing the measurement value average and 1-bit quantization of each edge server:

[0097]

[0098] wherein, represents the edge server model measurement value after cloud aggregation, and s represents the cloud server.

[0099] The cloud aggregated edge server model measurement value is reconstructed using the BIHT reconstruction method to obtain the global model update:

[0100]

[0101] Therefore, the updated cloud server global model can be obtained:

[0102]

[0103] wherein, w t-1 represents the global model issued by the cloud server to the edge server and the client in the initial step of the current t-round hierarchical federated learning training iteration, η t is the update factor of the cloud server.

[0104] The completion of the second phase represents that a complete 1-bit CS-HFL iteration based on compression sensing is completed, and a total of t rounds are required to complete.

[0105] Embodiment 2

[0106] Please refer to Figure 2 , this embodiment 2 is based on embodiment 1, and a hierarchical federated learning training method based on compression sensing communication is applied in embodiment 1, and a hierarchical federated learning training system based on compression sensing communication is provided, which comprises:

[0107] The cloud server unit is used for broadcasting the global model of the cloud server to the edge server and the client participating in the training, and performing cloud aggregation and binary iterative hard thresholding (BIHT) reconstruction to obtain the updated global model of the cloud server.

[0108] a client unit, configured to train the local model using the local private dataset of the client for several rounds to obtain a model update value, compress the model update value using a 1-bit compressive sensing technology to obtain 1-bit signed information, and upload the 1-bit signed information to an edge server to which the client belongs;

[0109] an edge server unit, configured to receive the 1-bit signed information of all participating clients under the edge server, perform edge aggregation on the 1-bit signed information in each edge server to obtain an edge model update measurement value, reconstruct the edge model update measurement value using a binary iterative hard thresholding (BIHT) reconstruction method to obtain an edge model update, obtain a first local update model based on the edge model update, distribute the first local update model to the clients, compress a second local update model obtained after several rounds of edge aggregation using the 1-bit compressive sensing technology to obtain an edge server model measurement value, and upload the edge server model measurement value to a cloud server.

[0110] The other steps of this embodiment 2 are the same as those of embodiment 1, which are not described here again.

[0111] Embodiment 3

[0112] This embodiment 3 is based on embodiments 1 and 2, and applies the layered federated learning training method and system based on compressive sensing communication of the above-mentioned embodiments 1 and 2. Real dataset MNIST is used for multiple simulation experiments to demonstrate the practicability and feasibility of the present application. As a comparison, benchmark algorithm HFL and improved algorithm QHFL are introduced as comparison algorithms.

[0113] Experimental settings:

[0114] The simulation experiments of 1-bit CS-HFL performed in this embodiment 3 are all completed on the Pytorch platform. In the experimental settings, the framework has one cloud server, five edge servers, and 50 clients. The clients are evenly distributed under the edge servers. Real dataset MNIST is used for layered federated learning experiment simulation. The participating clients use stochastic gradient descent when participating in training. The local training learning rate of the training task under the MNIST dataset is 0.01, and the batch size is set to 20. The update factor γ of the edge server and the update factor η of the cloud server are 0.2 and 0.1, respectively. The participation rate C of the clients under each edge server participating in one round of layered federated learning is 0.3. The sparsity of the model in 1-bit compressive sensing is 0.005, and the measurement matrix Φ is t t t ​​The compression rate is 0.1. Under this setting, the advantages of the algorithm proposed in this paper compared with the existing hierarchical federated learning algorithm are tested to prove the feasibility and performance of the algorithm proposed in this paper.

[0115] Experimental results comparison:

[0116] Please refer to Figure 3 , Figure 4 , Figure 5 , Figure 6 , respectively, are the accuracy comparison chart of the method of the application and other algorithms under the condition of E1=60, E2=1, E1=20, E2=3, E1=15, E2=4, E1=10, E2=6. It can be seen from the experimental result comparison chart that the model accuracy obtained by the application when the hierarchical federated learning reaches t=300 iterations is better than the existing hierarchical federated learning quantization algorithm QHFL, and approaches the accuracy convergence value of the benchmark algorithm HFL without communication optimization, i.e. compression processing. In QHFL, the quantization method reduces the parameter quantity to 1 / 8 of the original when the client transmits the model to the edge server and the edge server transmits the model to the cloud server. The algorithm proposed in this paper only needs to transmit the positive and negative signs of the model, i.e. only needs to transmit 1 bit of data, and the observation value obtained by the compression sensing algorithm is 1 / 10 of the size of the original model, which greatly reduces the model transmission quantity in the process of client to edge server and edge server to cloud server in hierarchical federated learning. In the case of extreme compression of the model, the difference between the model accuracy convergence value and the existing benchmark algorithm HFL after communication compression quantization processing can be ignored.

[0117] The other steps of this embodiment 3 are the same as those of embodiment 1, which are not described here.

[0118] In summary of the embodiments, the application designs an efficient cloud-edge-end hierarchical federated learning communication framework, which effectively solves the problem of large communication overhead in the hierarchical federated learning communication process by using 1-bit compression sensing technology (1-bit CS), and in the case of extreme compression of the model parameters, the noise introduced in the model quantization process is greatly reduced by the BIHT reconstruction method, and the overall training efficiency is improved.

[0119] Through the comparison test in embodiment 3, the obvious advantages of the present application compared with the prior art in the case of extreme compression of the model are compared. The convergence performance in the training process can be guaranteed, and the difference between the benchmark algorithm HFL before communication compression optimization can be basically ignored.

[0120] Obviously, the above embodiments of the present application are merely exemplary but not intended to limit the embodiments of the present application. Based on the above description, any other variations or changes can be made by those skilled in the art without departing from the spirit and principles of the present application. It is not necessary to list all the embodiments here. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A hierarchical federated learning training method based on compressed sensing communication, characterized in that, The method comprises the following steps: broadcasting a global model of a cloud server to edge servers and clients participating in training; using the global model as a local model, training the local model using a local private data set of the client for several rounds to obtain a model update value; using a 1-bit compressed sensing technology to compress the model update value to obtain 1-bit signed information, and uploading the 1-bit signed information to an edge server to which each client belongs; performing edge aggregation on the 1-bit signed information in each edge server to obtain an edge model update measurement value and reconstructing the edge model update measurement value using a binary iterative hard thresholding (BIHT) reconstruction method to obtain an edge model update, and obtaining a first local update model based on the edge model update; returning the first local update model to the client as a local model for training again for several rounds, repeating the above steps for several times, and obtaining a second local update model after several rounds of edge aggregation; using a 1-bit compressed sensing technology to compress the second local update model in each edge server to obtain an edge server model measurement value and uploading the edge server model measurement value to the cloud server; performing cloud aggregation on the edge server model measurement value, reconstructing the edge server model measurement value using a binary iterative hard thresholding (BIHT) reconstruction method, and obtaining an updated global model of the cloud server based on the reconstructed value, to complete one complete hierarchical federated learning training iteration; The 1-bit compression sensing technology is used to compress the model update value to obtain 1-bit symbol information, specifically including: sparsifying the model update value by topk sparsification to obtain a sparse model update value , and then using a measurement matrix , compressing the sparse model update value to obtain a local observation value of the client : wherein is an independent zero-mean Gaussian matrix; quantizing the local observation value to 1-bit signed information using a sign function: wherein sign(·) represents a sign function; the edge aggregation method comprises: using the sample size of the local private data set of each client participating in training under the edge server as a weight, multiplying the 1-bit signed information uploaded by the corresponding client, and then accumulating, and then using a sign function to obtain an edge model update measurement value: wherein, denotes an edge server, denotes a client participating in the training under each edge server, denotes a sample size of the local private dataset of the client.

2. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that, the client participating in training uses a stochastic gradient descent method to train the local model.

3. The hierarchical federated learning training method based on compressed sensing communication according to claim 2, characterized in that, The specific steps of obtaining the model update value are that each client participating in the round of edge aggregation performs local model training on the local private data set to obtain a local model update value after completing training , wherein the local model update value is compared with the local model before the update to obtain the model update value : wherein, denotes a client, denotes a round of a current hierarchical federated learning training iteration, E 1denotes a round of a local training iteration, denotes a round of a current edge aggregation.

4. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that, The measurement value of the edge model update is obtained and reconstructed by using a binary iterative hard thresholding (BIHT) reconstruction method to obtain the edge model update, and based on the edge model update, a first local update model is obtained, specifically: the measurement value of the 1-bit edge model update is reconstructed by using a BIHT reconstruction technology to obtain the edge model update ​ Based on , a local update model of the edge server is obtained : wherein, denotes the first local updated model obtained after the edge aggregation of the last round, denotes the update factor of the edge server.

5. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that, the use of a 1-bit compressed sensing technology to compress the second local update model in each edge server to obtain an edge server model measurement value specifically comprises: the second local update model after several times of edge aggregation the global model initially received by the edge server, i.e., an initial edge model the edge model update value obtained by comparison : The edge model update value is topk-sparse to obtain a sparse model update value The sparse model update value is further quantized to obtain a quantized sparse model update value The 1-bit compressive sensing technology is used to compress and quantize to obtain an edge server model measurement value : 。 6. The hierarchical federated learning training method based on compressed sensing communication according to claim 1, characterized in that, the cloud aggregation method specifically comprises: summing the edge server model measurement values, and then 1-bit quantizing the edge server model measurement values through a sign function sign(·): wherein, represents the edge server model measurement value after cloud aggregation, represents a cloud server.

7. The hierarchical federated learning training method based on compressed sensing communication according to claim 6, characterized in that, the use of a binary iterative hard thresholding (BIHT) reconstruction method to reconstruct the edge server model measurement value after cloud aggregation, and obtaining an updated global model of the cloud server based on the reconstructed value, specifically comprises: reconstructing the edge server model measurement value after cloud aggregation to obtain a global model update: Global model update based on , obtaining an updated cloud server global model: wherein, represents the global model issued by the cloud server to the edge server and the client in the initial step of the current round t of federated learning training iteration, is an update factor of the cloud server.

8. A hierarchical federated learning training system based on compressed sensing communication, applying a hierarchical federated learning training method based on compressed sensing communication according to any one of claims 1-7, characterized in that, the system comprises: a cloud server unit configured to broadcast a global model of a cloud server to edge servers and clients participating in training, and to perform cloud aggregation and binary iterative hard thresholding (BIHT) reconstruction to obtain an updated global model of the cloud server; A client unit is configured to train a local model using a local private dataset of the client for several rounds to obtain a model update value, compress the model update value using a 1-bit compressive sensing technique to obtain 1-bit signed information, and upload the 1-bit signed information to an edge server to which the client belongs. An edge server unit is configured to receive the 1-bit signed information of all participating clients under the edge server, aggregate the 1-bit signed information in each edge server to obtain an edge model update measurement value, reconstruct the edge model update measurement value using a binary iterative hard thresholding (BIHT) reconstruction method to obtain an edge model update, obtain a first local update model based on the edge model update, distribute the first local update model to the clients, compress a second local update model obtained after several rounds of edge aggregation using a 1-bit compressive sensing technique to obtain an edge server model measurement value, and upload the edge server model measurement value to a cloud server.

Images