Asynchronous federated learning method and system based on saliency depth compressive sensing
By employing a frequency-domain driven saliency deep compressed sensing framework and a noise-gated aggregation mechanism, the problems of high communication latency and gradient staleness in federated learning are solved, achieving efficient and robust asynchronous federated learning, reducing communication volume and latency, and ensuring stable convergence of the model.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
- Filing Date
- 2026-02-28
- Publication Date
- 2026-06-09
AI Technical Summary
Existing federated learning methods suffer from high communication latency and low model transmission efficiency in bandwidth-constrained wireless network environments. Furthermore, the conflict between gradient staleness and reconstruction noise during asynchronous updates leads to severe model divergence.
By employing a frequency-domain driven saliency deep compressed sensing framework combined with a noise-gated aggregation mechanism, and through adaptive frequency adjustment and structural adaptive grouping design, efficient and robust asynchronous federated learning is achieved under extremely low communication bandwidth.
It significantly reduced communication volume and client upload latency, improved model training efficiency, ensured stable convergence in bandwidth-constrained environments, and achieved a balance between communication efficiency and model accuracy.
Smart Images
Figure CN122174872A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and communication technology, specifically relating to an asynchronous federated learning method and system based on saliency-based deep compressed sensing. Background Technology
[0002] The statements herein provide only background information in relation to this invention and do not necessarily constitute prior art.
[0003] With the proliferation of Internet of Things (IoT) devices, massive amounts of data are generated at the network edge. To leverage this data for intelligent model training while protecting user privacy, Federated Learning (FL) has emerged. Federated Learning allows clients to train models locally, uploading only model parameters or gradients to a central server for aggregation, thus avoiding the transmission of raw data. However, in real-world applications, Federated Learning faces significant challenges.
[0004] First, there's the communication bottleneck. As deep learning models become increasingly complex, with massive numbers of parameters (reaching millions or even billions), frequent model transmissions in bandwidth-constrained wireless network environments lead to extremely high communication latency, severely limiting training efficiency. While existing quantization and sparsification techniques alleviate this problem to some extent, they often struggle to maintain model performance at extremely high compression rates. Second, there are the heterogeneity and staleness issues. In synchronous federated learning (SFL), the server must wait for all clients to complete training before aggregation, causing system efficiency to be limited by the least computationally powerful "laggard." Asynchronous federated learning (AFL) allows clients to upload gradients as they arrive, solving the laggard problem, but introducing severe gradient staleness. This means that gradients uploaded by clients are calculated based on older versions of the global model; direct aggregation can lead to deviations in optimization direction and even model divergence.
[0005] Existing solutions such as Overlap-FedAvg, while improving throughput through overlapping computation and communication, do not fundamentally reduce the amount of data transmitted and exacerbate version discrepancies. On the other hand, applying Compressed Sensing (CS) or Discrete Cosine Transform (DCT) to federated learning can reduce communication volume, but research has found that deep learning-based CS introduces structural artifacts during reconstruction. In asynchronous update scenarios, this reconstruction noise clashes with staleness compensation mechanisms. Specifically, traditional Taylor expansion-based staleness compensation treats the noise introduced by DCS as part of the signal and amplifies it nonlinearly, leading to destructive interference and severely hindering model convergence. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide an asynchronous federated learning method and system based on saliency deep compressed sensing. Through a frequency domain-driven saliency deep compressed sensing framework combined with a noise-gated aggregation mechanism, efficient and robust asynchronous federated learning is achieved under extremely low communication bandwidth.
[0007] To achieve the above objectives, the present invention is implemented through the following technical solution: In a first aspect, the technical solution of the present invention provides an asynchronous federated learning method based on saliency-based deep compressed sensing, including: Initialize the global model and obtain the gradient of the local model; Based on discrete cosine transform and adaptive dynamic scaling mechanism, frequency domain adaptive sparsification is performed on the local model gradient to obtain frequency domain coefficients. A structure-adaptive grouped deep compressed sensing mechanism is used to compress frequency domain coefficients and combine them with precision sensing quantization to generate compressed observations. An approximate gradient is obtained by reconstructing the compressed observations through an inverse transform. Adaptive gating weights are calculated based on the estimated reconstruction noise level, and the error feedback cache and global model are updated using the gating weights.
[0008] In at least one embodiment, the frequency domain adaptive sparsity processing specifically includes: transforming the gradient to the frequency domain using a two-dimensional discrete cosine transform, and normalizing the frequency domain coefficients to the sensitive region of the deep compressed sensing model through adaptive dynamic scaling to obtain the frequency domain coefficients.
[0009] In at least one embodiment, the structure-adaptive grouped deep compressed sensing mechanism compresses frequency domain coefficients, specifically including: grouping network layers according to layer structure characteristics, using a saliency network to generate a saliency map reflecting the importance of information and calculating the saliency score of each gradient block; constructing a heterogeneous measurement matrix based on the saliency score and dynamically sampling the gradient blocks to generate compressed measurement vectors.
[0010] In at least one embodiment, the compressed measurement vector is specifically represented as follows:
[0011] In the formula, Indicates the first Layer The normalization coefficient of the block; Indicates the first Layer Heterogeneous measurement matrix of the block; Indicates the first Layer The compression measurement vector of the block.
[0012] In at least one embodiment, generating compressed observations by combining precision-aware quantization specifically includes: calculating the peak scalar of the compressed measurement vector, normalizing the compressed measurement vector using the peak scalar as a scaling factor, quantizing the normalized compressed measurement vector, truncating its data format from 32-bit floating-point to 16-bit half-precision floating-point, and obtaining the quantized compressed observations.
[0013] In at least one embodiment, the error feedback cache is updated using gating weights, specifically as follows:
[0014] In the formula, Indicates the gating weight; Indicates error feedback buffer; This indicates the updated error feedback cache; This represents the ideal gradient.
[0015] In at least one embodiment, a global model update is performed based on the updated error feedback cache, specifically as follows:
[0016] In the formula, Indicates the current number Global model parameters during round iteration; This represents the updated global model; This represents the learning rate.
[0017] Secondly, the technical solution of the present invention also provides an asynchronous federated learning system based on saliency deep compressed sensing, comprising: The initialization module is configured to: initialize the global model and obtain the gradient of the local model; The ADS module is configured to: perform frequency domain adaptive sparsification on the local model gradient based on discrete cosine transform and adaptive dynamic scaling mechanism to obtain frequency domain coefficients. The DCS module is configured to compress frequency domain coefficients based on a structure-adaptive grouped deep compressed sensing mechanism and generate compressed observations by combining precision sensing quantization. The inverse transform module is configured to reconstruct an approximate gradient by performing an inverse transform on the compressed observations. The update module is configured to: calculate adaptive gating weights based on the estimated reconstruction noise level, and use the gating weights to update the error feedback cache and the global model.
[0018] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the asynchronous federated learning method based on saliency-based deep compressed sensing as described in the first aspect.
[0019] Fourthly, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the asynchronous federated learning method based on saliency-based deep compressed sensing as described in the first aspect.
[0020] The beneficial effects of the above-described technical solution of the present invention are as follows: 1) This invention fully utilizes the frequency domain sparsity of gradients and the nonlinear mapping capability of deep compression sensing through a multi-compression strategy involving DCT energy compression, saliency dynamic sampling, and precision-aware quantization. Experiments show that on tasks such as CIFAR-10, compared with the traditional FedAvg method, the communication volume is reduced by more than 85%, reducing the transmission volume from GB to MB.
[0021] 2) This invention reveals the conflict between DCS reconstruction noise and traditional staleness compensation, and proposes a noise-gated aggregation mechanism. This mechanism effectively suppresses "destructive interference" by adaptively adjusting the gating weights, solves the noise amplification problem in asynchronous updates, and ensures that the model can still converge stably under the dual challenges of reconstruction error and gradient staleness.
[0022] 3) The method proposed in this invention significantly reduces client upload latency (including computation and transmission time), achieving a speedup of more than 5.5 times compared to benchmark methods in edge environments with limited bandwidth (e.g., 2Mbps). Furthermore, the adaptive grouping design allows this method to flexibly adapt to different types of model structures, such as convolutional neural networks (CNNs). Attached Figure Description
[0023] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0024] Figure 1 This is a schematic diagram of the asynchronous federated learning method based on saliency deep compressed sensing disclosed in Embodiment 1 of the present invention; Figure 2 This is an architecture diagram of the asynchronous federated learning method based on saliency deep compressed sensing disclosed in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the structure-adaptive grouped deep compressed sensing mechanism disclosed in Embodiment 1 of the present invention; Figure 4This is a comparison of the total communication volume of the asynchronous federated learning method based on saliency deep compressed sensing disclosed in Embodiment 1 of the present invention with the existing FedAvg and Overlap-FedAvg methods on the EMNIST, Fashion-MNIST, and CIFAR-10 datasets. Figure 5 This is a comparison chart of client single-round upload latency decomposition under bandwidth-constrained conditions using the asynchronous federated learning method based on saliency deep compressed sensing disclosed in Embodiment 1 of the present invention. Detailed Implementation
[0025] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0026] As described in the background section, the purpose of this invention is to overcome the shortcomings of the prior art and provide an asynchronous federated learning method and system based on saliency deep compressed sensing. Through a frequency domain-driven saliency deep compressed sensing framework combined with a noise-gated aggregation mechanism, efficient and robust asynchronous federated learning is achieved under extremely low communication bandwidth.
[0027] Example 1 In a typical embodiment of the present invention, such as Figures 1 to 5 As shown, this embodiment discloses an asynchronous federated learning method based on saliency deep compressed sensing, which specifically includes the following steps: S1. Initialize the global model and obtain the global model gradient; S2. Based on the discrete cosine transform and adaptive dynamic scaling mechanism, the global model gradient is subjected to frequency domain adaptive sparsification to obtain frequency domain coefficients; S3. A structure-adaptive grouped deep compressed sensing mechanism is used to compress frequency domain coefficients and combine them with precision sensing quantization to generate compressed observations. S4. Perform an inverse transformation on the compressed observations to reconstruct the approximate gradient; S5. Calculate adaptive gating weights based on the estimated reconstruction noise level, and use the gating weights to update the error feedback cache and global model.
[0028] The asynchronous federated learning method based on saliency deep compressed sensing will be described in detail below with reference to specific implementation methods.
[0029] S1. Initialize the global model and obtain the gradient of the local model.
[0030] In this step, the global model is initialized via the server. (Right now Set the total number of training rounds. Learning rate sparsity and sampling rate and the initialized global model Broadcast to all available clients.
[0031] Client In the global model that receives the broadcast Then, using local private datasets Perform several rounds of stochastic gradient descent training, calculate the difference between the local model and the global model, and obtain the gradient accumulation. .
[0032] S2. Based on discrete cosine transform and adaptive dynamic scaling mechanism, frequency domain adaptive sparsification is performed on the global model gradient to obtain frequency domain coefficients.
[0033] This step utilizes the coefficient characteristics of the gradient in the frequency domain to maximize compression efficiency, specifically including: S21. Transform the gradient to the frequency domain based on the discrete cosine transform.
[0034] In this step, the gradient cumulant matrix is first... Divided into The gradient is divided into blocks of a certain size, and a two-dimensional discrete cosine transform (2D-DCT) is applied to each block to transform the gradient to the frequency domain. The discrete cosine transform (DCT) has energy compression properties, which concentrates the spatial correlation of the gradient into low-frequency components, while high-frequency components often contain noise or non-significant information. The block size... The value is typically set to 32 or 16 to balance the compute / storage budget of edge devices and the energy compression efficiency of DCT.
[0035] S22. The frequency domain coefficients are normalized to the sensitive region of the deep compressed sensing model by adaptive dynamic scaling (ADS) to obtain the frequency domain coefficients.
[0036] The numerical range of gradients in neural networks is typically very small (e.g., This dynamic range (on orders of magnitude) is mismatched with the dynamic range typically designed for natural images (numerical range 0-1 or 0-255) in deep compressed sensing networks. To ensure the coding efficiency of DCS, this step introduces an adaptive dynamic scaling (ADS) mechanism for the frequency domain coefficients of each gradient block. Calculate its maximum absolute value as the scaling factor, specifically expressed as:
[0037] In the formula, Indicates the adaptive dynamic scaling factor; This represents the frequency domain coefficients of the gradient block.
[0038] Normalizing the frequency domain coefficients of the gradient block by division yields the normalized frequency domain coefficients, which are specifically expressed as follows:
[0039] In the formula, This represents the normalized frequency domain coefficients, i.e., the normalized frequency domain coefficients. Represents the frequency domain coefficients of the gradient block. Scaling factor. It will be sent along with the compressed data as metadata for subsequent recovery.
[0040] S3. A structure-adaptive grouped deep compressed sensing mechanism is used to compress frequency domain coefficients and combine them with precision sensing quantization to generate compressed observations.
[0041] Because different layers of a neural network (such as convolutional layers and fully connected layers) have completely different structural characteristics, this embodiment employs a structure-adaptive grouped depth-compressed sensing mechanism to compress the normalized frequency domain coefficients to accommodate these differences. Figure 3 As shown.
[0042] S31. Group the network layers according to their layer structure characteristics.
[0043] In this step, all layers of the model are divided into G groups based on their architecture type; for example, all convolutional layers are in one group, and all fully connected layers are in another. For each group, a dedicated DCS model is pre-trained. This ensures that the DCS model learns the statistical characteristics of a specific structure, thereby improving the quality of reconstruction.
[0044] S32. Dynamically sample gradient blocks based on a saliency network.
[0045] In this step, a lightweight saliency detection network (SaliencyNet) is used to process the frequency domain coefficients. This generates a saliency map reflecting the importance of information and calculates the saliency score for each gradient block. Specifically, the saliency detection network first outputs the saliency probability distribution map of the entire image. Then, based on the total sampling budget, this saliency probability distribution map is mapped to a pre-allocated sampling map. Subsequently, this pre-allocated sampling map is divided into blocks, and the saliency probability distribution map for each block is calculated. The sum of all values within a block is the significance score of that gradient block.
[0046] Significance score This reflects the importance of the gradient block to model updates. Gradient blocks with high significance (i.e., high significance scores) indicate that they are key features and require high-fidelity sampling, while gradient blocks with low significance (i.e., low significance scores) indicate that they are non-key features and can be compressed at a high factor. Based on this, this embodiment pre-defines a global basic measurement matrix, and calculates the significance score in step S32. Determine the number of measurement rows to be assigned to the gradient block. (The higher the score, the better) (The larger the value), then extract the first part sequentially from the preset global basic measurement matrix. Lines form the heterogeneous measurement matrix specific to this gradient block. Its size is Based on the constructed heterogeneous measurement matrix Gradient blocks are dynamically sampled. For highly significant gradient blocks, a higher sampling rate (i.e., more rows in the measurement matrix) is assigned to achieve high-fidelity sampling and retain more information; for low-significance gradient blocks, an extremely low sampling rate is assigned for heavy compression. The compressed measurement vector is generated by dynamically sampling gradient blocks based on the saliency network, specifically as follows:
[0047] In the formula, Indicates the first Layer The normalization coefficient of the block; Indicates the first Layer Heterogeneous measurement matrix of the block; Indicates the first Layer The compression measurement vector of the block.
[0048] S33. Combine precision-sensing quantization to generate compressed observations.
[0049] To further reduce communication overhead, this step introduces a block-level scaling quantization strategy. However, since directly truncating the gradient to FP16 may cause numerical underflow, this step first calculates the compressed measurement vector. The peak scalar is specifically represented as:
[0050] In the formula, Represents a compressed measurement vector; The peak value scalar represents the compressed measurement vector; This represents the maximum absolute value of all elements in the vector; This represents a local minimum constant.
[0051] The compressed measurement vector is normalized using this peak scalar as a scaling factor, specifically as follows:
[0052] In the formula, This represents the normalized compressed measurement vector.
[0053] The normalized compressed measurement vector is quantized, and its data format is truncated from 32-bit floating-point (FP32) to 16-bit half-precision floating-point (FP16) to obtain the quantized compressed observations in FP16 format, specifically represented as follows:
[0054] In the formula, This represents compressed observations, and its format is FP16.
[0055] To compress observations Scaling factor for (FP16 format) and Deep Compressed Sensing (DCS) (FP32 format) and ADS scaling factor As the final transmitted data packet, this method reduces the communication volume by an additional 50% compared to directly transmitting FP32.
[0056] S4. Perform an inverse transformation on the compressed observations to reconstruct the approximate gradient.
[0057] The client will include compressed observations. (FP16 format), DCS scaling factor (FP32 format) and ADS scaling factor data packets Uploaded to the server, the server received Then, the inverse transform reconstruction operation is performed, which specifically includes: first, performing an inverse quantization operation to restore the FP16 format data to the FP32 format, and then comparing it with the deep compressed sensing scaling factor. Multiply to obtain the compressed measurement vector; then, based on the layer grouping information, call the corresponding DCS decoder. Deep compressed sensing decoding is performed on the compressed measurement vectors to reconstruct the frequency domain coefficients from the compressed observations. Finally, perform the inverse discrete cosine transform to convert the frequency domain coefficients. Recovering the approximate gradient in the spatial domain .
[0058] S5. Calculate adaptive gating weights based on the estimated reconstruction noise level, and use the gating weights to update the error feedback cache and global model.
[0059] Theoretical derivation reveals that while DCS reconfiguration is efficient, it inevitably introduces reconfiguration errors. In asynchronous federated learning, if traditional staleness compensation (such as Taylor expansion compensation) is directly applied, the nonlinear part of the compensation term will interact with noise. Coupling, producing Item. Due to If the error is always positive, it will accumulate over time (i.e., destructive interference), causing the model to diverge.
[0060] To address this, this step proposes a noise-gated aggregation mechanism: an adaptive gating weight is calculated by estimating the reconstruction noise level, and this gating weight is used to buffer the error feedback. Adaptive stale error attenuation control is implemented to prevent the accumulation and amplification of noise during asynchronous updates, thereby achieving robust global model updates.
[0061] In this step, the error feedback cache is updated using gating weights, which can be specifically represented as follows:
[0062] In the formula, This represents the gating weight, used to suppress error terms when high noise is detected; Indicates error feedback buffer; This indicates the updated error feedback cache; The ideal gradient is represented, but in practice, it can be approximated using historical statistical estimation or partial validation data. When significant reconstruction noise or severe obsolescence is detected, the gating weights... By approaching 0, historical noise is actively "gated" out to prevent it from entering the next iteration, thus ensuring the convergence stability of the algorithm.
[0063] The global model update is based on the updated error feedback cache, specifically as follows:
[0064] In the formula, Indicates the current number Global model parameters during round iteration; This represents the updated global model; This represents the learning rate.
[0065] This embodiment also conducted a total communication volume comparison experiment on the EMNIST, Fashion-MNIST, and CIFAR-10 datasets, comparing the above-mentioned asynchronous federated learning method based on saliency deep compressed sensing with the existing full-precision synchronous federated learning FedAvg and the asynchronous benchmark Overlap-FedAvg method. The comparison results are as follows: Figure 4As shown, this method reduces the communication volume by several orders of magnitude. For example, in the CIFAR-10 mission, the communication volume was reduced from 1675MB in FedAvg to 250MB, a reduction of about 85%, which greatly improved communication efficiency.
[0066] This embodiment also decomposes and compares the client's single-round upload latency under bandwidth-constrained conditions, and the results are as follows: Figure 5 As shown, this method significantly reduces the total latency at a bandwidth of 2Mbps. Although it introduces approximately 0.55 seconds of DCS computation overhead, the transmission time is greatly reduced. On EMNIST, the total latency is reduced from 18.41 seconds to 3.31 seconds, achieving a speedup of 5.5 times.
[0067] This embodiment also analyzes the model accuracy of this method, and the results are shown in Table 1.
[0068] Table 1. Model accuracy analysis results
[0069] As shown in Table 1, in the later stage of training (Round 100), the accuracy of this method is almost on par with the full-precision FedAvg, proving that the ASED mechanism effectively overcomes the negative impact of compression noise.
[0070] The experimental results above demonstrate that, despite the use of lossy compression, this method still achieves convergence accuracy comparable to full-precision synchronous federated learning (FedAvg) and asynchronous benchmarks (Overlap-FedAvg) on datasets such as EMNIST, Fashion-MNIST, and CIFAR-10, achieving the best balance between communication efficiency and model utility.
[0071] Example 2 In a typical embodiment of the present invention, this embodiment discloses an asynchronous federated learning system based on saliency deep compressed sensing, comprising: The initialization module is configured to: initialize the global model and obtain the gradient of the local model; The ADS module is configured to: perform frequency domain adaptive sparsification on the local model gradient based on discrete cosine transform and adaptive dynamic scaling mechanism to obtain frequency domain coefficients. The DCS module is configured to compress frequency domain coefficients based on a structure-adaptive grouped deep compressed sensing mechanism and generate compressed observations by combining precision sensing quantization. The inverse transform module is configured to reconstruct an approximate gradient by performing an inverse transform on the compressed observations. The update module is configured to: calculate adaptive gating weights based on the estimated reconstruction noise level, and use the gating weights to update the error feedback cache and the global model.
[0072] Example 3 In a typical embodiment of the present invention, this embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the steps in the asynchronous federated learning method based on saliency-based deep compressed sensing as described in Embodiment 1. These steps include: S1. Initialize the global model and obtain the global model gradient; S2. Based on the discrete cosine transform and adaptive dynamic scaling mechanism, the global model gradient is subjected to frequency domain adaptive sparsification to obtain frequency domain coefficients; S3. A structure-adaptive grouped deep compressed sensing mechanism is used to compress frequency domain coefficients and combine them with precision sensing quantization to generate compressed observations. S4. Perform an inverse transformation on the compressed observations to reconstruct the approximate gradient; S5. Calculate adaptive gating weights based on the estimated reconstruction noise level, and use the gating weights to update the error feedback cache and global model.
[0073] Example 4 In a typical embodiment of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the asynchronous federated learning method based on saliency-based deep compressed sensing as described in Embodiment 1. These steps include: S1. Initialize the global model and obtain the global model gradient; S2. Based on the discrete cosine transform and adaptive dynamic scaling mechanism, the global model gradient is subjected to frequency domain adaptive sparsification to obtain frequency domain coefficients; S3. A structure-adaptive grouped deep compressed sensing mechanism is used to compress frequency domain coefficients and combine them with precision sensing quantization to generate compressed observations. S4. Perform an inverse transformation on the compressed observations to reconstruct the approximate gradient; S5. Calculate adaptive gating weights based on the estimated reconstruction noise level, and use the gating weights to update the error feedback cache and global model.
[0074] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An asynchronous federated learning method based on saliency-based deep compressed sensing, characterized in that, include: Initialize the global model and obtain the gradient of the local model; Based on discrete cosine transform and adaptive dynamic scaling mechanism, frequency domain adaptive sparsification is performed on the local model gradient to obtain frequency domain coefficients. A structure-adaptive grouped deep compressed sensing mechanism is used to compress frequency domain coefficients and combine them with precision sensing quantization to generate compressed observations. An approximate gradient is obtained by reconstructing the compressed observations through an inverse transform. Adaptive gating weights are calculated based on the estimated reconstruction noise level, and the error feedback cache and global model are updated using the gating weights.
2. The asynchronous federated learning method based on saliency-based deep compressed sensing as described in claim 1, characterized in that, The frequency domain adaptive sparsity processing specifically includes: using two-dimensional discrete cosine transform to transform the gradient to the frequency domain, and using adaptive dynamic scaling to normalize the frequency domain coefficients to the sensitive interval of the deep compressed sensing model to obtain the frequency domain coefficients.
3. The asynchronous federated learning method based on saliency-based deep compressed sensing as described in claim 1, characterized in that, The structure-adaptive grouped deep compressed sensing mechanism compresses frequency domain coefficients, specifically including: grouping network layers according to layer structure characteristics, using a saliency network to generate a saliency map reflecting the importance of information and calculating the saliency score of each gradient block; constructing a heterogeneous measurement matrix based on the saliency score and dynamically sampling the gradient blocks to generate compressed measurement vectors.
4. The asynchronous federated learning method based on saliency-based deep compressed sensing as described in claim 3, characterized in that, The compressed measurement vector is specifically represented as follows: In the formula, Indicates the first Layer The normalization coefficient of the block; Indicates the first Layer Heterogeneous measurement matrix of the block; Indicates the first Layer The compression measurement vector of the block.
5. The asynchronous federated learning method based on saliency-based deep compressed sensing as described in claim 3, characterized in that, The process of generating compressed observations by combining precision-aware quantization includes: calculating the peak scalar of the compressed measurement vector, normalizing the compressed measurement vector using the peak scalar as a scaling factor, quantizing the normalized compressed measurement vector, truncating its data format from 32-bit floating-point to 16-bit half-precision floating-point, and obtaining the quantized compressed observations.
6. The asynchronous federated learning method based on saliency-based deep compressed sensing as described in claim 1, characterized in that, Updating the error feedback cache using gating weights is specifically represented as follows: In the formula, Indicates the gating weight; Indicates error feedback buffer; This indicates the updated error feedback cache; This represents the ideal gradient.
7. The asynchronous federated learning method based on saliency-based deep compressed sensing as described in claim 6, characterized in that, The global model update is based on the updated error feedback cache, specifically as follows: In the formula, Indicates the current number Global model parameters during round iteration; This represents the updated global model; This represents the learning rate.
8. An asynchronous federated learning system based on saliency-based deep compressed sensing, characterized in that, include: The initialization module is configured to: initialize the global model and obtain the gradient of the local model; The ADS module is configured to: perform frequency domain adaptive sparsification on the local model gradient based on discrete cosine transform and adaptive dynamic scaling mechanism to obtain frequency domain coefficients. The DCS module is configured to compress frequency domain coefficients based on a structure-adaptive grouped deep compressed sensing mechanism and generate compressed observations by combining precision sensing quantization. The inverse transform module is configured to reconstruct an approximate gradient by performing an inverse transform on the compressed observations. The update module is configured to: calculate adaptive gating weights based on the estimated reconstruction noise level, and use the gating weights to update the error feedback cache and the global model.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the asynchronous federated learning method based on saliency-based deep compressed sensing as described in any one of claims 1-7.
10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the asynchronous federated learning method based on saliency-based deep compressed sensing as described in any one of claims 1-7.