Bandwidth distortion adaptive multi-layer scalable feature compressor and method and system thereof

Abstract

The invention provides a bandwidth distortion self-adaptive multi-layer scalable feature compressor and a method and system thereof, and relates to the technical field of machine learning. According to the invention, through a code rate distortion joint controller, bit width distribution and fragment size of each layer of a layering number are dynamically decided according to a real-time network state training stage and a precision demand; the input features are mapped to a subspace with more concentrated energy through a layered learnable orthogonal transformation module, and feature channels are clustered and grouped; different bit widths are distributed to the coefficients based on the importance degree through a non-uniform quantization module; dynamically adjusting the fragmentation size and the fragmentation sending rhythm through an RTT adaptive scheduling module according to the round-trip delay and the jitter degree; recording an accumulated reconstruction error through an error tracking compensation module and carrying out residual encoding; and periodically transmitting the anchoring layer through the key frame synchronization module to eliminate accumulated errors. According to the method, the robustness, convergence stability and communication efficiency of distributed training in a complex network environment are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed machine learning, in particular to a bandwidth-distortion adaptive multi-layer scalable feature compressor and a method and system thereof. BACKGROUND

[0002] In distributed training scenarios such as federated learning and edge intelligence, the balance between communication efficiency and model accuracy is a key challenge. Due to the existence of link bandwidth fluctuations and latency constraints, there are significant differences in the accuracy requirements of different training stages. For example, higher reconstruction quality is needed in the early stage of training to ensure stable convergence of the model, while in the middle and later stages, more emphasis is placed on compression efficiency and throughput, allowing for moderate distortion. However, most existing technologies use fixed bit width or single-layer compression methods, making it difficult to dynamically adjust the compression strategy according to the task stage and link state. Fixed bit width methods can lead to resource waste when bandwidth is abundant, while in bandwidth-constrained situations, they can result in low communication efficiency or even irreversible accuracy loss due to under-transmission or retransmission. Single-layer encoding methods lack fine-grained quality and bit control capabilities, making it difficult to meet real-time and accuracy requirements in complex network environments.

[0003] To alleviate the communication bottleneck in distributed learning, researchers have proposed various gradient compression methods. These methods mainly include gradient sparsification, quantization, low-rank decomposition, and differential encoding. Gradient sparsification reduces the transmission dimension by retaining important gradient elements or randomly discarding some elements, but it lacks adaptability to training stages and task characteristics, which can lead to loss of critical information or introduction of excessive noise. Quantization methods reduce communication overhead by reducing the number of bits required to represent gradients, but fixed bit width quantization cannot adapt to the dynamic changes in gradient distribution, and low bit width can result in insufficient information retention, while high bit width is difficult to meet the needs of low-bandwidth scenarios. Low-rank decomposition methods rely on singular value decomposition to approximate high-dimensional gradient tensors as low-rank matrix products, thereby reducing communication costs, but they have high computational overhead and limited ability to express complex gradient change patterns. Differential encoding methods use the temporal correlation of gradients to transmit only the difference and use error feedback mechanisms to compensate for information loss, but their stability and effectiveness are still not ideal in environments with large gradient fluctuations or weak networks.

[0004] Overall, existing technologies focus on a single compression dimension, such as sparsification, quantization, or low-rank decomposition, and lack the ability to jointly optimize from multiple perspectives such as space, frequency domain, and time series. This limitation makes it difficult to achieve an ideal balance between compression rate and information retention, and it is difficult to simultaneously achieve high compression rate, low latency, and high accuracy. In addition, existing methods lack applicability in diverse distributed learning scenarios, making it difficult to meet the actual needs of complex environments such as low bandwidth, high latency, and strong jitter.

[0005] Therefore, the present application is proposed. SUMMARY

[0006] The present application aims to provide a bandwidth and distortion adaptive multi-layer scalable feature compressor and its method and system to solve the problem that communication efficiency and model accuracy are difficult to be considered in the existing distributed training.

[0007] To solve the above technical problems, the present application realizes the following technical solutions: A bandwidth and distortion adaptive multi-layer scalable feature compressor is applied between the client and the server of a distributed machine learning system to compress the original data obtained by the client or the server and output compressed data, including a rate-distortion joint controller, a hierarchical learnable orthogonal transform module, a non-uniform quantization module, an RTT adaptive scheduling module and an error tracking compensation module. The rate-distortion joint controller is used to dynamically decide the total number of layers, the bit width allocation of each layer and the slice size according to the real-time network state, the training stage and the accuracy requirement, to dynamically adjust the input original data and minimize the reconstruction distortion of the original data and the reconstructed data; the reconstructed data is obtained by decoding and reconstructing the compressed data by the decoder of the receiving end; the network state includes available bandwidth, round-trip delay and jitter; The hierarchical learnable orthogonal transform module is used to map the original data to a subspace with more concentrated energy and cluster and group feature channels, so that in the data transmission process, the basic layer only carries the center coefficient and low-frequency component of each group, and the enhancement layer encodes the high-frequency details and deviation part in each group layer by layer, performs progressive data reconstruction from coarse to fine, and obtains transmissible hierarchical data; The non-uniform quantization module is used to calculate the importance score based on the transformed hierarchical data to dynamically allocate bit width; The RTT adaptive scheduling module is used to dynamically adjust the slice size and the slice interval according to the real-time network state; The error tracking compensation module is used to calculate the cumulative reconstruction error of each round according to the residual coding of the enhancement layer and dynamically adjust the bit width allocation of the next round, so as to obtain the compressed data.

[0008] Preferably, the original data includes original features obtained from the client or global gradients obtained from the server; The rate-distortion joint controller includes a network state awareness sub-module, a training stage awareness sub-module and an accuracy requirement sub-module; The network state awareness sub-module calculates the effective available bandwidth by monitoring the available bandwidth, round-trip delay and jitter, and the formula is: ; Wherein, is the effective available bandwidth; For real-time monitoring of available bandwidth; RTT stands for Round Trip Time. For RTT jitter; This is a conservative coefficient used to address protocol overhead and network fluctuation redundancy. The training phase perception submodule is used to calculate the distortion weight coefficient based on the current training round and the total number of rounds, using the following formula: ; in, The distortion weight coefficient for the current training round t; Total number of rounds; , These represent the maximum and minimum distortion weight coefficients, respectively; The decay rate parameter is used to control the transition speed from high precision to high compression. The accuracy requirement submodule is used to minimize the weighted reconstruction distortion of the original data and the reconstructed data by jointly optimizing the total number of layers, the bit width of each layer, and the size of each fragment, based on the available bandwidth and distortion weight coefficient, under a preset latency budget constraint. The objective function formula for the joint optimization is: ; in, This represents the total number of layers; For the first Layer width; This refers to the size of the slice; Distortion due to weighted reconstruction; For the first Layer weights; The original data; For the first The reconstructed data corresponding to the layer; It is an L2 norm; This indicates a minimize operation; The constraints are that the total bit rate does not exceed the bandwidth-delay product, and the actual transmission time does not exceed the delay budget. The formula is: ; in, For the first Layer bitrate; Budget for delay; This refers to the actual transmission time. Indicates being bound by; Total number of layers The determination is based on available communication resources and is dynamically determined using the following formula: ; in, Fixed bitrate at base layer The average bit rate for a single enhancement layer; This indicates taking the maximum value; The optimal bit width is assigned to each layer based on the Lagrange multiplier method, and the formula is as follows: ; in, For the first Optimal bit width for the layer; For the first Layer weighted reconstruction distortion; For the first Layer in position width The predicted distortion is below; B is the candidate bit width set; The fragment size is dynamically adjusted based on the bandwidth-delay product and the number of layers, using the following formula: ; in, To adjust the fragment size; , These are the lower and upper bounds of the fragment size, respectively.

[0009] Preferably, it also includes an early stop decision mechanism, that is, by evaluating the marginal benefit in real time, if the ratio of the distortion improvement to the bitrate overhead at the current layer is lower than a set threshold. If it is estimated that completing the next layer of transmission will exceed the latency budget, the data transmission of subsequent enhancement layers should be stopped immediately to avoid wasting communication resources when the marginal benefit is extremely low. The expression is: ; in, , Representing the current layer The next layer Reconstruction distortion after transmission is complete; Indicates the elapsed transmission time; Indicates the estimated transmission time for the next layer; Budget for time delay.

[0010] Preferably, in the hierarchical learnable orthogonal transformation module, a learnable orthogonal transformation method is used to map the original data to a subspace with more concentrated energy, and a partitioning method based on energy thresholds is used to cluster and group the feature channels in the subspace according to the channel entropy characteristics, as shown in the following expression: The formula for learnable orthogonal transformation is: ; in, The transformed data; The input is the raw data, where C is the number of channels, H is the height, and W is the width. It is a learnable orthogonal transformation matrix; , It is the identity matrix; The formula for classifying based on energy thresholds is: , ; in, For the first Energy of each channel; For the first Data after transformation of each channel; Represents the L2 norm; For the first The entropy value of each channel; Let be the normalized probability distribution of the channel coefficients; i represents the index of the element in the c-th channel. All channels were grouped into clusters based on their energy thresholds. There are several groups, and the formula is: ; in, For the first Group, containing energy in All channels within the interval; , The first The energy threshold of group k-1; Total number of groups; Indicates the current channel; After clustering, the mean of all channel data in each group is calculated, and the low-frequency component is extracted using a low-pass filter as the base layer transmission content. The base layer transmission content is represented as follows: ; in, Indicates the first The group's base layer transmits content; This indicates a low-pass filter to preserve key low-frequency information; To calculate the within-group centrality coefficient; For the first All channel data for the group; Subtract the previous data from all channels in each layer and group. The residuals from the cumulative reconstruction results of each layer are used as the content transmitted in the enhancement layer, and the residuals are encoded layer by layer. The expression is as follows: ; ; in, For the first Layer The group's enhancement layer transmits content; For the first Layer enhancement layer transmission content of the group; represents the 0th layer 1st enhancement layer transmission content of the group, i.e. the 1st base layer transmission content of the group.

[0011] Preferably, the formula of the importance score is: ; wherein, is the importance of the i-th channel; is the energy of the i-th channel; is the entropy value of the i-th channel, is the L2 norm of the gradient; is the gradient of the i-th channel; is the weight coefficient, adjusted according to the task characteristics; then, for the coefficients with high importance scores, higher bit widths are assigned to reduce the quantization error; while for the coefficients with low importance scores, lower bit widths are used; the bit width allocation formula is: ; wherein, is the allocated bit width of the i-th channel; , , respectively represent the set high, medium and low level bit widths; , respectively represent the set high and low importance score levels; , , respectively represent the set high, medium and low level bit widths; , respectively represent the set high and low importance score levels; Meanwhile, combined with the µ-law adaptive non-uniform quantization method, the effective resolution of the coefficients with low importance scores under low bit conditions is improved through logarithmic compression, and the expression is: ; wherein, represents the result of the µ-law quantization of the input to-be-quantized coefficient ; is the compression parameter; is the maximum value of the quantized coefficient; is the sign function, used to retain the sign of ; is the logarithmic function, which, through logarithmic compression, makes the to-be-quantized coefficients with small values obtain higher resolution.

[0012] Preferably, in the RTT adaptive scheduling module, the jitter level of the round-trip time (RTT) is monitored in real time, and the fragment size P and the fragmentation interval are dynamically adjusted. When the network jitter is large, the fragment size is reduced and an acknowledgment interval is inserted to reduce the risk of timeout and packet loss. When the bandwidth is stable and sufficient, the fragment size is increased to reduce header overhead and improve overall communication efficiency, as shown in the following expression: The formula for real-time monitoring of RTT jitter is: ; in, RTT jitter is used to measure the degree of fluctuation in round-trip time. Indicates standard deviation; This represents the round-trip time delay value obtained from the nth measurement; ; in, ; , They represent , The size of the slice at any given moment; , These represent the maximum and minimum values ​​of the fragment size, respectively. , These represent the high and low thresholds for RTT jitter, respectively. This indicates the adjustment step size for the slice size, used to adjust the slice size in non-extreme jitter conditions; The formula for the transmission interval is: ; in, This indicates the transmission interval, which is the time interval between two transmissions of a fragment; This represents the average round-trip time delay. , This is the adjustment coefficient.

[0013] Preferably, in the error tracking compensation module, an error tracking table is established to record the cumulative reconstruction error of different channels and frequency bands, for use in the next round of dynamic bit width allocation and early stop decision, as follows: First, calculate the residual coding of the enhancement layer for each layer, using the following formula: ; in, For the first Layer The residuals of the group; For the first The group's enhancement layer transmits content. For the first Layer Group reconstructed data; Next, the cumulative reconstruction error is calculated, which consists of the sum of the squares of the errors from the previous round and the current round, as shown in the formula: ; in, For the first The first channel in the The cumulative reconstruction error of the wheel; For the first The first channel in the The wheel's enhancement layer transmits content. For the first The first channel in the Reconstructed data for the wheel; The bit width is dynamically adjusted based on the cumulative reconstruction error, using the following formula: ; in, , They represent the first The first channel in the Wheel width; This is the error threshold; To round down; When the maximum error of all channels is lower than the target error threshold, the enhancement layer coding is stopped to avoid invalid transmission. The stopping condition is: ; in, This represents the maximum cumulative error among all channels in round t; The target error threshold is set.

[0014] Preferably, it also includes a keyframe synchronization module, that is, it introduces a synchronization strategy to clear accumulated reconstruction errors by periodically transmitting keyframes, and the triggering conditions are as follows: In the first case, an independently encoded keyframe is sent every N training rounds, and reconstruction is performed without relying on historical residuals. The expression is: ; in, Indicates a keyframe; Indicates the current round; Indicates the modulo operation; For synchronization period; In the second scenario, a sudden surge in innovation energy triggers the transmission of keyframes, namely: the total energy of the current round. Compared to the previous round of energy When the ratio exceeds the set energy surge threshold, a keyframe is triggered to re-establish the baseline, as expressed by: ; in, Threshold of energy burst; The third case, when the accumulated distortion exceeds the set threshold, namely: calculate the accumulated reconstruction error sum of the last round , when the accumulated reconstruction error sum exceeds the set threshold , trigger the transmission of key frames, the expression is: ; Wherein, is the average reconstruction error of the first round; is the accumulated distortion threshold; The fourth case, when the long packet loss condition, empty error tracking state, to ensure that the encoding end and the decoding end re-synchronization, avoid the error drift caused by the loss of accuracy.

[0015] The application also provides a bandwidth distortion adaptive multi-layer scalable feature compression method, comprising: Collecting real-time network state, training phase and accuracy requirement; wherein, the network state includes available bandwidth, round-trip delay and jitter; Obtaining original data; wherein, the original data includes original features sent by the client or original gradient sent by the server; The original data is mapped to a subspace with more concentrated energy through a learnable orthogonal transformation, and the transformed feature channels are clustered and grouped, so that in the data transmission process, the basic layer only carries the center coefficient and low frequency component of each group, and the enhancement layer encodes the high frequency details and deviation part in each group layer by layer residual, performs progressive data reconstruction from coarse to fine, and obtains transmissible layered data; Based on the transformed layered data, importance score is calculated to dynamically allocate bit width; According to the real-time network state, adjust the slice size and the interval of sending slices; According to the residual coding of the enhancement layer, calculate the accumulated reconstruction error of each round and dynamically adjust the bit width allocation of the next round, so as to obtain compressed data; According to the real-time network state, training phase and accuracy requirement, dynamically decide the total number of layers, bit width allocation of each layer and slice size, so as to dynamically adjust the input original data, minimize the reconstruction distortion between the original data and the reconstructed data; wherein, the reconstructed data is obtained by decoding and reconstructing the compressed data by the decoder.

[0016] The application further provides a distributed machine learning system comprising a plurality of clients and a server, and a bandwidth-distortion adaptive multi-layer scalable feature compressor as described above between the clients and the server; the clients send original features to the multi-layer scalable feature compressor for compression, and output compressed data; the server receives and aggregates the compressed data, and generates reconstructed data, i.e. global gradients, which are compressed by the multi-layer scalable feature compressor to generate compressed gradients and are then sent to each client; each client updates a local model according to the compressed gradients, and continues the next round of local training based on the updated model using the original features until the model converges.

[0017] The application further provides a computer-readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor of a device in which the computer-readable storage medium is located, implement a bandwidth-distortion adaptive multi-layer scalable feature compression method as described above.

[0018] In summary, compared with the prior art, the application has the following beneficial effects: The application proposes a bandwidth-distortion adaptive multi-layer scalable feature compression framework by integrating six modules, i.e. a rate-distortion joint controller, a hierarchical learnable orthogonal transform module, a non-uniform quantization module, an RTT adaptive scheduling module, an error tracking compensation module and a key frame synchronization module, which significantly improves the robustness, convergence stability and communication efficiency of distributed training in a complex network environment. The application fully utilizes the energy concentration characteristics, spatial redundancy, frequency domain sparsity and temporal consistency of features, and has good adaptability and scalability, and is particularly suitable for weak network, low bandwidth, high delay, strong jitter and other resource-constrained environments.

[0019] Specifically, the rate-distortion joint controller dynamically decides the number of layers, the allocation of bit widths of each layer and the size of each slice according to real-time network status, training stage and precision requirement, so as to minimize the reconstruction distortion under bandwidth constraints. The hierarchical learnable orthogonal transform module uses learnable orthogonal transform (LoT) to map input features to a subspace with more concentrated energy, and clusters and groups feature channels according to the entropy value characteristics of the channels. The intra-layer non-uniform quantization module uses a non-uniform quantization strategy based on importance, and allocates different bit widths according to the importance of coefficients. The RTT adaptive scheduling module designs an adaptive on-chip rate adjustment mechanism based on round-trip time (RTT). The error tracking compensation module specifies that the object of enhancement layer coding is the residual on the reconstructed result of the base layer or lower layers, and establishes an error tracking table to record the cumulative reconstruction error. The key frame synchronization module clears the cumulative error by periodically transmitting anchor layers (key frames). BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to make the technical solutions of the embodiments of the present application clearer, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope. Other related drawings can also be obtained by those skilled in the art without any creative effort.

[0021] Figure 1 The system framework diagram of the bandwidth and distortion adaptive multi-layer scalable feature compression method provided for the embodiment one.

[0022] Figure 2 The flowchart of the bandwidth and distortion adaptive multi-layer scalable feature compression method provided for the embodiment one.

[0023] The present application will be further described below in conjunction with the drawings and specific embodiments. DETAILED DESCRIPTION

[0024] In order to make the technical solutions of the embodiments of the present application clearer, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope. Other related drawings can also be obtained by those skilled in the art without any creative effort.

[0025] Embodiment one The embodiment one of the present application provides a bandwidth and distortion adaptive multi-layer scalable feature compression method, which can be realized by a bandwidth and distortion adaptive multi-layer scalable feature compression device (hereinafter referred to as a feature compression device), and in particular, by one or more processors in the feature compression device.

[0026] In the present embodiment, the feature compression device can be an electronic device equipped with a processor, the processor is provided with a computer program of the bandwidth and distortion adaptive multi-layer scalable feature compression method and the computer program can be executed, such as a computer, a smart phone, a smart tablet, a workstation, etc., which are not limited here.

[0027] In existing technologies, balancing communication efficiency and model accuracy in distributed machine learning systems faces significant challenges. Fixed-bit-width or single-layer compression methods struggle to adapt to dynamic network environments and training phase requirements, leading to resource waste or accuracy loss. For example, high accuracy is needed in the early stages of training to ensure model convergence, while compression efficiency becomes more important in the later stages. Existing methods lack adaptive adjustment capabilities and cannot achieve the optimal trade-off under bandwidth fluctuations and latency constraints.

[0028] To address the aforementioned issues, a mechanism is needed to dynamically adjust the compression strategy based on real-time network status and training phases. By analyzing the varying demands for reconstruction quality at different stages, we consider mapping features to an energy-concentrated subspace and employing hierarchical encoding to progressively refine the reconstruction. Furthermore, we introduce dynamic bit-width allocation and slice scheduling mechanisms to handle network fluctuations, while simultaneously reducing the impact of accumulated errors on subsequent training through an error tracking compensation mechanism.

[0029] The core of distributed learning is "parallel training by multiple nodes (such as clients) + global aggregation by a central node (such as a server)". Figure 1 The distributed machine learning system shown has an overall architecture including m clients, a server, and a multi-layer scalable feature compressor based on bandwidth distortion adaptation. The multi-layer scalable feature compressor is used to implement the feature compression process and the gradient compression process. The multi-layer scalable feature compressor compresses the original features obtained from the clients or the global gradients from the server (original features and global gradients are collectively referred to as original data) and outputs compressed data (compressed features or compressed gradients).

[0030] The core idea of ​​this invention is to propose a bandwidth-distortion adaptive multilayer scalable feature compression framework (BD-SLTC), namely a multilayer scalable feature compressor based on bandwidth distortion adaptation. It includes: a rate-distortion joint controller, a hierarchical learnable orthogonal transform module, a non-uniform quantization module, an RTT adaptive scheduling module and an error tracking compensation module, and a keyframe synchronization module. These modules work together to ensure efficient communication and high-precision model training in complex network environments.

[0031] Existing methods often employ fixed compression strategies or single-dimensional optimization, making it difficult to adapt to dynamic network environments and training requirements. For example, fixed-bit-width quantization cannot be flexibly adjusted when bandwidth fluctuates, and single-layer encoding lacks progressive reconstruction capabilities. This scheme combines multi-layer scalable encoding with dynamic parameter adjustment. When bandwidth is limited, it prioritizes the transmission of the base layer to ensure basic accuracy, and gradually enhances details when bandwidth is ample. At the same time, it reduces cumulative distortion through an error compensation mechanism, significantly improving the balance between communication efficiency and model accuracy in complex network environments.

[0032] (a) Rate Distortion Joint Controller In this framework, the encoder first introduces a rate-distortion joint controller, which jointly determines the number of layers, the bit-width of each layer, and the size of each slice, by real-time sensing the network bandwidth, round-trip delay, and the accuracy requirement in the training phase, so as to minimize the overall distortion under the constraint condition. Specifically, the encoder adopts a hierarchical structure of 'base layer + multiple enhancement layers', in which the base layer carries the most important low-frequency components and dominant features to ensure that the reconstruction accuracy is still available under the minimum communication condition; the enhancement layer compensates for the detailed information of different frequency bands or groups through the inter-layer incremental residual, so that the model can gradually improve the reconstruction quality layer by layer when the bandwidth condition allows.

[0033] The rate-distortion joint controller is used to dynamically decide the total number of layers, the bit-width allocation of each layer, and the slice size according to the real-time network state, the training phase, and the accuracy requirement, to dynamically adjust the input original data, and minimize the reconstruction distortion of the original data and the reconstructed data; wherein the reconstructed data is obtained by decoding the compressed data by the decoder; the network state includes available bandwidth, round-trip delay, and jitter.

[0034] First, the input feature or gradient data is dynamically parameter-adjusted by the rate-distortion joint controller. The controller real-time senses three types of information: network state, training phase, and accuracy requirement. The network state sensing module calculates the effective available bandwidth by monitoring the available bandwidth, round-trip time (RTT), and jitter. The available bandwidth is the amount of data that can be actually transmitted per unit time (such as 5 Mbps), which reflects the upper limit of transmission capacity. The round-trip time (RTT) is the time from the sending end to the receiving end and then back (such as 100 ms), which reflects the network response speed. The jitter is the fluctuation value of the RTT (such as ±20 ms), which reflects the network stability.

[0035] The rate-distortion joint controller includes a network state sensing submodule, a training phase sensing submodule, and an accuracy requirement submodule. The network state sensing submodule calculates the effective available bandwidth by monitoring the available bandwidth, round-trip time (RTT), and jitter, and the formula is: ; Wherein, is the effective available bandwidth; is the real-time monitoring available bandwidth; RTT is the round-trip time; is the RTT jitter; is a conservative coefficient, which is used to cope with protocol overhead and network fluctuation redundancy.

[0036] The training phase sensing submodule is used to calculate the distortion weight coefficient according to the current training round and the total number of rounds, and the formula is: ; in, The distortion weight coefficient for the current training round t; Total number of rounds; , These represent the maximum and minimum distortion weight coefficients, respectively; This is the decay rate parameter, used to control the transition speed from high precision to high compression.

[0037] The accuracy requirement submodule is used to minimize the weighted reconstruction distortion of the original data and the reconstructed data by jointly optimizing the total number of layers, the bit width of each layer, and the size of each fragment, based on the available bandwidth and distortion weight coefficient, under a preset latency budget constraint. The objective function formula for the joint optimization is: ; in, This represents the total number of layers; For the first Layer width; This refers to the size of the slice; Distortion due to weighted reconstruction; For the first Layer weights; The original data; For the first The reconstructed data corresponding to the layer; It is an L2 norm; This indicates a minimize operation.

[0038] The constraints are that the total bit rate does not exceed the bandwidth-delay product, and the actual transmission time does not exceed the delay budget. The formula is: ; in, For the first Layer bitrate; Budget for delay; This refers to the actual transmission time. It indicates that one is bound by something.

[0039] Total number of layers The determination is based on available communication resources and is dynamically determined using the following formula: ; in, Base layer fixed bit rate, The average bit rate for a single enhancement layer; This indicates taking the maximum value.

[0040] The optimal bit width is assigned to each layer based on the Lagrange multiplier method, and the formula is as follows: ; in, is the first layer optimal bit width; is the first layer weighted reconstruction distortion; is the first layer estimated distortion under bit width B; B is a candidate bit width set.

[0041] According to the bandwidth delay product and the number of layers, the slice size is dynamically adjusted, and the formula is: ; wherein, is the adjusted slice size; , are the lower bound and the upper bound of the slice size, respectively.

[0042] Compared with the prior art, the traditional method adopts fixed number of layers and uniform quantization bit width, which cannot adapt to dynamic network environment and training stage requirements. For example, fixed 3-layer encoding may exceed the delay budget when the bandwidth suddenly drops, and fixed 8-bit quantization may waste bandwidth resources in the later training stage. The scheme dynamically adjusts the layered structure and bit width allocation by joint optimization of the model, for example, increasing the enhancement layer to improve accuracy when the bandwidth is sufficient, and reducing the number of layers to ensure real-time performance when the bandwidth is limited. Compared with the single-layer encoding method, the multi-layer progressive reconstruction mechanism can reduce the reconstruction error under the same code rate, for example, the basic layer transmits the main energy information, and the enhancement layer gradually supplements the details.

[0043] Through the above technical scheme, the application realizes adaptive feature compression according to network state and training stage. In a network fluctuation scenario, the transmission reliability is ensured by dynamically adjusting the number of layers and the slice size, for example, automatically reducing the slice to reduce the impact of packet loss when jitter increases. In different training stages, the balance between accuracy and compression is adjusted by the distortion weight coefficient, for example, maintaining high bit width to ensure model convergence stability in the early training stage, and appropriately reducing bit width to improve throughput in the later stage. The scheme effectively solves the problem of insufficient adaptability of fixed compression strategy in complex network environment, and improves the communication efficiency and model accuracy collaborative optimization ability of distributed machine learning system.

[0044] In order to adapt to unstable network environment, the application designs a slice mechanism based on delay window, ensures that data is progressively transmitted in small granularity and controllable delay, and triggers the early stop mechanism when the target distortion is reached or the delay budget is exhausted, thereby avoiding invalid communication overhead. The receiving end can complete one-time elastic reconstruction at any receiving level, which can meet the minimum accuracy requirement under weak network conditions, and can gradually improve the training effect in a high bandwidth scenario.

[0045] The early stop decision mechanism is to evaluate the marginal benefit in real time, and when the ratio of the distortion improvement of the current layer to the code rate overhead is lower than the set threshold​ , or when it is estimated that the next layer transmission will exceed the delay budget, the data transmission of the subsequent enhancement layer is immediately stopped to avoid wasting communication resources when the marginal benefit is very low, and to ensure that high-value information is transmitted preferentially within the delay constraint, the expression is: ; wherein, , respectively represent the reconstruction distortion of the current layer , the reconstruction distortion of the previous layer after the transmission is completed; represents the transmission time that has been spent; represents the estimated transmission time of the next layer; is the delay budget. Compared with the prior art, the traditional method usually adopts a fixed number of layers or a static stopping strategy, which cannot dynamically adjust the number of transmission layers according to real-time network fluctuations and task requirements, and is prone to resource waste or timeout packet loss when the bandwidth suddenly drops or the delay suddenly increases. The early stopping decision mechanism can adaptively terminate inefficient transmission and achieve more precise resource regulation in complex network environments by introducing marginal benefit evaluation and delay budget double constraints.

[0046] Through the above technical solutions, the present application effectively solves the problems of low transmission efficiency and resource waste of the prior art in a dynamic network environment, and can intelligently terminate redundant data transmission while ensuring basic reconstruction quality, improve communication resource utilization and reduce the risk of transmission timeout according to real-time channel conditions and task requirements.

[0047] (II) Hierarchical learnable orthogonal transform module This embodiment first uses a learnable orthogonal transform (Learnable Orthogonal Transform, LoT) to map the input features or gradients to a subspace with more concentrated energy. In this subspace, the feature channels are clustered and grouped according to the entropy value characteristics of the channels, and an energy threshold-based division method is used. After grouping, the base layer only carries the center coefficient and low-frequency component of each group to ensure transmission of the most critical information under limited bit budget; the enhancement layer encodes the high-frequency details and deviation parts in each group layer by layer to realize progressive reconstruction from coarse to fine. This method can prioritize areas with high energy proportion or high uncertainty under the same code rate, making optimal use of limited communication resources.

[0048] In the hierarchical learnable orthogonal transform module, the original data is mapped to a subspace with more concentrated energy using a learnable orthogonal transform method, and a division method based on energy threshold is used to cluster and group feature channels in the subspace according to the entropy value characteristics of the channels, and the expression is as follows: The learnable orthogonal transform formula is: ; in, The transformed data; The input is the raw data, where C is the number of channels, H is the height, and W is the width. It is a learnable orthogonal transformation matrix; , It is an identity matrix.

[0049] The formula for classifying based on energy thresholds is: , ; in, For the first Energy of each channel; For the first Data after transformation of each channel; Represents the L2 norm; For the first The entropy value of each channel; Let be the normalized probability distribution of the channel coefficients; i represents the index of the element in the c-th channel.

[0050] All channels were grouped into clusters based on their energy thresholds. There are several groups, and the formula is: ; in, For the first Group, containing energy in All channels within the interval; , The first The energy threshold of group k-1; Total number of groups; Indicates the current channel.

[0051] After clustering, the mean of all channel data in each group is calculated, and the low-frequency component is extracted using a low-pass filter as the base layer transmission content. The base layer transmission content is represented as follows: ; in, Indicates the first The group's base layer transmits content; This indicates a low-pass filter to preserve key low-frequency information; To calculate the within-group centrality coefficient; For the first All channel data for the group.

[0052] Subtract the previous data from all channels in each layer and group. The residuals from the cumulative reconstruction results of each layer are used as the content transmitted in the enhancement layer, and the residuals are encoded layer by layer. The expression is as follows: ; ; in, For the first Layer The group's enhancement layer transmits content; For the first Layer The group's enhancement layer transmits content; Indicates the 0th layer. The enhanced layer of the group transmits the content, namely the first The group's base layer transmits content.

[0053] Traditional methods often employ fixed orthogonal transforms, such as the discrete cosine transform, whose basis functions cannot adapt to the data distribution characteristics of specific tasks. This scheme achieves data-driven basis function optimization through a learnable orthogonal transform, improving energy concentration efficiency by approximately 30% compared to fixed transforms. Existing technologies often use fixed rules or random division for channel grouping, such as dividing into four groups according to channel order. This approach struggles to adapt to the varying importance of different data features. This scheme uses dynamic clustering based on a joint index of energy and entropy, effectively distinguishing between critical and secondary channels. Experimental results show that this method can reduce the quantization error of important channels by approximately 40%. Traditional residual coding typically uses a fixed number of layers, such as forcibly dividing into three enhancement layers, which can easily lead to bandwidth waste or insufficient accuracy. This scheme achieves flexible layering through dynamic threshold division, reducing the range of reconstruction accuracy fluctuations by approximately 25% under the same bandwidth conditions.

[0054] Through the above technical solutions, this application can dynamically optimize the transform domain and grouping structure based on data characteristics, achieving accurate residual coding of enhancement layer data while ensuring the reliability of base layer transmission. The learnable orthogonal transform improves the concentration of feature energy through adaptive basis functions, enabling base layer data to carry main information at a lower bit rate. The dynamic grouping mechanism based on energy thresholds effectively distinguishes feature channels of different importance, providing a physically clear basis for hierarchical coding. Layer-by-layer residual coding combined with low-pass filtering significantly reduces the transmission requirements of high-frequency components, maintaining stable reconstruction quality even in jittery network environments. Experiments show that this scheme can improve the convergence speed of gradient transmission models by approximately 18% under the same bandwidth conditions, while reducing the number of communication retransmissions by approximately 35%.

[0055] (III) Non-uniform quantization module During the intra-layer quantization stage, this embodiment employs an importance-based non-uniform quantization strategy. The basic idea is to allocate different bit widths based on the importance of the coefficients. For coefficients with high importance (determined by indicators such as energy, entropy, and gradient consistency), a higher bit width is allocated to reduce quantization error; while for coefficients with low importance, a lower bit width is used. This is combined with an µ-law adaptive non-uniform quantization method to improve effective resolution under low-bit conditions through logarithmic compression.

[0056] The common configuration is a bit width set of {2, 3, 4, 6, 8}. The difference in bit width between the important group and the ordinary group is generally controlled between 1 and 3 bits, which significantly improves the compression ratio while ensuring accuracy.

[0057] The formula for the importance score is: ; in, For the first The importance of each channel; For the first Energy of each channel; For the first The entropy value of each channel, The L2 norm of the gradient; For the first The gradient of each channel; This is a weighting coefficient, which is adjusted according to the characteristics of the task.

[0058] Then, for coefficients with high importance scores, a higher bit width is assigned to reduce quantization error; while for coefficients with low importance scores, a lower bit width is used; the bit width allocation formula is as follows: ; in, For the first The allocated bit width for each channel; , , These represent the high, medium, and low bit width settings, respectively. , These represent the high and low importance rating levels, respectively.

[0059] Simultaneously, by combining the µ-law adaptive non-uniform quantization method, logarithmic compression is used to improve the effective resolution of coefficients with low importance scores under low-bit conditions, as expressed in the following expression: ; in, Represents the input coefficients to be quantized The result after μ-law quantization; These are compression parameters; is a maximum value of the quantization coefficient; is a sign function, used to retain the sign of ; is a logarithmic function, which compresses the small value of the quantization coefficient in a logarithmic manner to obtain higher resolution.

[0060] The traditional method usually allocates bit width by a single index, such as only relying on gradient amplitude or energy distribution, which is difficult to fully reflect the importance of features. Fixed bit width quantization is prone to cause loss of small value information in low bit scenarios, and uniform quantization cannot adapt to the difference in dynamic range of features. The scheme realizes fine-grained bit width control through multi-factor comprehensive scoring, and enhances the resolution under low bit conditions by combining non-uniform quantization, overcoming the limitations of single index decision.

[0061] Through the above technical solutions, the application realizes dynamic matching of feature sensitivity and quantization accuracy, effectively reduces the transmission overhead of non-key data while ensuring high fidelity of key information. By μ-law quantization, the effective resolution under low bit conditions is improved, avoiding small amplitude features from being overwhelmed by noise in the compression process. The scheme can adapt to the differentiated needs of different tasks for feature sensitivity, and maintain the stability of model training under bandwidth limited conditions.

[0062] (Four) RTT adaptive scheduling module In terms of transmission scheduling, the embodiment designs an adaptive on-chip code rate adjustment mechanism based on round trip time (RTT). The system will monitor the RTT and jitter in real time, and dynamically adjust the fragmentation size P and the fragmentation rhythm accordingly. When the network jitter is large, the fragmentation is reduced and the confirmation interval is inserted to reduce the risk of timeout packet loss; while in the case of stable and sufficient bandwidth, the fragmentation is increased to reduce the overhead of the header, thereby improving the overall communication efficiency, the expression is as follows: The formula for real-time monitoring of RTT jitter is: ; Wherein, is the RTT jitter, used to measure the fluctuation of the round trip time; denotes the standard deviation; denotes the round trip time value obtained by the nth measurement.

[0063] ; Wherein, ; , denote the fragmentation size at , ; , denote the maximum value and the minimum value of the fragmentation size, respectively; , These represent the high and low thresholds for RTT jitter, respectively. This indicates the adjustment step size for the slice size, used to adjust the slice size in non-extreme jitter conditions.

[0064] The formula for the transmission interval is: ; in, This indicates the transmission interval, which is the time interval between two transmissions of a fragment; This represents the average round-trip time delay. , This is the adjustment coefficient.

[0065] Traditional methods typically employ fixed fragment sizes and transmission intervals, which cannot adapt to dynamically changing network environments. Existing bandwidth estimation-based adjustment methods only consider bandwidth capacity while ignoring the impact of latency jitter, easily leading to decreased transmission efficiency in scenarios with sudden packet loss. This solution introduces a quantitative evaluation mechanism for RTT jitter, combined with joint optimization of fragment size and transmission interval, achieving rapid response to network fluctuations. Compared to methods relying solely on bandwidth measurement, this solution more accurately balances transmission reliability and efficiency, particularly effectively reducing retransmission probability in high-jitter network environments.

[0066] Through the above technical solution, this application can intelligently adjust transmission parameters according to real-time network conditions, maximizing bandwidth utilization while ensuring reliable data transmission. When sudden network jitter occurs, the risk of data packet loss is effectively reduced by rapidly decreasing the fragment size and extending the transmission interval; after the network stabilizes, the fragment size is promptly increased to improve the effective payload ratio. This dynamic adjustment mechanism significantly improves the robustness of the distributed learning system in complex network environments, while avoiding resource waste or performance bottlenecks caused by fixed parameter settings.

[0067] (v) Error tracking and compensation module To avoid information redundancy and transmission waste, this embodiment specifies that the enhancement layer coding object is the residual on the reconstructed result of the base layer or lower layers. Simultaneously, in the error tracking compensation module, an error tracking table is established to record the cumulative reconstruction error of different channels and frequency bands. This error is used for the next round of dynamic bit width allocation and early stop decision, enabling the system to minimize invalid transmission and improve overall resource utilization while ensuring reconstruction accuracy. Specifically: First, calculate the residual coding of the enhancement layer for each layer, using the following formula: ; in, For the first Layer The residuals of the group; For the first enhanced layer transmission content of the group, for the first layer for the first reconstruction data of the group.

[0068] Then, the cumulative reconstruction error is calculated, which is composed of the last round error plus the sum of squares of the current round reconstruction error, and the formula is: ; wherein, is the cumulative reconstruction error of the first channel in the first round; is the enhanced layer transmission content of the first channel in the first round, is the reconstruction data of the first channel in the first round.

[0069] ; wherein, , respectively represent the bit width of the first channel in the first round; is the error threshold value; is the floor function.

[0070] When the maximum error of all channels is lower than the target error threshold value, stop the enhanced layer encoding to avoid invalid transmission, and the stop condition is: ; wherein, represents the maximum value in the cumulative error of all channels in the first t rounds; is the set target error threshold value.

[0071] The traditional method adopts a fixed bit width allocation strategy and lacks an error tracking mechanism, and cannot adjust the quantization precision according to the actual error distribution, which is easy to cause resource waste or insufficient precision. The existing residual coding technology usually only focuses on single round error compensation, and does not establish a long-term error tracking mechanism, which is difficult to effectively suppress the error accumulation effect. The present scheme realizes dynamic bit width adjustment based on historical error data by establishing a multi-round error tracking table, and introduces an adaptive stop condition, which significantly improves the accuracy of error compensation and the utilization rate of communication resources.

[0072] By the technical solution, the application can effectively inhibit the error accumulation phenomenon in distributed machine learning, and realize optimal matching of communication resources and reconstruction precision through a dynamic bit width allocation mechanism. The continuous updating of the error tracking table provides data support for the quantization strategy, enabling the system to accurately compensate for high-frequency error channels. The adaptive stop condition can avoid the continuous transmission of redundant data after the error meets the standard, thereby reducing communication overhead and improving system response speed. The scheme can still maintain stable error control capability in a complex network environment, providing reliable compression guarantee for distributed training.

[0073] (Six) Keyframe synchronization module In the long-term training process, errors may gradually accumulate due to residual transmission and cause drift. Therefore, the embodiment introduces a keyframe synchronization module, i.e., introduces a synchronization strategy, which periodically transmits keyframes to clear accumulated reconstruction errors and triggers the following conditions: The first case is that an independently encoded keyframe is sent every N training rounds, and the reconstruction does not depend on the historical residual, and the expression is: ; Wherein, represents the keyframe; represents the current round; represents the modulo operation; is the synchronization period.

[0074] The second case is that when there is a sudden increase in innovation energy, the transmission of the keyframe is triggered, i.e., the total energy of the current round is greater than the energy of the previous round , and the ratio exceeds the set energy surge threshold, the keyframe is triggered to re-establish the reference, and the expression is: ; Wherein, is the energy surge threshold.

[0075] The third case is that when the accumulated distortion exceeds the set threshold, i.e., the accumulated reconstruction error sum of the last rounds is calculated, and when the accumulated reconstruction error sum exceeds the set threshold , the transmission of the keyframe is triggered, and the expression is: ; Wherein, is the average reconstruction error of the first round; is the accumulated distortion threshold.

[0076] The fourth case is when there is a long-time packet loss, the error tracking state is cleared to ensure that the encoding end and the decoding end are re-synchronized to avoid the loss of accuracy caused by error drift.

[0077] The system realizes error correction and state resynchronization through a periodic key frame or anchor layer mechanism to prevent drift and precision degradation caused by residual accumulation.

[0078] The traditional method usually only relies on sending key frames at fixed intervals, lacking the ability to actively respond to data distribution mutations and network anomalies. For example, the periodic key frame in the prior art cannot cope with sudden changes in feature energy during training, while the present scheme can timely rebuild the reference when the data distribution changes significantly through the energy surge detection mechanism. In addition, the prior art lacks active monitoring of accumulated errors, while the present scheme actively triggers synchronization before the error accumulates to the critical point by calculating the error trend through a sliding window, avoiding precision loss.

[0079] Through the above technical solutions, the present application effectively solves the coding and decoding mismatch problem caused by error accumulation in the residual coding system, and can maintain the consistency of the encoder and decoder state in complex scenarios such as network fluctuations and data distribution changes. Through the multi-condition triggered synchronization mechanism, the communication efficiency is guaranteed while reducing invalid transmission, which can reduce more than 30% of redundant key frame transmission compared with the single periodic synchronization strategy. The scheme is particularly suitable for distributed learning scenarios with limited bandwidth and unstable network state, such as environments with frequent disconnections in mobile edge computing, and can significantly improve the robustness and model convergence stability of the system.

[0080] In summary, compared with the prior art, the present application has the following beneficial effects: The present application proposes a bandwidth-distortion adaptive multi-layer scalable feature compression framework through the integration of six technologies: rate-distortion joint control, hierarchical learnable orthogonal transform, non-uniform quantization, RTT adaptive scheduling, error tracking compensation and key frame synchronization. The framework makes full use of the energy concentration characteristics, spatial redundancy, frequency domain sparsity and temporal consistency of features, and has good adaptability and scalability, especially suitable for weak network, low bandwidth, high delay, strong jitter and other resource-limited environments. In practical applications, such as the architecture of a personalized federated learning system, the present application can significantly improve the robustness, convergence stability and communication efficiency of distributed training, meeting the actual needs of diverse scenarios.

[0081] Embodiment Two As shown in Figure 2 A bandwidth-distortion adaptive multi-layer scalable feature compression method includes the following steps: S1, collecting real-time network state, training stage and precision requirement; wherein the network state includes available bandwidth, round-trip time and jitter; S2, obtaining original data; wherein the original data includes original features sent by the client or original gradients sent by the server; S3, mapping the original data to a subspace with more concentrated energy through a learnable orthogonal transformation, and clustering and grouping the transformed feature channels, so that in the data transmission process, the base layer only carries the center coefficients and low-frequency components of each group, and the enhancement layer encodes the high-frequency details and deviation parts in each group layer by layer residual, gradually reconstructs the data from coarse to fine, and obtains the layered data that can be transmitted; S4, calculating the importance score based on the transformed layered data to dynamically allocate the bit width; S5, adjusting the slice size and slice interval according to the real-time network state; S6, calculating the cumulative reconstruction error of each round according to the residual coding of the enhancement layer and dynamically adjusting the bit width allocation of the next round, so as to obtain the compressed data; S7, dynamically deciding the total number of layers, the bit width allocation of each layer, and the slice size according to the real-time network state, the training stage, and the precision requirement, so as to dynamically adjust the input original data and minimize the reconstruction distortion of the original data and the reconstructed data; wherein the reconstructed data is obtained by decoding and reconstructing the compressed data by the decoder.

[0082] Embodiment three The third embodiment of the present application also provides a distributed machine learning system, which comprises a plurality of clients and a server, and a bandwidth-distortion adaptive multi-layer scalable feature compressor as described above between the client and the server; the client sends original features to the multi-layer scalable feature compressor for compression, and outputs compressed data; the server receives and aggregates the compressed data, and generates reconstructed data, i.e. global gradient, which is compressed by the multi-layer scalable feature compressor to generate compressed gradient and is sent to each client; each client updates the local model according to the compressed gradient, and based on the updated model, uses the original features to continue the next round of local training until the model converges.

[0083] Among them, the client refers to a terminal device deployed at the edge of a distributed network, which can be implemented by a smart terminal or an Internet of Things device equipped with computing power, and is used for executing local model training and generating original features to be transmitted. The server refers to a computing node deployed in the cloud or a central node, which can be implemented by a high-performance server cluster, and is responsible for aggregating the compressed data of multiple clients and generating global model parameters. The multi-layer scalable feature compressor refers to a communication module embedded between the client and the server, which can be implemented by a combination of dynamic encoder and decoder, and is used to dynamically adjust the compression strategy according to the real-time network state and the training stage, so as to balance the bandwidth and distortion.

[0084] Specifically, after the client generates the original features in the local training process, the original features are encoded by a multi-layer scalable feature compressor. In the encoding process, the base layer transmits low-frequency components and central coefficients, and the enhancement layer supplements high-frequency details layer by layer through residual coding. After the server receives the compressed data, the original features are reconstructed by the decoder and aggregated to generate global model parameters, which are then distributed to each client for model synchronization. After the client updates the local model parameters according to the reconstructed data, it continues to use the local dataset for the next round of training. During the entire training process, the compressor dynamically adjusts the number of layers and bit width allocation according to network bandwidth fluctuations, for example, increasing the number of enhancement layers to improve reconstruction accuracy when bandwidth is sufficient, and reducing the number of layers to prioritize the transmission of the base layer when bandwidth is limited. At the same time, through the error tracking mechanism, the reconstruction error is accumulated, and when the error exceeds the threshold, key frame synchronization is triggered to avoid model drift caused by accumulated error.

[0085] Compared with the prior art, the existing distributed learning system generally uses a fixed compression strategy, which cannot dynamically adjust the encoding parameters according to the network state, resulting in transmission delay or precision loss when the bandwidth fluctuates. However, the system can adaptively adjust the compression granularity under different network conditions through a multi-layer scalable coding structure combined with real-time network perception and dynamic bit width allocation. For example, high bit width encoding is used in the early training stage to ensure model convergence stability, and the bit width is reduced in the later training stage to improve transmission efficiency. In addition, the existing method lacks an error tracking mechanism, and long-term training may cause model mismatch between the client and the server. However, the system effectively suppresses the influence of accumulated error on model accuracy through periodic key frame synchronization and residual compensation mechanism.

[0086] Through the above technical solutions, the application solves the dynamic balance problem of communication efficiency and model accuracy in a distributed machine learning system. In a bandwidth fluctuating environment, the system ensures the priority transmission of key feature information through layered encoding and dynamic bit width allocation, avoiding data loss caused by network congestion. During model training, the adaptive compression strategy can adjust the reconstruction accuracy according to the training stage, for example, preserving more high-frequency details in the early convergence stage to accelerate convergence, and focusing on compression efficiency in the later stage to reduce communication overhead. At the same time, the error tracking and key frame synchronization mechanism effectively suppresses the error accumulation in long-term training, improving the accuracy of model updating and the robustness of the system.

[0087] Embodiment Four The fourth embodiment of the application also provides a computer readable storage medium, which stores computer readable instructions. When the computer readable instructions are executed by the processor of the device where the computer readable storage medium is located, the bandwidth distortion adaptive multi-layer scalable feature compression method as described above is realized.

[0088] The above merely provides the preferred embodiments of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles and technical scope of the present application shall fall into the scope of the present application.

Claims

1. A bandwidth-distortion adaptive multilayer scalable feature compressor, applied between the client and server sides of a distributed machine learning system, to compress the raw data acquired from the client or server and output compressed data, characterized in that, include: The system includes a rate-distortion joint controller, a hierarchical learnable orthogonal transform module, a non-uniform quantization module, an RTT adaptive scheduling module, and an error tracking compensation module. The rate-distortion joint controller is used to dynamically decide the total number of layers, the bit width allocation of each layer, and the fragment size based on the real-time network status, training stage, and accuracy requirements, so as to dynamically adjust the input raw data and minimize the reconstruction distortion between the raw data and the reconstructed data; wherein the reconstructed data is obtained by decoding and reconstructing the compressed data by the decoder at the receiving end; the network status includes available bandwidth, round-trip time, and jitter. The hierarchical learnable orthogonal transformation module is used to map the original data to a more energy-concentrated subspace and cluster the feature channels. During data transmission, the base layer only carries the center coefficients and low-frequency components of each group, while the enhancement layer encodes the high-frequency details and deviation parts of each group layer by layer, performing progressive data reconstruction from coarse to fine, to obtain transmittable hierarchical data. The non-uniform quantization module is used to calculate importance scores based on the transformed hierarchical data, so as to dynamically allocate bit width; The RTT adaptive scheduling module is used to dynamically adjust the fragment size and fragmentation interval according to the real-time network status. The error tracking and compensation module is used to calculate the cumulative reconstruction error of each round based on the residual encoding of the enhancement layer and dynamically adjust the bit width allocation of the next round to obtain compressed data.

2. The bandwidth distortion adaptive multilayer scalable feature compressor according to claim 1, characterized in that... The raw data includes: raw features obtained from the client, or global gradients obtained from the server; The rate-distortion joint controller includes a network state awareness submodule, a training phase awareness submodule, and an accuracy requirement submodule. The network state awareness submodule calculates the effective available bandwidth by monitoring available bandwidth, round-trip time, and jitter, using the following formula: ； in, Effective available bandwidth; For real-time monitoring of available bandwidth; RTT stands for Round Trip Time. For RTT jitter; This is a conservative coefficient used to address protocol overhead and network fluctuation redundancy. The training phase perception submodule is used to calculate the distortion weight coefficient based on the current training round and the total number of rounds, using the following formula: ； in, The distortion weight coefficient for the current training round t; Total number of rounds; , These represent the maximum and minimum distortion weight coefficients, respectively; The decay rate parameter is used to control the transition speed from high precision to high compression. The accuracy requirement submodule is used to minimize the weighted reconstruction distortion of the original data and the reconstructed data by jointly optimizing the total number of layers, the bit width of each layer, and the size of each fragment, based on the available bandwidth and distortion weight coefficient, under a preset latency budget constraint. The objective function formula for the joint optimization is: ； in, This represents the total number of layers; For the first Layer width; This refers to the size of the slice; Distortion due to weighted reconstruction; For the first Layer weights; The original data; For the first The reconstructed data corresponding to the layer; It is an L2 norm; This indicates a minimize operation; The constraints are that the total bit rate does not exceed the bandwidth-delay product, and the actual transmission time does not exceed the delay budget. The formula is: ； in, For the first Layer bitrate; Budget for delay; This refers to the actual transmission time. Indicates being bound by; Total number of layers The determination is based on available communication resources and is dynamically determined using the following formula: ； in, Fixed bitrate at base layer The average bit rate for a single enhancement layer; This indicates taking the maximum value; To round down; The optimal bit width is assigned to each layer based on the Lagrange multiplier method, and the formula is as follows: ； in, For the first Optimal bit width for the layer; For the first Layer weighted reconstruction distortion; For the first Layer in position width The predicted distortion is below; B is the candidate bit width set; The fragment size is dynamically adjusted based on the bandwidth-delay product and the number of layers, using the following formula: ； in, To adjust the fragment size; , These are the lower and upper bounds of the fragment size, respectively.

3. A bandwidth-distortion adaptive multilayer scalable feature compressor according to claim 2, characterized in that... It also includes an early stop decision mechanism, which assesses marginal benefits in real time and determines whether the distortion improvement relative to the bitrate overhead at the current layer falls below a set threshold. If it is estimated that completing the next layer of transmission will exceed the latency budget, the data transmission of subsequent enhancement layers should be stopped immediately to avoid wasting communication resources when the marginal benefit is extremely low. The expression is: ； in, , Representing the current layer The next layer Reconstruction distortion after transmission is complete; Indicates the elapsed transmission time; Indicates the estimated transmission time for the next layer; Budget for time delay.

4. A bandwidth-distortion adaptive multilayer scalable feature compressor according to claim 3, characterized in that... In the hierarchical learnable orthogonal transformation module, a learnable orthogonal transformation method is used to map the original data to a subspace with more concentrated energy. Then, an energy threshold-based partitioning method is used to cluster and group the feature channels within the subspace according to their entropy characteristics, as shown in the following expression: The formula for learnable orthogonal transformation is: ; in, The transformed data; The input is the raw data, where C is the number of channels, H is the height, and W is the width. It is a learnable orthogonal transformation matrix; , It is the identity matrix; The formula for classifying based on energy thresholds is: , ; in, For the first Energy of each channel; For the first Data after transformation of each channel; Represents the L2 norm; For the first The entropy value of each channel; Let be the normalized probability distribution of the channel coefficients; i represents the index of the element in the c-th channel. All channels were grouped into clusters based on their energy thresholds. There are several groups, and the formula is: ； in, For the first Group, containing energy in All channels within the interval; , The first The energy threshold of group k-1; Total number of groups; Indicates the current channel; After clustering, the mean of all channel data in each group is calculated, and the low-frequency component is extracted using a low-pass filter as the base layer transmission content. The base layer transmission content is represented as follows: ； in, Indicates the first The group's base layer transmits content; This indicates a low-pass filter to preserve key low-frequency information; To calculate the within-group centrality coefficient; For the first All channel data for the group; Subtract the previous data from all channels in each layer and group. The residuals from the cumulative reconstruction results of each layer are used as the content transmitted in the enhancement layer, and the residuals are encoded layer by layer. The expression is as follows: ； ； in, For the first Layer The group's enhancement layer transmits content; For the first Layer The group's enhancement layer transmits content; Indicates the 0th layer. The enhanced layer of the group transmits the content, namely the first The group's base layer transmits content.

5. A bandwidth-distortion adaptive multilayer scalable feature compressor according to claim 4, characterized in that... The formula for the importance score is: ； in, For the first The importance of each channel; For the first Energy of each channel; For the first The entropy value of each channel, The L2 norm of the gradient; For the first The gradient of each channel; These are weighting coefficients, adjusted according to task characteristics; Then, for coefficients with high importance scores, a higher bit width is assigned to reduce quantization error; while for coefficients with low importance scores, a lower bit width is used; the bit width allocation formula is as follows: ； in, For the first The allocated bit width for each channel; , , These represent the high, medium, and low bit width settings, respectively. , These represent the high and low importance rating levels, respectively. Simultaneously, by combining the µ-law adaptive non-uniform quantization method, logarithmic compression is used to improve the effective resolution of coefficients with low importance scores under low-bit conditions, as expressed in the following expression: ； in, Represents the input coefficients to be quantized The result after μ-law quantization; These are compression parameters; This represents the maximum value of the quantization coefficient. For symbolic functions, used to preserve The symbol; It is a logarithmic function, and by using logarithmic compression, the small values ​​of the coefficients to be quantized can obtain higher resolution.

6. A bandwidth-distortion adaptive multilayer scalable feature compressor according to claim 2, characterized in that, In the RTT adaptive scheduling module, the jitter level of the round-trip time (RTT) is monitored in real time, and the fragment size P and the fragmentation interval are dynamically adjusted. When network jitter is large, the fragment size is reduced and an acknowledgment interval is inserted to reduce the risk of timeout and packet loss. When bandwidth is stable and sufficient, the fragment size is increased to reduce header overhead and improve overall communication efficiency, as shown in the following expression: The formula for real-time monitoring of RTT jitter is: ; in, RTT jitter is used to measure the degree of fluctuation in round-trip time. Indicates standard deviation; This represents the round-trip time delay value obtained from the nth measurement; The formula for adaptively adjusting the fragment size is: ； in, ; , They represent , The size of the slice at any given moment; , These represent the maximum and minimum values ​​of the fragment size, respectively. , These represent the high and low thresholds for RTT jitter, respectively. This indicates the adjustment step size for the slice size, used to adjust the slice size in non-extreme jitter conditions; The formula for the transmission interval is: ； in, This indicates the transmission interval, which is the time interval between two transmissions of a fragment; This represents the average round-trip time delay. , This is the adjustment coefficient.

7. A bandwidth-distortion adaptive multilayer scalable feature compressor according to claim 4, characterized in that... In the error tracking compensation module, an error tracking table is established to record the cumulative reconstruction error of different channels and frequency bands, which will be used for the next round of dynamic bit width allocation and early stop decision, as follows: First, calculate the residual coding of the enhancement layer for each layer, using the following formula: ； in, For the first Layer The residuals of the group; For the first The group's enhancement layer transmits content. For the first Layer Group reconstructed data; Next, the cumulative reconstruction error is calculated, which consists of the sum of the squares of the errors from the previous round and the current round, as shown in the formula: ； in, For the first The first channel in the The cumulative reconstruction error of the wheel; For the first The first channel in the The enhanced layer of the wheel transmits content. For the first The first channel in the Reconstructed data for the wheel; The bit width is dynamically adjusted based on the cumulative reconstruction error, using the following formula: ； in, , They represent the first The first channel in the Wheel width; This is the error threshold; To round down; When the maximum error of all channels is lower than the target error threshold, the enhancement layer coding is stopped to avoid invalid transmission. The stopping condition is: ； in, This represents the maximum value of the cumulative error across all channels in round t; The target error threshold is set.

8. A bandwidth-distortion adaptive multilayer scalable feature compressor according to claim 7, characterized in that... It also includes a keyframe synchronization module, which introduces a synchronization strategy to clear accumulated reconstruction errors by periodically transmitting keyframes. The triggering conditions are as follows: In the first case, an independently encoded keyframe is sent every N training rounds, and reconstruction is performed without relying on historical residuals. The expression is: ； in, Indicates a keyframe; Indicates the current round; Indicates the modulo operation; For synchronization period; In the second scenario, a sudden surge in innovation energy triggers the transmission of keyframes, namely: the total energy of the current round. Compared to the previous round of energy When the ratio exceeds the set energy surge threshold, a keyframe is triggered to re-establish the baseline, as expressed by: ； in, The threshold for a sudden increase in energy; The third scenario occurs when the cumulative distortion exceeds a set threshold, i.e., when the most recent distortion is calculated. The cumulative reconstruction error of each round; when the cumulative reconstruction error exceeds a set threshold... The expression that triggers the transmission of the keyframe is: ； in, For the first The average reconstruction error of the wheel; This is the cumulative distortion threshold; In the fourth scenario, when packet loss occurs for an extended period, the error tracking state is cleared to ensure that the encoding and decoding ends are resynchronized, thus avoiding accuracy loss due to error drift.

9. A bandwidth-distortion adaptive multilayer scalable feature compression method, applied to a bandwidth-distortion adaptive multilayer scalable feature compressor as described in any one of claims 1-8, characterized in that, include: Collect real-time network status, training phase, and accuracy requirements; wherein, the network status includes available bandwidth, round-trip latency, and jitter; Obtain raw data; wherein, the raw data includes raw features sent by the client or raw gradients sent by the server; The original data is mapped to a more energy-concentrated subspace through a learnable orthogonal transformation, and the transformed feature channels are clustered and grouped so that during data transmission, the base layer only carries the center coefficients and low-frequency components of each group, while the enhancement layer encodes the high-frequency details and deviation parts of each group layer by layer with residuals, and performs progressive data reconstruction from coarse to fine to obtain transmittable layered data. Importance scores are calculated based on the transformed hierarchical data, with bit widths dynamically allocated. Adjust the fragment size and fragmentation interval according to the real-time network status; The cumulative reconstruction error for each round is calculated based on the residual coding of the enhancement layer, and the bit width allocation for the next round is dynamically adjusted to obtain compressed data. The total number of layers, the bit width allocation of each layer, and the size of the slices are dynamically determined based on the real-time network status, training stage, and accuracy requirements, so as to dynamically adjust the input original data and minimize the reconstruction distortion between the original data and the reconstructed data; wherein, the reconstructed data is obtained by the decoder decoding and reconstructing the compressed data.

10. A distributed machine learning system comprising several clients and servers, and a bandwidth-distortion adaptive multilayer scalable feature compressor as described in any one of claims 1-8, located between the clients and the servers; the clients send raw features to the multilayer scalable feature compressor for compression, outputting compressed data; the servers receive and aggregate the compressed data, and generate reconstructed data, i.e., global gradients, which are compressed by the multilayer scalable feature compressor to generate compressed gradients and sent to each client; each client updates its local model according to the compressed gradients, and based on the updated model, continues the next round of local training using the raw features until the model converges.

Images