Communication segmentation learning system and method for adaptive channel compression, and medium

Abstract

The invention provides a communication segmentation learning system and method for adaptive channel compression, and a medium, and relates to the technical field of segmentation learning and communication compression. The system comprises a plurality of clients, a server side and a channel compression device arranged between the clients and the server side, the channel compression device comprises a channel sensitivity modeling module, a rate distortion adaptive compression module and a cross-client fair coordination module. In a channel sensitivity modeling module, channel importance is dynamically evaluated through intermediate layer activation value fusion; differentiated quantization and compression strategies are designed on the basis of sensitivity scores in a rate-distortion self-adaptive compression module, so that key information is reserved while communication overhead is reduced; and meanwhile, the cross-client fair coordination module realizes balanced distribution of communication resources among multiple clients through a fairness regularization and dual optimization mechanism, so that the influence of excessive compression on global convergence is avoided. According to the method, the communication efficiency and the training stability of segmentation learning in a complex heterogeneous environment are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of data communication and distributed deep learning technology, and more specifically, to an adaptive channel compression communication segmentation learning system, method, and medium. Background Technology

[0002] With the rapid development of IoT devices, the amount of data generated at the network edge is exploding. According to predictions by IDC, the total global data volume will reach approximately 221 ZB by 2026. This trend has driven the widespread application of machine learning technology in many fields, such as autonomous driving, smart healthcare, and natural language processing. However, current mainstream centralized learning frameworks require data from terminal devices to be centralized on servers for processing. This approach not only poses serious privacy risks but also results in high communication overhead, making it difficult to meet practical deployment needs. To address these issues, federated learning has been proposed to allow devices to collaboratively train without leaving their local data sources. Nevertheless, with the continuous expansion of model size, especially the emergence of large models containing billions of parameters, resource-constrained edge devices struggle to handle the computational load, becoming a bottleneck for federated learning in edge scenarios.

[0003] To alleviate this problem, segmentation learning has been proposed as an alternative. Segmentation learning significantly reduces the computational burden on devices by layering the model and migrating most of the computational tasks to the server, performing only a small amount of forward propagation on the device side. However, with the increase in the number of participating devices, the amount of "fragmented data," such as intermediate layer activation values ​​and gradients, transmitted between devices and the server increases dramatically, forming a new communication bottleneck and severely impacting training efficiency. To address this challenge, various compression methods have been proposed to reduce the transmission requirements of fragmented data. For example, Eshratifar et al. proposed using an auxiliary neural network to learn a more compact activation representation; Zheng et al. proposed retaining the top k elements with the largest amplitudes in the activations and combining them with a small number of randomly sampled non-top-k elements for compression; and Oh et al. used standard deviation to discard low-variance features and quantize the remaining parts. While these methods reduce the amount of data transmission to some extent, they still have significant shortcomings.

[0004] Existing technologies generally employ a uniform compression strategy, failing to consider the varying information contributions of different channels during model training. In reality, the semantic information carried by different channels plays different roles during training; some channels may be crucial for task performance, while others may be redundant or even noisy. A uniform compression strategy can easily lead to over-compression of important channels or wasted bandwidth on useless channels, thus impacting the final model performance. Furthermore, these methods lack adaptive mechanisms and cannot adjust to the dynamic changes in channel importance during training. Since model training is an evolving process, the contributions of each channel may change at different stages; a fixed compression strategy struggles to adapt to this dynamism, further weakening the balance between communication efficiency and training accuracy.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This invention aims to provide an adaptive channel compression communication segmentation learning system, method, and medium to solve the problems of high communication overhead, lack of adaptability of compression strategies, and uneven distribution of communication resources in multi-client scenarios in existing segmentation learning frameworks.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: An adaptive channel compression communication segmentation learning system includes: Several client and server terminals, and a channel compression device located between the client and server terminals; The channel compression device includes a channel sensitivity modeling module, a rate-distortion adaptive compression module, and a cross-client fair coordination module. The client is used to receive gradient data sent by the server to update the client model, and to perform forward propagation based on local data to generate intermediate layer activation values ​​and upload them to the channel compression device. The channel sensitivity modeling module is used to obtain the intermediate layer activation values ​​of each client during the forward propagation process, and to perform correction and fusion through Fisher information, HSIC statistical dependency and consistency to evaluate the importance of each channel and obtain the sensitivity score of each channel. The rate-distortion adaptive compression module is used to design differentiated quantization and compression strategies based on the sensitivity score for compression, generate quantization activation values ​​and compression distortion metrics to model the communication bit rate, and send the quantization activation values, compression distortion metrics and communication bit rate to the server. The server receives quantized activation values, compression distortion metrics, and communication bitrates, performs forward propagation to generate prediction results and calculate loss, and generates gradients of activation values ​​through backpropagation to send to the client. It also dynamically coordinates the allocation of communication resources among clients in conjunction with a cross-client fair coordination module. The cross-client fair coordination module is used to achieve balanced resource allocation in multi-client scenarios by dynamically adjusting the communication budget and communication code rate among each client.

[0008] Preferably, the channel is a feature channel formed after preliminary processing of the raw data input to the client model; in the channel sensitivity modeling module, the sensitivity score of each channel is calculated based on the intermediate layer activation value combined with Fisher information, HSIC dependency, and consistency, specifically as follows: First, based on gradient-based Fisher information, the local sensitivity of the capture channel is obtained, resulting in a Fisher sensitivity score, expressed as: ; in, This represents the Fisher sensitivity score for channel c; Indicates the expected value; Represents the task loss function; This represents the activation value of channel c; This represents the gradient of the loss function with respect to the activation values ​​of channel c; Represents the L2 norm; Next, based on the Hilbert–Schmidt Independence Criterion (HSIC), the dependency between channel features and labels is measured using a kernel matrix to obtain the HSIC sensitivity score, expressed as: ; in, This represents the HSIC sensitivity score of channel c; n represents the total number of clients. The trace of a matrix is ​​calculated by summing the elements on its diagonal. The kernel matrix represents channel c and is used to capture the similarity between channel features; This represents a centered matrix to ensure the zero mean of the kernel matrix; The kernel matrix represents the tags and is used to calculate the dependencies between tags; A consistency factor is used to suppress cross-client variance, highlighting globally stable channels in multi-client scenarios and reducing bias caused by data heterogeneity. The resulting consistency sensitivity score is expressed as: ; ; in, The consistency sensitivity score represents channel c. This represents the weighted sensitivity score of channel c on client i, which combines the Fisher sensitivity score and the HSIC sensitivity score. Indicate the variance of client i; This represents a weighted hyperparameter used to balance the contributions of Fisher information and HSIC dependency; , These represent the Fisher sensitivity score and HSIC sensitivity score of channel c on client i, respectively. Then, the consistency sensitivity score, Fisher sensitivity score, and HSIC sensitivity score are weighted and fused, and an exponential moving average is used to control the balance between old and new information to reduce instability caused by batch fluctuations. The expression is: ; in, This represents the smoothness sensitivity score of channel c in round t; This represents the smoothness sensitivity score of channel c in round t-1; The smoothing factor representing the exponential moving average; Finally, the smoothness sensitivity score is normalized to maintain scale consistency across different rounds, resulting in the final sensitivity score, expressed as: ; in, This represents the sensitivity score of channel c after normalization, i.e., the final sensitivity score. Indicates the total number of channels. Represents a variable.

[0009] Preferably, in the rate-distortion adaptive compression module, each channel is assigned an independent quantization step size and zero point for quantization, resulting in differentiated quantization activation values, expressed as: ; in, This represents the quantization activation value of channel c; This represents the original activation value of channel c; Indicates the zero point of channel c; This indicates the rounding operation; This indicates the quantization step size of channel c; Then, based on the sensitivity score and combined with differentiated quantitative activation values, the compression error is dynamically constrained through a sensitivity-weighted distortion function to achieve on-demand allocation of communication resources. The expression is: ; in, This represents a measure of compression distortion, calculated by the original activation values. With quantized activation value The error between; Indicates the total number of channels; This represents the sensitivity score of channel c after normalization; This represents the original activation value of channel c; This represents the quantization activation value of channel c; This represents the L2 norm.

[0010] Preferably, the method further includes, when modeling the communication code rate, estimating the probability distribution of the quantization symbol based on the quantization activation value using a super-prior learned entropy model, the expression of which is: ; in, Indicates the quantized activation value The communication code rate; Represents the advanced prior model right Probability estimation; express The expected value.

[0011] Preferably, the server combines task loss, communication bit rate, and compression distortion metrics, and uses a Lagrangian objective function to jointly optimize the channel bit width and step size via gradient descent. The expression is: ; in, Represents the total loss function; Represents the task loss function; This represents the global model output from the server side. Indicates the true label; , The hyperparameters are represented by weights that balance task loss, communication bitrate, and distortion. Indicates the expected value; During training and optimization, the bit width is relaxed to a continuous variable to facilitate gradient optimization and is dynamically updated during iterations, as shown in the formula: ; ; in, , These represent the bit widths of channel c in rounds t and t+1, respectively. , Let represent the quantization step size of channel c in rounds t and t+1, respectively; , Indicates the learning rate; , These represent the gradients of the total loss function L with respect to the bit width and step size, respectively.

[0012] Preferably, in the cross-client fair coordination module, a fairness regularization term is introduced to constrain the difference between the communication bitrate of each client and the global average, as shown in the formula: ; in, , They represent the client respectively. , The actual communication code rate; Indicates the client The actual communication code rate difference; Indicates the total number of clients; This indicates the total number of clients that have communicated in the current round; Represents a variable.

[0013] Preferably, the cross-client fair coordination module further includes setting a dual variable for each client on the server side, and optimizing the objective based on the dual variable using a dual optimization mechanism to measure the difference between the communication budget and the communication code rate, as shown in the formula: ; ; in, , Representing the client in rounds t and t+1 respectively. The dual variable; This represents the learning rate for dual optimization, used to control the magnitude of dual variable updates; Indicates client Communication budget; Integers are retained, and negative numbers are set to zero; Represents the client in round t. The actual communication code rate; Indicates the client Optimize the regularized weights during the (t+1)th round of training; This represents the baseline weight, an initial value shared by all clients; This represents the amplification factor, used to control the strength of the influence of the dual variable on the regularization weights.

[0014] Preferably, in the cross-client fair coordination module, the communication budget of each client is dynamically adjusted according to the global situation so that the system gradually converges to a reasonable communication level. The expression is: ; in, Indicates client The communication budget in round t; This indicates the adjusted communication budget, i.e., the client's... The communication budget in round t+1; Indicates the smoothing coefficient; Indicates the learning rate; Indicates client Communication budget; The gradient representing the fairness regularization; This indicates the difference in actual communication code rate.

[0015] This invention also provides an adaptive channel compression communication segmentation learning method, applied to a channel compression device between a client and a server, comprising: The intermediate layer activation values ​​of each client during the forward propagation process are obtained and corrected and fused using Fisher information, HSIC statistical dependency and consistency to evaluate the importance of each channel and obtain the sensitivity score of each channel. Based on the sensitivity score, a differentiated quantization and compression strategy is designed for compression, generating quantization activation values ​​and compression distortion metrics to model the communication bit rate, and then sending the quantization activation values, compression distortion metrics, and communication bit rate to the server. When the gradient for generating activation values ​​is sent down to the client via backpropagation on the server side, the communication budget and communication bitrate between each client are dynamically adjusted to achieve balanced resource allocation in multi-client scenarios.

[0016] The present invention also provides an adaptive channel compression communication segmentation learning device, including a processor and a memory, wherein the memory stores a computer program that can be executed by the processor to implement the adaptive channel compression communication segmentation learning method described above.

[0017] The present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device on which the computer-readable storage medium resides, implement the communication segmentation learning method for adaptive channel compression as described above.

[0018] In summary, compared with the prior art, the present invention has the following beneficial effects: This invention introduces three major modules—channel sensitivity modeling, rate-distortion adaptive compression, and cross-client fair coordination—to form a closed-loop optimization structure, thereby reducing communication costs while maintaining model performance.

[0019] Specifically, in the channel sensitivity modeling module, this invention achieves accurate assessment of channel importance by fusing multi-source signals through Fisher information, HSIC statistical dependence and consistency, and combining it with a time smoothing mechanism, effectively mitigating interference caused by data heterogeneity and training fluctuations.

[0020] In the rate-distortion adaptive compression module, differentiated quantization and compression strategies are designed based on sensitivity scores, enabling high-importance channels to obtain more bit resources and low-importance channels to be compressed at a lower cost, thereby reducing the overall communication volume while preserving key information to the greatest extent.

[0021] In the cross-client fair coordination module, the communication budget of each client is dynamically adjusted through fairness regularization and dual optimization mechanism to avoid some clients affecting the convergence of the global model due to excessive compression.

[0022] The three modules work together to form a closed-loop optimization structure, which significantly enhances the convergence speed and robustness of distributed segmentation learning, enabling the method to maintain stable performance in complex heterogeneous environments. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the framework of an adaptive channel compression communication segmentation learning system provided in Embodiment 1.

[0025] Figure 2 This is a flowchart of a communication segmentation learning method for adaptive channel compression provided in Example 2.

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0028] Example 1 Embodiment 1 of the present invention provides an adaptive channel compression communication segmentation learning method, which can be implemented by an adaptive channel compression communication segmentation learning device (hereinafter referred to as the learning device), and in particular, executed by one or more processors within the learning device.

[0029] In this embodiment, the learning device may be an electronic device equipped with a processor, the processor having a computer program for the adaptive channel compression communication segmentation learning method and the computer program being executable, such as a computer, smartphone, smart tablet, workstation, etc., without limitation.

[0030] In this embodiment, the essence of segmentation learning is to break down a complete machine learning model (such as a neural network) into layers and deploy them on the client and server sides respectively. Together, they constitute a "complete model" capable of performing end-to-end tasks. Client-side model: Responsible for the "first half" of the model (usually feature extraction layers, such as convolutional layers and embedding layers), deployed locally on the client, only processing local private data, and not involved in the final prediction / decision logic.

[0031] The server-side global model is responsible for the "second half" of the model (usually the task processing layer, such as the fully connected layer, attention layer, and output layer). It is deployed on the server side, receives intermediate features from the client, completes the final prediction, and calculates the loss.

[0032] In distributed deep learning, the high communication overhead between the client and server is a major bottleneck limiting its application in resource-constrained environments (such as low-bandwidth or high-latency networks). Feature gradients typically have high dimensionality and high precision, resulting in massive data transmission volumes, significantly increasing communication latency and energy consumption, especially when training large-scale models. Existing compression methods, such as simple sparsification or fixed quantization, often struggle to balance compression ratio and model performance, easily leading to information loss or decreased convergence speed. This invention aims to address the problems of high communication costs and lack of flexibility in compression strategies in current segmentation learning. It proposes a fragmented data compression communication segmentation learning system with channel importance recognition capabilities and an adaptive compression mechanism. This system significantly reduces communication burden while maximizing the preservation of information relevant to training performance, thereby achieving efficient, stable, and communication-friendly distributed deep learning.

[0033] like Figure 1 As shown, an adaptive channel compression communication segmentation learning system includes several client and server ends, as well as a channel compression device located between the client and server ends.

[0034] The channel compression device includes a Channel Sensitivity Modeling (CSM) module, a Rate-Distortion Adaptive Compression (RDAC) module, and a Fairness-Aware Coordination (FAC) module. These three parts form a closed loop in each training round: CSM identifies channel importance, RDAC performs adaptive quantization and compression based on this, and FAC coordinates the communication budget among multiple clients and feeds it back to the next round of optimization training, thereby achieving efficient, fair, and stable training.

[0035] On the client side, gradient data sent by the server is received to update the local client model, and forward propagation is performed based on the local data to generate intermediate layer activation values, which are then uploaded to the channel compression device.

[0036] Specifically, the client loads local private data, inputs it into the local model in batches, and performs forward propagation. After passing through the feature extraction layer of the client model, the intermediate layer activation values ​​corresponding to that batch of data are generated.

[0037] In segmentation learning, a channel refers to the feature path formed after the raw data input to the client model has undergone preliminary processing (e.g., applying Gaussian filtering to reduce noise, followed by nonlinear transformation using the ReLU activation function, resulting in a feature vector of dimension d as the feature channel). Essentially, it's the dimension or path in the model used to transmit and represent specific signal features, serving as the core carrier connecting the raw data and model parameters. The importance of channels varies significantly, and different channels contribute unevenly to task performance. Applying the same strategy to all channels during the compression stage can easily lead to the loss of crucial information, thus affecting the model's final convergence accuracy.

[0038] To address this issue, this embodiment proposes a Channel Sensitivity Modeling (CSM) module. Based on intermediate layer activation values, Fisher information, HSIC statistical dependency, and consistency sensitivity scores are calculated, and these scores are weighted and fused to generate a reliable weighted score for each channel.

[0039] Specifically, the channel sensitivity modeling module is used to obtain the intermediate layer activation values ​​of each client during the forward propagation process, and to perform correction and fusion through Fisher information, HSIC statistical dependency (such as Gaussian kernel) and consistency to evaluate the importance of each channel and obtain the sensitivity score of each channel.

[0040] In the channel sensitivity modeling module, input the intermediate layer activation values ​​of all clients and the server-side task labels (for HSIC calculation); output the final sensitivity scores of each channel in this round.

[0041] The intermediate layer activation values ​​are high-dimensional feature matrices generated by the client model after processing local data. They are the only "data interaction carrier" between the client and the server, and are also the core design of segmentation learning for "data privacy protection" and "communication efficiency optimization". The client inputs local raw data (such as medical images and sensor signals) into the local model. After processing by the first half of the network (such as convolutional layers and the ReLU activation function), the output feature map is the "intermediate layer activation value".

[0042] Specifically, in the channel sensitivity modeling module, the sensitivity score for each channel is calculated based on the intermediate layer activation values ​​combined with Fisher information, HSIC dependency, and consistency, as follows: First, based on gradient-based Fisher information, the local sensitivity of the capture channel is obtained, resulting in a Fisher sensitivity score, expressed as: ; in, This represents the Fisher sensitivity score for channel c; Indicates the expected value; Represents the task loss function; This represents the activation value of channel c; This represents the gradient of the loss function with respect to the activation values ​​of channel c; L2 norm represents the Euclidean norm of a vector.

[0043] In this embodiment, Fisher information is a statistical indicator that measures the amount of information the data provides for estimating model parameters. Essentially, it reflects the degree of influence of small changes in parameters on the data distribution. The more Fisher information contained in the data, the more accurate the parameter estimation (the smaller the variance). It is the core theoretical basis for maximum likelihood estimation, Bayesian inference, and model optimization.

[0044] If a channel has a large gradient magnitude during backpropagation, it indicates that it contributes significantly to the optimization of the current task, and therefore needs to be allocated more bit resources during compression.

[0045] However, relying solely on gradient information cannot fully characterize the value of a channel, because gradients primarily reflect instantaneous sensitivity rather than the statistical relationship between channels and labels. To address this deficiency, we further introduce the Hilbert–Schmidt Independence Criterion (HSIC). Based on HSIC, the dependency between channel features and labels is measured using a kernel matrix, yielding the HSIC sensitivity score, expressed as: ; in, This represents the HSIC sensitivity score of channel c; n represents the total number of clients. The trace of a matrix is ​​calculated by summing the elements on its diagonal. The kernel matrix representing channel c is used to capture the similarity between channel features (e.g., calculated using a Gaussian radial basis function (RBF) kernel). This represents a centered matrix to ensure the zero mean of the kernel matrix; The kernel matrix represents the labels and is used to calculate the dependencies between labels (it can be calculated using a linear kernel based on the one-hot encoded label vectors). If a channel is highly correlated with a label, it means that it contains richer discriminative information.

[0046] In this embodiment, the Hilbert–Schmidt Independence Criterion (HSIC) is a statistical tool based on the Reproducing Kernel Hilbert Space (RKHS). Its core function is to quantify the degree of independence between two random variables. The closer the value is to 0, the stronger the independence of the variables; the larger the value, the stronger the correlation (including linear and nonlinear correlations). It is particularly good at handling correlation analysis of high-dimensional and nonlinear data.

[0047] In multi-client scenarios, the local data distribution across different devices can lead to inconsistent channel sensitivity estimation results. Directly using local estimates can easily result in instability and noise. Therefore, a consistency factor is introduced to suppress cross-client variance, thereby highlighting channels that are more globally stable. Specifically, a consistency factor is used to suppress cross-client variance, highlighting globally stable channels in multi-client scenarios and reducing bias caused by data heterogeneity, resulting in a consistency sensitivity score, expressed as: ; ; in, The consistency sensitivity score represents channel c. This represents the weighted sensitivity score of channel c on client i, which combines the Fisher sensitivity score and the HSIC sensitivity score. Indicate the variance of client i; This represents a weighted hyperparameter used to balance the contributions of Fisher information and HSIC dependency; , These represent the Fisher sensitivity score and HSIC sensitivity score of channel c on client i, respectively.

[0048] This step can significantly reduce bias caused by data heterogeneity and improve overall robustness.

[0049] Then, the consistency sensitivity score, Fisher sensitivity score, and HSIC sensitivity score are weighted and fused, and an exponential moving average is used to control the balance between old and new information to reduce instability caused by batch fluctuations. The expression is: ; in, This represents the smoothness sensitivity score of channel c in round t; This represents the smoothness sensitivity score of channel c in round t-1; This represents the smoothing factor for exponential moving averages.

[0050] The resulting sensitivity not only reflects the importance of the channel, but also maintains stability and consistency during training.

[0051] Finally, to ensure numerical comparability, the smoothness sensitivity score is normalized to maintain scale consistency across different rounds, resulting in the final sensitivity score, expressed as: ; in, This represents the sensitivity score of channel c after normalization, i.e., the final sensitivity score. Indicates the total number of channels. Represents a variable.

[0052] After obtaining the sensitivity scores, the next task is to use these scores to design an adaptive compression mechanism. Intuitively, channels with high sensitivity carry more critical information and therefore should be given higher quantization precision; while channels with low sensitivity can be compressed more coarsely without significantly affecting performance. To achieve this goal, Rate-Distortion Adaptive Compression (RDAC) is proposed.

[0053] The rate-distortion adaptive compression module is used to design differentiated quantization and compression strategies based on the sensitivity score, generate quantization activation values ​​and compression distortion metrics to model the communication bit rate, and send the compression distortion metrics and communication bit rate to the server.

[0054] Specifically, in the rate-distortion adaptive compression module, the intermediate layer activation value and channel sensitivity score of each client are input, and the quantization activation value, compression distortion metric and communication bit rate are output.

[0055] Furthermore, each channel is assigned an independent quantization step size and zero point for quantization, resulting in differentiated quantization activation values, expressed as: ; in, This represents the quantization activation value of channel c; This represents the original activation value of channel c; Indicates the zero point of channel c; This indicates the rounding operation; This indicates the quantization step size of channel c.

[0056] High-sensitivity channels use smaller step sizes to ensure accuracy, while low-sensitivity channels use larger step sizes to reduce bit consumption. This differentiated design avoids a "one-size-fits-all" quantization approach, making the overall compression strategy more flexible.

[0057] Then, based on the sensitivity score and combined with differentiated quantitative activation values, the compression error is dynamically constrained through a sensitivity-weighted distortion function to achieve on-demand allocation of communication resources. The expression is: ; in, This represents a measure of compression distortion, calculated by the original activation values. With quantized activation value The error between; Indicates the total number of channels; This represents the sensitivity score of channel c after normalization; This represents the original activation value of channel c; This represents the quantization activation value of channel c; This represents the L2 norm.

[0058] While reducing distortion, we also need to accurately model the communication code rate. To this end, based on the quantization activation values, a super-prior learned entropy model is used to estimate the probability distribution of the quantization symbols to model the communication code rate, expressed as: ; in, Indicates the quantized activation value The communication code rate; Represents the advanced prior model right Probability estimation; express The expected value.

[0059] This model can provide a differentiable bitrate approximation in end-to-end training, thus effectively balancing communication overhead and task accuracy.

[0060] To balance task loss, bit rate, and distortion, we combine them into a unified Lagrangian objective function and jointly optimize the channel width and step size using gradient descent.

[0061] Specifically, the server receives the quantized activation value, compression distortion metric, and communication bit rate, performs forward propagation to generate prediction results and calculate the loss, and generates the gradient of the activation value through backpropagation and sends it to the client. At the same time, it dynamically coordinates the allocation of communication resources among clients in conjunction with the cross-client fair coordination module.

[0062] During the forward propagation process, the server inputs the quantized activation values ​​of each client into the global model to perform forward propagation, generating prediction results, which are then combined with the real labels. Calculate task loss Then, combining task loss, communication bit rate, and compression distortion metrics, a Lagrangian objective function is used to jointly optimize the channel bit width and step size via gradient descent. The expression is: ; in, Represents the total loss function; Represents the task loss function; This represents the global model output from the server side. Indicates the true label; , The hyperparameters are represented by weights that balance task loss, communication bitrate, and distortion. This represents the expected value.

[0063] During training and optimization, the bit width is relaxed to a continuous variable to facilitate gradient optimization and is dynamically updated during iterations, as shown in the formula: ; ; in, , These represent the bit widths of channel c in rounds t and t+1, respectively. , Let represent the quantization step size of channel c in rounds t and t+1, respectively; , Indicates the learning rate; , These represent the gradients of the total loss function L with respect to the bit width and step size, respectively.

[0064] During backpropagation, the gradient of the total loss function with respect to the quantized activation values ​​is calculated and used for client model updates.

[0065] In multi-client segmentation learning scenarios, differences in bandwidth and data distribution across different devices can lead to imbalances in compression strategies. If some clients are forced to over-compress, the quality of their model updates will degrade, thus affecting the convergence of the global model. To address this issue, we designed a cross-client fair coordination (FAC) module to ensure the overall balance of training.

[0066] The cross-client fair coordination module is used to achieve balanced resource allocation in multi-client scenarios by dynamically adjusting the communication budget and communication bitrate among each client.

[0067] Specifically, in the cross-client fair coordination module, a fairness regularization term is introduced to constrain the difference between each client's communication bitrate and the global average, ensuring that resource allocation is not excessively skewed. The formula is as follows: ; in, , They represent the client respectively. , The actual communication code rate; Indicates client The actual communication code rate difference; Indicates the total number of clients; This indicates the total number of clients that have communicated in the current round; Represents a variable.

[0068] Relying solely on regularization terms is insufficient for dynamic adaptation; therefore, a dual optimization mechanism is introduced in FAC. The server sets a dual variable for each client and, based on this dual variable, uses the dual optimization mechanism to optimize the objective, measuring the difference between the communication budget and the communication bit rate. The formula is: ; ; in, , Representing the client in rounds t and t+1 respectively. The dual variable; This represents the learning rate for dual optimization, used to control the magnitude of dual variable updates; Indicates client Communication budget; Integers are retained, and negative numbers are set to zero; Represents the client in round t. The actual communication code rate; Indicates client Optimize the regularized weights during the (t+1)th round of training; This represents the baseline weight, an initial value shared by all clients; This represents the amplification factor, used to control the strength of the influence of the dual variable on the regularization weights.

[0069] In this way, clients that exceed the budget will be penalized more severely in subsequent rounds, while clients that stay below the budget will have more freedom.

[0070] The cross-client fair coordination module (FAC) also includes dynamically adjusting the communication budget of each client based on the global situation, so that the system gradually converges to a reasonable communication level. The expression is: ; in, Indicates client The communication budget in round t; This indicates the adjusted communication budget, i.e., the client's... The communication budget in round t+1; Indicates the smoothing coefficient; Indicates the learning rate; Indicates client Communication budget; The gradient representing the fairness regularization; This indicates the difference in actual communication code rate.

[0071] This mechanism ensures overall efficiency while preventing individual clients from being unfairly restricted for extended periods.

[0072] During overall operation, the three main modules of the channel compression device form a closed-loop optimization structure. The channel sensitivity modeling module generates sensitivity scores, the rate-distortion adaptive compression module designs compression strategies based on the sensitivity scores and generates compressed intermediate layer activation values ​​and gradient data, and the cross-client fair coordination module dynamically adjusts the communication budget and achieves balanced global resource allocation through communication budget and bitrate feedback information. The modules interact with each other via data flow to ensure the system maintains stable performance in complex heterogeneous environments.

[0073] In practical applications, this communication segmentation learning system is suitable for distributed deep learning scenarios, especially in resource-constrained edge computing environments. For example, in smart home networks, multiple IoT devices need to collaborate to complete image classification tasks, but the bandwidth between devices is limited and their computing capabilities vary significantly. In this case, this communication segmentation learning system can significantly reduce communication overhead while ensuring model performance.

[0074] Taking a smart home network as an example, suppose multiple camera devices need to collaboratively complete an object detection task. Due to the varying data quality collected by different devices and limited network bandwidth, traditional segmentation learning methods are prone to communication bottlenecks and model performance degradation. Using the framework of this invention, the channel sensitivity modeling module can accurately identify the importance of key channels, and the rate-distortion adaptive compression module designs differentiated compression strategies based on sensitivity scores, thereby reducing communication overhead while retaining important information. The cross-client fair coordination module ensures that each device can complete the training task within a reasonable range through dynamic budget adjustment and dual optimization mechanisms, avoiding model performance degradation caused by over-compression. Compared to existing technologies, this framework achieves higher model accuracy and faster convergence speed under the same communication budget, while significantly improving the robustness and fairness of the system.

[0075] In summary, compared with the prior art, the present invention has the following beneficial effects: This invention effectively solves the problems of high communication costs, lack of flexibility in compression strategies, and uneven resource allocation in multi-client scenarios by introducing three major modules: channel sensitivity modeling, rate-distortion adaptive compression, and cross-client fair coordination. It has broad application prospects and significant technical advantages.

[0076] Example 2 like Figure 2 As shown, the second embodiment of the present invention also provides an adaptive channel compression communication segmentation learning method, applied to a channel compression device between a client and a server, comprising: The intermediate layer activation values ​​of each client during the forward propagation process are obtained and corrected and fused using Fisher information, HSIC statistical dependency and consistency to evaluate the importance of each channel and obtain the sensitivity score of each channel. Based on the sensitivity score, a differentiated quantization and compression strategy is designed for compression, generating quantization activation values ​​and compression distortion metrics to model the communication bit rate, and then sending the quantization activation values, compression distortion metrics, and communication bit rate to the server. When the gradient for generating activation values ​​is sent down to the client via backpropagation on the server side, the communication budget and communication bitrate between each client are dynamically adjusted to achieve balanced resource allocation in multi-client scenarios.

[0077] Example 3 The third embodiment of the present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device where the computer-readable storage medium is located, implement the communication segmentation learning method for adaptive channel compression as described above.

[0078] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0079] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0080] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0081] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0082] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0083] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0084] The use of "first" and "second" in the embodiments is merely to distinguish similar objects and does not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A communication partition learning system with adaptive channel compression, characterized by, The method comprises the following steps: a plurality of clients and a server, and a channel compression device arranged between the clients and the server; wherein the channel compression device comprises a channel sensitivity modeling module, a rate-distortion adaptive compression module, and a cross-client fairness coordination module; the client is configured to receive gradient data issued by the server to update the client model, and perform forward propagation based on local data to generate intermediate layer activation values uploaded to the channel compression device; the channel sensitivity modeling module is configured to obtain the intermediate layer activation values of each client during the execution of the forward propagation, and correct and fuse the Fisher information, HSIC statistical dependence, and consistency to evaluate the importance of each channel and obtain the sensitivity score of each channel; the rate-distortion adaptive compression module is configured to design a differentiated quantization and compression strategy based on the sensitivity score to perform compression, generate quantized activation values and compression distortion metrics, model the communication code rate, and send the quantized activation values, compression distortion metrics, and communication code rate to the server; the server is configured to receive the quantized activation values, compression distortion metrics, and communication code rate to perform forward propagation to generate a prediction result and calculate a loss, and through back propagation, generate gradient of the activation values and issue the gradient to the client, while the cross-client fairness coordination module dynamically coordinates the communication resource allocation of each client; the cross-client fairness coordination module is configured to dynamically adjust the communication budget and communication code rate between each client to achieve resource allocation balance in a multi-client scenario.

2. The communication partition learning system with adaptive channel compression of claim 1, wherein The channel is a feature channel formed after the original data of the input client model is preliminarily processed; in the channel sensitivity modeling module, the sensitivity score of each channel is calculated based on the intermediate layer activation values in combination with the Fisher information, HSIC dependence, and consistency, specifically as follows: First, the local sensitivity of the channel is captured based on the Fisher information of the gradient to obtain a fisher sensitivity score, expressed as: ； wherein, represents a fisher sensitivity score for channel c; represents an expected value; represents a task loss function; represents an activation value for channel c; represents a gradient of the loss function with respect to the activation value for channel c; represents an L2 norm; Then, the dependence between the channel features and the label is measured by a kernel matrix based on the Hilbert-Schmidt independence criterion HSIC to obtain an HSIC sensitivity score, expressed as: ； where, HSIC sensitivity score for channel c; n represents the total number of clients; denotes the trace of a matrix, i.e., the sum of the diagonal elements of a matrix; denotes the kernel matrix for channel c, which is used to capture the similarity between channel features; denotes the centered matrix to ensure zero mean of the kernel matrix; denotes the kernel matrix for labels, which is used to compute the dependency between labels; A consistency factor is used to suppress the cross-client variance to highlight the globally stable channel in the multi-client scenario and reduce the bias caused by data heterogeneity to obtain a consistency sensitivity score, expressed as: ； ； wherein, denotes the consistency sensitivity score of channel c; denotes the weighted sensitivity score of channel c on client i, combining the fisher sensitivity score and the HSIC sensitivity score; denotes the variance of client i; denotes the weighted hyper-parameter to balance the contribution of fisher information and HSIC dependence; , denotes the fisher sensitivity score, the HSIC sensitivity score of channel c on client i, respectively; The consistency sensitivity score, fisher sensitivity score, and HSIC sensitivity score are then weighted and fused, and the balance between new and old information is controlled through an exponential moving average to reduce the instability caused by batch fluctuations, expressed as: ； wherein, denotes the smoothed sensitivity score of channel c at round t; denotes the smoothed sensitivity score of channel c at round t-1; denotes the smoothing factor of the exponential moving average; Finally, the smoothed sensitivity score is normalized to keep the scale consistent between different rounds to obtain the final sensitivity score, expressed as: ； wherein, represents the normalized sensitivity score of channel c, i.e. the final sensitivity score; represents the total number of channels, represents the variable.

3. The communication partition learning system with adaptive channel compression of claim 2, wherein In the rate-distortion adaptive compression module, each channel is assigned an independent quantization step and zero point for quantization to obtain differentiated quantized activation values, expressed as: ； wherein, represents a quantized activation value for channel c; represents an original activation value for channel c; represents a zero point for channel c; represents a rounding operation; represents a quantization step size for channel c; Then, based on the sensitivity score and the differentiated quantized activation values, the compression error is dynamically constrained by a sensitivity weighted distortion function to achieve on-demand allocation of communication resources, expressed as: ； wherein, represents a compression distortion measure, computed by calculating the error between the original activation values and the quantized activation values ; represents the total number of channels; represents the sensitivity score of the normalized channel c; represents the original activation value of channel c; represents the quantized activation value of channel c; represents the L2 norm.

4. The communication partition learning system with adaptive channel compression of claim 1, wherein In modeling the communication code rate, based on the quantized activation value, a super-prior learning entropy model is used to estimate the probability distribution of the quantized symbol, expressed as: ； wherein, denotes a quantized activation value a communication code rate; denotes a super prior model a probability estimate for denotes an expected value.​ 5. The communication partition learning system of claim 4, wherein In addition, the server combines the task loss, the communication code rate and the compression distortion metric to jointly optimize the channel bit width and the step size by gradient descent method using a Lagrange objective function, expressed as: ； wherein, represents the total loss function; represents the task loss function; represents the global model output at the server side; represents the true label; , represents a hyperparameter balancing the weights between the task loss, the communication code rate, and the distortion; represents the expected value; In the training optimization process, the bit width is relaxed as a continuous variable for gradient optimization and is dynamically updated in iterations, expressed as: ； ； wherein, , respectively represent the bit-width of the channel c in the t, t+1 round; , respectively represent the quantization step of the channel c in the t, t+1 round; , represents the learning rate; , respectively represent the gradient of the total loss function L with respect to the bit-width, step.

6. The communication partition learning system with adaptive channel compression of claim 5, wherein In the cross-client fairness coordination module, a fairness regularization term is introduced to constrain the difference between the communication code rate of each client and the global average, expressed as: ； wherein, , denote the actual communication code rates of the clients , , respectively; denote the actual communication code rate difference of the clients ; denote the total number of clients; denote the total number of clients that have communicated in the current round; denote variables.

7. The communication partition learning system of claim 6, wherein In the cross-client fairness coordination module, the server sets a dual variable for each client, and based on the dual variable, a dual optimization mechanism is used to optimize the objective to measure the difference between the communication budget and the communication code rate, expressed as: ； ； where, , denote the dual variables of the t, t+1th round of clients , respectively; denote the learning rate of the dual optimization, used to control the magnitude of the dual variable update; denote the communication budget of the client ; denote the integer reservation, negative numbers are zeroed out; denote the actual communication code rate of the tth round of clients . denotes the client regularization weight in the t+1th round of optimization training; denotes the benchmark weight, the initial value shared by all clients; denotes the amplification factor, used to control the influence strength of the dual variable on the regularization weight.

8. The communication partition learning system with adaptive channel compression of claim 7, wherein In the cross-client fairness coordination module, the communication budget of each client is dynamically adjusted according to the global situation to gradually converge the system to a reasonable communication level, expressed as: ； where, denotes the communication budget of client in the t-th round; denotes the adjusted communication budget, i.e., the communication budget of client in the t+1-th round; denotes the smoothing coefficient; denotes the learning rate; denotes the communication budget of client ; denotes the fairness-regularized gradient; denotes the actual communication rate difference. 9.A method of communication partition learning for adaptive channel compression, applied to a channel compression device between a client and a server, characterized in that, It includes: Obtain the intermediate layer activation value of each client in the forward propagation process, and fuse it by modifying the Fisher information, HSIC statistical dependence and consistency to evaluate the importance of each channel and obtain the sensitivity score of each channel; Based on the sensitivity score, design a differentiated quantization and compression strategy for compression to generate quantized activation values and compression distortion metrics to model the communication code rate, and send the quantized activation values, compression distortion metrics and communication code rate to the server side; When the server side generates the gradient of the activation value in the back propagation and sends it to the client side, the communication budget and the communication code rate among the clients are dynamically adjusted to achieve resource allocation balance in the multi-client scenario.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by the processor of the device where the computer readable storage medium is located to realize the adaptive channel compression communication segmentation learning method of claim 9.

Images