An optimization method for communication minimization maximization problem suitable for edge devices
By optimizing the quantization strategy through adaptive simulated annealing and selecting an appropriate bit width for edge devices, the problem of high communication overhead in federated learning is solved, achieving efficient communication and model training, and improving the scalability and real-time performance of edge networks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG UNIV
- Filing Date
- 2026-03-17
- Publication Date
- 2026-05-19
AI Technical Summary
In edge network environments with limited bandwidth and resources, the communication overhead caused by the frequent exchange of model updates between edge devices and servers in federated learning becomes a bottleneck restricting scalability and real-time performance.
An adaptive simulated annealing optimization quantization strategy is adopted, which reduces the amount of data transmitted per round by adaptively selecting an appropriate bit width for each element. Combined with heuristic initialization, directional neighborhood constraints and adaptive cooling mechanism, the communication process is optimized.
It significantly reduces the amount of data transmitted per round, improves communication efficiency in edge environments, balances model performance and communication costs, avoids suboptimal local minima, and improves convergence and efficiency.
Smart Images
Figure CN121864799B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of distributed optimization technology for edge devices, specifically relating to an optimization method for the communication minimization maximization problem applicable to edge devices. Background Technology
[0002] With the explosive growth of Internet of Things (IoT) devices, edge intelligence has become a key paradigm for enabling real-time decision-making. In scenarios such as smart manufacturing, smart cities, and autonomous driving, massive amounts of data generated by edge devices can be fully utilized for model training. However, this data is typically highly privacy-sensitive and cannot be shared in centralized model training.
[0003] Federated learning, as an emerging distributed machine learning paradigm, allows multiple edge devices (such as mobile devices and edge nodes) to collaboratively train a global model while maintaining data locality, effectively solving the challenges of data privacy and secure sharing. A typical federated learning framework consists of a server and multiple distributed edge devices. In each round of training, a global model is first initialized and sent to selected edge devices. The selected edge devices train the received model using their private local data and send the updated parts of the model back to the server. The server then aggregates the collected models to obtain a new global model. This process is iterated multiple times until the global model achieves satisfactory performance.
[0004] Although federated learning has developed a model training framework that protects privacy data, in edge network environments with limited bandwidth and resources, the communication overhead caused by the frequent exchange of model updates (such as gradients or parameters) between edge devices and servers has become a key bottleneck restricting the scalability and real-time performance of federated learning. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides an optimization method for minimizing communication in edge devices. This method adaptively selects a more suitable bit width for each element, thereby significantly reducing the amount of data transmitted per round and achieving high-efficiency communication in edge environments. To achieve the above objectives, the technical solution of this invention is as follows:
[0006] An optimization method for the communication minimization maxima problem applicable to edge devices includes the following steps:
[0007] S1. The server initializes the relevant global parameters and sends them to the edge devices;
[0008] S2. The edge device receives global parameters from the server, calculates the gradient value of the local parameters based on local data, and updates the parameters accordingly.
[0009] S3. Edge devices calculate the difference between local parameters and initial values;
[0010] S4. The edge device quantizes the difference values and uploads the quantized results to the server.
[0011] S5. The server aggregates the quantization results uploaded by all edge devices and updates the global parameters based on the aggregation results;
[0012] S6. The server sends the new global parameters to all edge devices.
[0013] Preferably, in step S1, the server initializes the global model, randomly selects edge devices to participate in this round of federated learning, and distributes the global model parameters. , ; x is the model parameter in round t; These are the model parameters for round t.
[0014] Preferably, in step S2, each edge device receives model parameters in parallel and initializes local parameters. , Execute after Local stochastic gradient ascent and descent updates are performed step by step. , Let x and y be the model parameters for the k-th edge device in the t-th and 0-th epochs, respectively.
[0015] Preferably, step S3 is performed for each local iteration. Edge devices Randomly sample small batches of local data The stochastic gradient is calculated as follows;
[0016] ;
[0017] ;
[0018] For the k-th edge device, in the t-th round and the r-th epoch, the gradient of the model parameter x is given. For the k-th edge device, in the t-th epoch, is the gradient of the model parameter y.
[0019] Each edge device updates its model parameters and sample weights via stochastic gradient descent-ascend:
[0020] ;
[0021] ;
[0022] , Here, represents the learning rate for parameter x and the learning rate for parameter y, respectively.
[0023] exist After each round of local iteration, each edge device calculates the update difference relative to the initial global parameters:
[0024] ;
[0025] ;
[0026] : The t-th round parameter of the k-th edge device Relative to the initial global parameters Update differences; The parameters of the k-th edge device in round t Relative to the initial global parameters The update differences.
[0027] Preferably, the S4 edge device quantifies the difference values:
[0028] ;
[0029] ;
[0030] in As an adaptive quantization operator, the following is calculated: , And upload it to the server.
[0031] Preferably, for quantification An adaptive simulated annealing optimization quantization strategy is adopted, including an adaptive bit allocation method:
[0032] Step 1: Initial bit estimation. Allocate bits according to the size of each parameter to obtain an initial estimate:
[0033] ;
[0034] in Indicates the first The absolute value of each parameter For communications budget;
[0035] Step 2: Range normalization, achieved by linear scaling. Adjust to target bit width range :
[0036] ;
[0037] Step 3: Discretize, each normalized... They will all be projected onto a predefined set. At the closest discrete bit width in:
[0038] ;
[0039] in is the set of available bit widths, and b is an element in the set of available bit widths.
[0040] Preferably, in the neighborhood generation process of simulated annealing, the adjustment of bit width has a directional constraint, that is, the bit width is increased for parameters with large values and decreased for parameters with small values.
[0041] Preferably, the fixed cooling scheme in adaptive simulated annealing is replaced with an adaptive cooling strategy based on real-time solution quality assessment:
[0042] Step 1: Calculate the quality coefficient and define a quality coefficient. To quantify the relative improvement of the current solution:
[0043] ;
[0044] in, This represents the absolute change in the objective value compared to the best solution found so far. The value corresponding to the optimal solution, and It is a small constant;
[0045] Step 2: Calculate the adaptive cooling rate, through Calculate the adaptive cooling factor :
[0046] ;
[0047] in It is the base cooling rate. It is a temperature control and Sensitivity coefficient indicating the closeness of the relationship between them;
[0048] Step 3: Calculate the temperature update, update the temperature, and limit the temperature using a minimum threshold:
[0049] ;
[0050] ;
[0051] This is the minimum temperature.
[0052] Preferably, if If it's too large, it indicates a poor solution. Increase the cooling rate and slow it down to encourage continued exploration, if This indicates that the solution is good. Maintain normal cooling.
[0053] Preferably, the server aggregates the quantized updates received from each edge device:
[0054] ;
[0055] ;
[0056] The server uses aggregate updates to update global model parameters:
[0057] ;
[0058] ;
[0059] The server's learning rate for parameter x. : The learning rate of the server for parameter y.
[0060] Compared with the prior art, the beneficial effects of this application are as follows:
[0061] 1) Adaptively select a more suitable bit width for each element, thereby significantly reducing the amount of data transmitted per round and achieving efficient communication in edge environments.
[0062] 2) This invention proposes an adaptive bit allocation method based on parameter importance. This adaptive scheme provides efficient and effective initialization for subsequent optimization, balancing model performance and communication cost.
[0063] 3) In the neighborhood generation process of simulated annealing, the adjustment of bit width has directional constraints. For parameters with large values, the bit width is increased; for parameters with small values, the bit width is decreased. This mechanism reduces unnecessary random exploration and encodes domain knowledge into the search process, enabling ASA to avoid suboptimal local minima more effectively than conventional SA without guidance.
[0064] 4) To further improve convergence and efficiency, ASA replaces the fixed cooling scheme with an adaptive cooling strategy based on real-time solution quality assessment. This adaptive scheduling method ensures effective exploration in the early stages and targeted refinement when approaching convergence. Attached Figure Description
[0065] Figure 1 A schematic diagram of the process disclosed in the embodiments provided in this disclosure;
[0066] Figure 2This is a schematic diagram illustrating the specific process of the adaptive simulated annealing (ASA) optimization quantization strategy disclosed in this embodiment.
[0067] Figure 3 For comparison, see the effect diagram. Detailed Implementation
[0068] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.
[0069] An optimization method for the communication minimization maxima problem applicable to edge devices includes the following steps:
[0070] S1. The server initializes the global model, randomly selects edge devices to participate in this round of federated learning, and distributes the global model. , .
[0071] S2. Each edge device receives model parameters in parallel and initializes its local parameters. , Execute after Step-by-step local stochastic gradient ascent and descent (SGDA) updates. For each local iteration... ,equipment Randomly sample small batches of local data The stochastic gradient is calculated as follows;
[0072] ;
[0073] .
[0074] S3. Each edge device updates its model parameters and sample weights via stochastic gradient descent-ascend:
[0075] ;
[0076] .
[0077] exist After each round of local iteration, each edge device calculates the update difference relative to the initial global parameters:
[0078] ;
[0079] .
[0080] S4. Quantify the update differences:
[0081] ;
[0082] ;
[0083] in As an adaptive quantization operator, the following is calculated: , And upload it to the server.
[0084] To further improve convergence, our goal is to minimize the upper bound of the quantization variance. The initial target value is .
[0085] S5. The server receives and aggregates the quantitative updates from various edge devices:
[0086] ;
[0087] ;
[0088] The server uses aggregate updates to update global model parameters:
[0089] ;
[0090] .
[0091] S6. The server sends the new global parameters to all edge devices and determines whether the maximum number of iterations has been reached. If "yes", the parameters are output and the process ends; if "no", the process returns to step S2.
[0092] For quantification To provide a specific implementation method, this invention proposes an adaptive simulated annealing (ASA) optimization quantization strategy. This strategy combines heuristic initialization, oriented neighborhood constraints, and an adaptive cooling mechanism. Its design is closely consistent with the structure of the optimization objective, and the specific process is as follows.
[0093] (1) Adaptive bit allocation: Given a communication budget, this invention proposes an adaptive bit allocation method based on parameter importance.
[0094] Step 1: Initial Bit Estimation. First, allocate bits to each parameter according to its size to obtain an initial estimate.
[0095] ;
[0096] in This represents the absolute value of the i-th parameter, indicating its relative importance. This step ensures that more important parameters receive a larger share of the bit budget proportionally.
[0097] Step 2: Range normalization. This is achieved by linear scaling. Adjust to target bit width range .
[0098] ;
[0099] This normalization avoids extreme bit allocations and ensures that all allocated bit values are within an acceptable bit width range.
[0100] Step 3: Discretization. Finally, each normalized... They will all be projected onto a predefined set. The closest discrete bit width in the range.
[0101] ;
[0102] in It is the available set of bit widths (e.g., This step ensures compatibility with hardware or protocol constraints that require discrete bit representation. This adaptive approach provides an efficient and effective initialization for subsequent optimization, balancing model performance and communication costs.
[0103] After implementing the adaptive bit allocation method, it is essential to ensure that the overall communication budget remains within acceptable limits.
[0104] 1. Traverse all quantization dimensions, calculate the initial bit allocation value for the current dimension, and accumulate the total number of bits.
[0105] 2. Calculate the difference between the total number of bits and the target value. If the difference is greater than 0, an initial bit count adjustment is required: calculate the bit adjustment priority for each dimension, select the dimension with the highest priority to reduce the number of bits, and update the bit width set. In, the number that is least smaller than the current bit width (e.g., Currently, b is 8, and after the update it will be 4, which is the number to the left of the current bit width in the set.
[0106] 3. Update the dimension and the total number of bits. Repeat the above steps until the difference is less than 0.
[0107] (2) Neighborhood Generation Constraint Mechanism: To improve search efficiency, this invention introduces a directional constraint mechanism in the neighborhood generation process. This invention finds a strong correlation between the magnitude of the parameters and the required bit precision: parameters with larger values play a more critical role in model accuracy, therefore a larger bit width can improve model accuracy. Smaller parameters have limited impact and can be allocated fewer bits to save communication costs.
[0108] In the neighborhood generation process of simulated annealing, the bit width adjustment has directional constraints. For parameters with large values, the bit width is increased; for parameters with small values, the bit width is decreased. This mechanism reduces unnecessary random exploration and encodes domain knowledge into the search process, enabling ASA to avoid suboptimal local minima more effectively than unguided conventional SA.
[0109] Figure 2 The process involves randomly selecting two dimensions to update the number of bits. After this step, the current objective function value is calculated, and the objective value of the current solution is compared with that of the previous solution. If the objective value of the current solution is smaller, the current solution is accepted.
[0110] (3) Adaptive cooling: In order to further improve convergence and efficiency, ASA replaces the fixed cooling scheme with an adaptive cooling strategy based on real-time solution quality assessment.
[0111] Step 1: Calculate the quality coefficient. This invention defines a quality coefficient. To quantify the relative improvement of the current solution:
[0112] ;
[0113] in, This represents the absolute change in the objective value compared to the best solution found so far. The value corresponding to the optimal solution, and It is a small constant used to avoid division by zero.
[0114] Step 2: Calculate the adaptive cooling rate. (By...) Calculate the adaptive cooling factor .
[0115] ;
[0116] in It is the base cooling rate. It is a temperature control and The sensitivity coefficient indicating the closeness of the connection between them. If the current solution is inadequate ( (larger), then Increase the cooling rate and slow it down to encourage continued exploration. If the solution is good... ,but Maintain normal cooling.
[0117] Step 3: Calculate temperature update. Update the temperature and limit it using a minimum threshold:
[0118] ;
[0119] ;
[0120] This adaptive scheduling approach ensures effective exploration in the early stages and targeted refinement as convergence approaches.
[0121] Figure 3 Compared to other models, the Fed-FAQ application minimizes data transmission without affecting convergence quality, and significantly reduces communication costs compared to all baselines.
[0122] This invention also discloses an optimization system for the communication minimization maximization problem in edge devices, comprising an edge server and a server, wherein:
[0123] Server: The server initializes relevant global parameters and sends them to the edge devices;
[0124] Edge server: Edge devices receive global parameters from the server, calculate the gradient values of local parameters based on local data, and update the parameters; edge devices calculate the difference between local parameters and initial values;
[0125] The interaction between the two is as follows:
[0126] Edge devices quantize the difference values and upload the quantized results to the server; the server aggregates the quantization results uploaded by all edge devices and updates the global parameters based on the aggregated results; the server then distributes the new global parameters to all edge devices.
[0127] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.
Claims
1. An optimization method for the communication minimization maximization problem applicable to edge devices, characterized in that, Includes the following steps: S1. The server initializes the relevant global parameters and sends them to the edge devices; S2. The edge device receives global parameters from the server, calculates the gradient value of the local parameters based on local data, and updates the parameters accordingly. S3. Edge devices calculate the difference between local parameters and initial values; S4. The edge device quantizes the difference values and uploads the quantized results to the server. Edge devices quantify the differences: ； ； in As an adaptive quantization operator, the following is calculated: , And upload it to the server; For quantification An adaptive simulated annealing optimization quantization strategy is adopted, including an adaptive bit allocation method: Step 1: Initial bit estimation. Allocate bits according to the size of each parameter to obtain an initial estimate: ； in Indicates the first The absolute value of each parameter For communications budget; Step 2: Range normalization, achieved by linear scaling. Adjust to target bit width range : ； Step 3: Discretize, each normalized... They will all be projected onto a predefined set. At the closest discrete bit width in: ； in is the set of available bit widths, and b is an element in the set of available bit widths; Replace the fixed cooling scheme in adaptive simulated annealing with an adaptive cooling strategy based on real-time solution quality assessment: Step 1: Calculate the quality coefficient and define a quality coefficient. To quantify the relative improvement of the current solution: ； in, This represents the absolute change in the objective value compared to the best solution found so far. The value corresponding to the optimal solution, and It is a small constant; Step 2: Calculate the adaptive cooling rate, through Calculate the adaptive cooling factor : ； in It is the base cooling rate. It is a temperature control and Sensitivity coefficient indicating the closeness of the relationship between them; Step 3: Calculate temperature update, update the temperature and limit the temperature through a minimum threshold: ； ； Minimum temperature; S5. The server aggregates the quantization results uploaded by all edge devices and updates the global parameters based on the aggregation results; S6. The server sends the new global parameters to all edge devices.
2. The optimization method for the communication minimization maximization problem applicable to edge devices according to claim 1, characterized in that, In step S1, the server initializes the global model, randomly selects edge devices to participate in this round of federated learning, and distributes the global model parameters. , ; x is the model parameter in round t; These are the model parameters for round t.
3. The optimization method for the communication minimization maximization problem applicable to edge devices according to claim 2, characterized in that, Step S2: Each edge device receives model parameters in parallel and initializes local parameters. , Execute after Local stochastic gradient ascent and descent updates are performed step by step. , Let x and y be the model parameters for the k-th edge device in the t-th and 0-th epochs, respectively.
4. The optimization method for the communication minimization maximization problem applicable to edge devices according to claim 3, characterized in that, Step S3 for each local iteration Edge devices Randomly sample small batches of local data The stochastic gradient is calculated as follows; ； ； For the k-th edge device, in the t-th round and the r-th epoch, the gradient of the model parameter x is given. For the k-th edge device, in the t-th epoch, is the gradient of the model parameter y. Each edge device updates its model parameters and sample weights via stochastic gradient descent-ascend: ； ； , Here, represents the learning rate for parameter x and the learning rate for parameter y, respectively. exist After each round of local iteration, each edge device calculates the update difference relative to the initial global parameters: ； ； : The t-th round parameter of the k-th edge device Relative to the initial global parameters Update differences; The parameters of the k-th edge device in round t Relative to the initial global parameters The update differences.
5. The optimization method for the communication minimization maximization problem applicable to edge devices according to claim 1, characterized in that, In the neighborhood generation process of simulated annealing, the adjustment of bit width has a directional constraint, that is, the bit width is increased for parameters with large values and decreased for parameters with small values.
6. The optimization method for the communication minimization maximization problem applicable to edge devices according to claim 1, characterized in that, if If it's too large, it indicates a poor solution. Increase the cooling rate and slow it down to encourage continued exploration, if This indicates that the solution is good. Maintain normal cooling.
7. The optimization method for the communication minimization maximization problem applicable to edge devices according to claim 1, characterized in that, The server receives and aggregates the quantized updates from various edge devices: ； ； The server uses aggregate updates to update global model parameters: ； ； The server's learning rate for parameter x. : The learning rate of the server for parameter y.