Adaptive quantization decentralized learning method and system for heterogeneous edge devices
By optimizing resource utilization of heterogeneous edge devices during the training and communication phases through adaptive quantization control, the problem of resource constraints in training and communication is solved, and efficient collaborative learning is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG UNIV
- Filing Date
- 2026-04-10
- Publication Date
- 2026-06-23
AI Technical Summary
In decentralized learning on heterogeneous edge devices, existing technologies have failed to effectively address the dual limitations of training and communication resources, resulting in excessive computational and storage overhead, low communication efficiency, and untimely model exchange, which affects the overall collaborative learning effect.
An adaptive quantization method is adopted, which performs resource adaptive quantization control in the training and communication phases respectively. A random unbiased quantization function is used for low-precision training and communication quantization. Combined with single-block and multi-block communication quantization schemes, the quantization accuracy is dynamically adjusted to adapt to equipment resources and communication conditions.
It improves the feasibility and adaptability of collaborative learning for heterogeneous edge devices under resource-constrained conditions, reduces computational and storage overhead, optimizes communication efficiency, and enhances model training performance and overall convergence stability.
Smart Images
Figure CN121998133B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of distributed machine learning, edge computing and intelligent collaborative optimization technology, specifically an adaptive quantization decentralized learning method for heterogeneous edge devices. Background Technology
[0002] With the increasing demand for edge intelligence, distributed sensing, and multi-node collaborative decision-making, more and more edge devices need to complete joint modeling and collaborative optimization without aggregating raw data. Decentralized learning achieves model information propagation and joint updates through point-to-point communication between devices, without relying on a central server. Therefore, it has the advantages of avoiding single points of failure, adapting to dynamic network connections, and improving system robustness, and has important application value in scenarios such as intelligent transportation, unmanned system collaboration, industrial edge detection, and intelligent sensing.
[0003] However, in real-world deployment environments, heterogeneous edge devices are typically constrained by multiple factors, including computing, storage, and communication resources. Significant differences exist between devices in terms of processor power, available memory or GPU memory capacity, remaining power, heat dissipation capabilities, and background task loads. This makes it difficult for some resource-constrained devices to sustain the high computational burden and storage overhead associated with full-precision model training. Simultaneously, communication links between edge devices exhibit significant dynamism; their bandwidth, latency, throughput, packet loss rate, and available transmission bit budget change over time. If full-precision model parameter exchange is still employed, it can easily lead to excessive communication overhead, link congestion, and decreased collaborative efficiency.
[0004] To address the aforementioned issues, some existing technologies have begun to focus on optimizing collaborative learning under resource-constrained conditions. However, most of these efforts only address single resource limitations in the training or communication phases. One type of approach primarily targets the local training phase, reducing computational and storage overhead on the device side through low-precision training, model pruning, or parameter compression. While this approach can alleviate the local training pressure on edge devices to some extent, it typically does not synchronously address bandwidth limitations, link fluctuations, and transmission budget constraints during model exchange between devices. Therefore, in multi-node decentralized collaborative scenarios, it may still face problems such as high communication costs, untimely parameter exchange, and limited overall collaborative efficiency. Another type of approach primarily targets the communication phase, reducing the model transmission burden through sparse transmission, fixed-quantization-level compression, or selective parameter exchange. While this approach can reduce communication overhead to some extent, it does not fully consider the insufficient computing power, storage constraints, and energy consumption limitations faced by edge devices during the local training phase. Devices with weaker resource conditions may still struggle to stably complete high-precision local updates, leading to insufficient local training, unstable parameter updates, and further affecting the quality of subsequent neighbor aggregation and the overall model convergence performance. Summary of the Invention
[0005] To address the challenge of simultaneously balancing local training efficiency, model exchange overhead, and overall convergence performance in decentralized learning for heterogeneous edge devices under constraints of both training and communication resources, this invention provides an adaptive quantization decentralized learning method and system suitable for heterogeneous edge devices. The technical solution is as follows:
[0006] An adaptive quantization decentralized learning method for heterogeneous edge devices includes the following steps:
[0007] S1. Initialize training parameters, communication relationships, and local model parameters for each heterogeneous edge device;
[0008] S2. Each heterogeneous edge device adaptively determines the training quantization parameters based on its available computing resources and performs low-precision training based on local data;
[0009] S3. Each heterogeneous edge device adaptively determines the communication quantization parameters based on its own available communication resources, and performs quantization processing on the model information to be exchanged obtained after local training;
[0010] S4. Each heterogeneous edge device sends the quantized model information to the adjacent edge device and receives the quantized model information sent by the adjacent edge device;
[0011] S5. Each heterogeneous edge device performs aggregation and update based on the received quantized model information and local model information to obtain the updated local model parameters;
[0012] S6. Repeat steps S2 to S5 until the preset termination condition is met, and output the model parameters corresponding to each heterogeneous edge device.
[0013] Preferably, step S1 specifically includes: setting training control parameters, wherein the training control parameters include at least the number of edge devices. , number of communication rounds Local training steps Learning rate Termination conditions; initialization of each heterogeneous edge device. Local model parameters Initialize the communication relationships between the heterogeneous edge devices.
[0014] Preferred, the first The communication relationships between heterogeneous edge devices in a communication round can be represented as an undirected graph. ,in For the collection of edge devices, This is the corresponding aggregate weight matrix; the weight matrix is a real symmetric double random matrix, and satisfies... , , Among them, when the edge device With edge devices In the When the wheels are able to communicate, there are When edge devices With edge devices In the When the wheel cannot communicate, there is Edge devices In the The neighbor set of the wheel is:
[0015] ;
[0016] in, Indicates edge device Weights for retaining information from one's own local model.
[0017] Preferably, the available computing resources mentioned in step S2 include at least one or more of the following: processor utilization, available memory or video memory capacity, power budget, device temperature, background task load, and remaining power; each heterogeneous edge device dynamically adjusts the training quantization scale based on the resource status. This allows the training accuracy to be matched with local resource constraints.
[0018] Preferably, the training quantization in step S2 uses a random unbiased quantization function. For any d-dimensional vector ,have:
[0019] ;
[0020] ;
[0021] Among them, scalar quantizer According to Quantization is performed using a random rule that rounds up and down the quantization scale to ensure that the quantization result remains unbiased relative to the original value, thereby reducing the systematic bias introduced by low-precision training. Preferably, the low-precision local training in step S2 includes: first performing initial training quantization on the t-th round of the local model to obtain... Then execute The local stochastic gradient update is performed step by step, with each step updating the model parameters within the quantization domain. The update formula is as follows:
[0022] ;
[0023] in, Indicates edge device In the Round Data obtained by sampling during local training.
[0024] Preferably, the currently available communication resources in step S3 include at least one or more of the following: link bandwidth, latency, throughput, packet loss rate, transmission bit budget, number of neighbors, and time slot length; each edge device dynamically adjusts the communication quantization scale according to the current communication status. To adapt to time-varying link conditions.
[0025] Preferably, the communication quantization in step S3 uses a random unbiased quantization function. For any d-dimensional vector ,have:
[0026] ;
[0027] ;
[0028] Among them, scalar quantizer According to Quantization is performed using random rules for rounding up and down the quantization scale to ensure that the quantization result remains unbiased with respect to the original value, thereby reducing model transmission overhead under conditions of limited communication resources.
[0029] Preferably, the communication quantization scheme in step S3 includes a monolithic communication quantization scheme. This scheme will... The transmission vector is treated as a single parameter block and is distributed across each edge device. Total communication bit budget Configure a unified communication quantization scale under constraints If edge devices To be transmitted to its adjacent edge devices The weighted model information can be represented as a vector. The unified communication quantization scale is then:
[0030] ;
[0031] ;
[0032] in, Indicates the first Wheel edge device To be sent to neighboring devices The weighted model information in the first Values on the dimension, Let j represent the set of neighbors that communicate with edge device j in round t. Indicates edge device In the The available communication bit budget for the round, Represents the complete set of coordinates. This represents the set of coordinate indices with a value of 0 in the weighted model information.
[0033] Preferably, the communication quantization scheme in step S3 further includes a multi-block communication quantization scheme. This scheme will... The set of coordinates of the vector to be transmitted is divided into Non-overlapping parameter sub-blocks And configure the communication quantization scale for each parameter sub-block. If edge devices To be transmitted to its adjacent edge devices The weighted model information can be represented as a vector. The communication quantization scale is then:
[0034] ;
[0035] in, , Indicates the first Wheel edge device To be sent to neighboring devices The weighted model information in the first The value can be taken on the dimension; and
[0036] ;
[0037] in, , .
[0038] Preferably, the aggregation update in step S5 satisfies:
[0039] .
[0040] Preferably, steps S2 to S5 are repeated until a preset termination condition is met, and the model parameters corresponding to each heterogeneous edge device are output. .
[0041] An adaptive quantization decentralized learning system for heterogeneous edge devices includes heterogeneous edge devices, an initialization module, a training quantization module, a low-precision training module, a communication quantization module, a model exchange module, an aggregation update module, and an output module; wherein:
[0042] The heterogeneous edge device is used to perform low-precision local training, quantization model information exchange, and aggregation update based on local data, local model parameters, and current communication relationships.
[0043] The initialization module is used to set training control parameters, initialize the communication relationships between heterogeneous edge devices, and initialize the local model parameters of each heterogeneous edge device.
[0044] The training quantization module is used to adaptively determine the training quantization scale based on the currently available computing resources of the device;
[0045] The low-precision training module is used to perform low-precision training on the local model according to the training quantization scale.
[0046] The communication quantization module is used to adaptively determine the communication quantization scale based on the currently available communication resources of the device, and to quantize and encode the information of the exchange model.
[0047] The model exchange module is used to complete the sending and receiving of quantized model information between adjacent edge devices;
[0048] The aggregation update module is used to perform weighted aggregation of the received quantized model information and local model information according to the aggregation weight, and update the local model parameters.
[0049] The output module is used to output the model parameters corresponding to each heterogeneous edge device after the termination condition is met.
[0050] Compared with the prior art, the beneficial effects of this application are as follows:
[0051] 1) This invention proposes an adaptive quantization decentralized learning method for heterogeneous edge devices. Unlike existing technologies that only optimize for single resource constraints in the training or communication phases, this invention performs joint adaptive quantization control on both the training and communication phases. This comprehensively alleviates the resource constraints of heterogeneous edge devices in terms of computing, storage, and communication, enabling each edge device to dynamically adjust the quantization accuracy according to its own resource status and communication conditions. This improves the feasibility and adaptability of decentralized learning in resource-constrained scenarios.
[0052] 2) In the training phase, this invention employs a random unbiased quantization function for low-precision local training, which reduces both the computational and model storage overhead of local training while minimizing the systematic bias introduced by low-precision training. In the communication phase, a random unbiased quantization function is used to quantize and encode the model information to be exchanged, reducing model transmission overhead and improving communication resource utilization efficiency under limited communication resources. Therefore, this invention can better balance resource conservation and model training performance while ensuring continuous collaborative learning.
[0053] 3) This invention further provides a single-block communication quantization scheme and a multi-block communication quantization scheme, which can flexibly determine the communication quantization scale according to the structural characteristics of the parameters of the model to be transmitted and the communication bit budget constraints. Among them, the single-block communication quantization scheme is relatively simple to implement and is easy to deploy quickly under resource-constrained conditions; the multi-block communication quantization scheme can perform finer-grained quantization control for different parameter sub-blocks, thereby improving the adaptability of quantization processing, improving the neighbor aggregation effect, and helping to improve the convergence stability and collaborative learning effect of the overall model. Attached Figure Description
[0054] Figure 1 This is a schematic diagram illustrating the stages of the present invention;
[0055] Figure 2 This is a schematic diagram illustrating the specific process of the present invention. Detailed Implementation
[0056] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
[0057] This invention proposes an adaptive quantization decentralized learning method suitable for heterogeneous edge devices, such as... Figure 1As shown, this method introduces adaptive quantization mechanisms in both the training and communication phases, enabling multiple heterogeneous edge devices with computing, storage, and communication capabilities to achieve collaborative learning without the involvement of a central server. These heterogeneous edge devices can be any one or more of vehicle-to-everything (V2X) nodes, roadside units, drone nodes, mobile robots, industrial inspection terminals, or other edge terminal devices, but the invention is not limited to specific device types.
[0058] like Figure 2 As shown, an adaptive quantization decentralized learning method for heterogeneous edge devices includes the following steps:
[0059] (1) Initialize training parameters, communication relationships, and local model parameters for each heterogeneous edge device: (1.1) Set training control parameters, which include at least the number of edge devices. , number of communication rounds Local training steps Learning rate and termination conditions;
[0060] (1.2) Initialize each heterogeneous edge device Local model parameters (1.3) Initialize the communication relationships between the heterogeneous edge devices and set the first... The principles for constructing communication relationships between heterogeneous edge devices. These communication relationships can be represented as an undirected graph. ,in For the collection of edge devices, This is the corresponding aggregate weight matrix; the weight matrix is a real symmetric double random matrix, and satisfies... , , Among them, when the edge device With edge devices In the When the wheels are able to communicate, there are When edge devices With edge devices In the When the wheel cannot communicate, there is Edge devices In the The neighbor set of the wheel is:
[0061] ;
[0062] in, Indicates edge device Weights for retaining information from one's own local model.
[0063] (2) Determination of training quantization parameters and low-precision local training process: (2.1) In the first... At the start of each round of training, each heterogeneous edge device adaptively determines the training quantization scale based on its own available computing resources. The available computing resources include at least one or more of the following: processor utilization, available memory or video memory capacity, power budget, device temperature, background task load, and remaining power.
[0064] (2.2) Each heterogeneous edge device performs training quantization on the current local model parameters. The training quantization uses a random unbiased quantization function. For any d-dimensional vector ,have:
[0065] ;
[0066] ;
[0067] Among them, scalar quantizer According to Quantization is performed using random rules for rounding up and down the quantization scale to ensure that the quantization result remains unbiased with respect to the original value, thereby reducing the systematic bias introduced by low-precision training;
[0068] (2.3) Each heterogeneous edge device first performs initial training quantization on the local model in round t, and obtains Then execute The local stochastic gradient update is performed step by step, with each step updating the model parameters within the quantization domain. The update formula is as follows:
[0069] ;
[0070] in, Indicates edge device In the Round Data obtained by sampling during local training.
[0071] (3) Determination of communication quantization parameters and quantization process of information to be exchanged in the model:
[0072] (3.1) In the first During the round-robin communication phase, each heterogeneous edge device adaptively determines the communication quantization scale based on its own currently available communication resources. The available communication resources include at least one or more of the following: link bandwidth, latency, throughput, packet loss rate, transmission bit budget, number of neighbors, and time slot length.
[0073] (3.2) Each heterogeneous edge device performs communication quantization on the model information to be exchanged obtained after local training. The communication quantization adopts a random unbiased quantization function. For any d-dimensional vector ,have:
[0074] ;
[0075] ;
[0076] Among them, scalar quantizer According to Quantization is performed using random rules for rounding up and down the quantization scale to ensure that the quantization result remains unbiased with respect to the original value, thereby reducing model transmission overhead under conditions of limited communication resources.
[0077] (3.3) In one embodiment, the communication quantization scheme includes a monolithic communication quantization scheme. This scheme will... The transmission vector is treated as a single parameter block and is distributed across each edge device. Total communication bit budget Configure a unified communication quantization scale under constraints If edge devices To be transmitted to its adjacent edge devices The weighted model information can be represented as a vector. The unified communication quantization scale is then:
[0078] ;
[0079] ;
[0080] in, Indicates the first Wheel edge device To be sent to neighboring devices The weighted model information in the first Values on the dimension, Let j represent the set of neighbors that communicate with edge device j in round t. Indicates edge device In the The available communication bit budget for the round, Represents the complete set of coordinates. This represents the set of coordinate indices with a value of 0 in the weighted model information.
[0081] (3.4) In another embodiment, the communication quantization scheme further includes a multi-block communication quantization scheme. This scheme will... The set of coordinates of the vector to be transmitted is divided into Non-overlapping parameter sub-blocks And configure the communication quantization scale for each parameter sub-block. If edge devices To be transmitted to its adjacent edge devices The weighted model information can be represented as a vector. The communication quantization scale is then:
[0082] ;
[0083] in, , Indicates the first Wheel edge device To be sent to neighboring devices The weighted model information in the first The value can be taken on the dimension; and
[0084] ;
[0085] in, , .
[0086] (4) Information exchange process of the quantification model:
[0087] (4.1) In the first During the round-robin communication phase, each heterogeneous edge device sends the quantized model information to its neighbor set. It connects to adjacent edge devices and receives quantization model information sent by adjacent edge devices for subsequent aggregation updates.
[0088] (5) Aggregation update process:
[0089] (5.1) Each heterogeneous edge device performs weighted aggregation of the received neighbor quantized model information and local model information according to the aggregation weight to obtain the updated local model parameters. The aggregation update satisfies:
[0090] .
[0091] (6) Terminate the output process:
[0092] (6.1) Repeat steps (2) to (5) until the preset number of communication rounds is reached. Alternatively, if other preset termination conditions are met, the model parameters corresponding to each heterogeneous edge device will be output. .
[0093] This application relates to an adaptive quantization decentralized learning method and system applicable to vehicle network nodes, roadside units, mobile robots, industrial inspection terminals, drone nodes, and other heterogeneous edge devices.
[0094] An adaptive quantization decentralized learning system suitable for heterogeneous edge devices includes heterogeneous edge devices, an initialization module, a training quantization module, a low-precision training module, a communication quantization module, a model exchange module, an aggregation update module, and an output module.
[0095] Heterogeneous edge devices: perform low-precision local training, quantization model information exchange, and aggregation updates based on local data, local model parameters, and current communication relationships;
[0096] Initialization module: used to set training control parameters, initialize the communication relationships between heterogeneous edge devices, and initialize the local model parameters of each heterogeneous edge device;
[0097] Training quantization module: Used to adaptively determine the training quantization scale based on the currently available computing resources of the device;
[0098] Low-precision training module: used to perform low-precision training on the local model according to the training quantization scale;
[0099] Communication quantization module: used to adaptively determine the communication quantization scale based on the available communication resources of the device, and to quantize and encode the information of the model to be exchanged;
[0100] Model exchange module: Used to send and receive quantized model information between adjacent edge devices;
[0101] The aggregation update module is used to perform weighted aggregation of the received quantized model information and local model information based on the aggregation weight, and update the local model parameters.
[0102] Output module: Used to output the model parameters corresponding to each heterogeneous edge device after the termination condition is met.
[0103] Each logical step of the method described in this invention can be implemented by a computer program in a real system and ultimately compiled or interpreted into a sequence of machine instructions executable by the processor. Specifically, during the initialization phase, independent address spaces and data structure descriptors are allocated in memory for training control parameters, local model parameters, communication relationships, aggregated weight matrices, and training quantization scales and communication quantization scales, and data management structures are established for local parameter caching, neighbor transmit buffers, and neighbor receive buffers.
[0104] The training quantization parameter determination and low-precision local training phase mainly triggers instructions to read local resource status information and read-only load instructions for local model parameters. The training framework calculates the training quantization scale based on information such as processor utilization, available memory or GPU memory capacity, power budget, device temperature, background task load, and remaining power. During the computation graph scheduling process, it executes quantization, gradient calculation, gradient write-back, and parameter update instructions, enabling the local model to complete initial quantization and multi-step stochastic gradient updates within the quantization domain.
[0105] The communication quantization parameter determination and model exchange phase mainly triggers the reading of communication status information such as link bandwidth, latency, throughput, packet loss rate, transmission bit budget, number of neighbors, and time slot length, and calculates the communication quantization scale or the communication quantization scale corresponding to each parameter sub-block based on the current communication conditions; then, the model information to be exchanged is quantized and encoded, the quantization result is written into the transmission buffer, and the quantization model information is sent and received between adjacent edge devices through the network interface.
[0106] During the aggregation update phase, the received neighbor quantized model information and local model information are subjected to vector matrix operations based on aggregation weights, and the aggregation result is written back to the local model parameter area to obtain the local model parameters corresponding to the next communication round. During the aggregation process, only the quantized model information and local model information are read and written, without the need to introduce a central server for coordination.
[0107] Because this application employs adaptive quantization control in both training and communication, the data bit width of relevant local training instructions and cross-node transmission instructions is reduced. At the same time, the communication object is restricted to the quantized model information to be exchanged, thereby reducing local computational overhead, storage overhead, communication buffer read / write and cross-node data transfer burden, making it easier for decentralized collaborative learning under resource-constrained conditions to maintain training efficiency and convergence stability.
[0108] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not limited to the embodiments shown herein, but should be accorded the widest scope of protection consistent with the principles and novel features disclosed herein.
Claims
1. An adaptive quantization decentralized learning method for heterogeneous edge devices, characterized in that, Includes the following steps: S1. Initialize training parameters, communication relationships, and local model parameters for each heterogeneous edge device; S2. Each heterogeneous edge device adaptively determines the training quantization parameters based on its available computing resources and performs low-precision training based on local data; Training quantization uses a random unbiased quantization function. For any d-dimensional vector ,have: ； ； Among them, scalar quantizer According to Quantization is performed using random rules for rounding up and down the quantization scale to ensure that the quantization result remains unbiased about the original value. S3. Each heterogeneous edge device adaptively determines the communication quantization parameters based on its own available communication resources, and performs quantization processing on the model information to be exchanged obtained after local training; S4. Each heterogeneous edge device sends the quantized model information to the adjacent edge device and receives the quantized model information sent by the adjacent edge device; S5. Each heterogeneous edge device performs aggregation and update based on the received quantized model information and local model information to obtain the updated local model parameters; Aggregate updates satisfy: ； For edge devices In the The set of neighbors of the wheel; This is the corresponding aggregate weight matrix; Indicates edge device Weights that retain information about the local model; perform initial training quantization on the local model in round t, and then execute... Step-by-step local stochastic gradient update ; S6. Repeat steps S2 to S5 until the preset termination condition is met, and output the model parameters corresponding to each heterogeneous edge device.
2. The adaptive quantization decentralized learning method for heterogeneous edge devices according to claim 1, characterized in that, Step S1 specifically includes: setting training control parameters, wherein the training control parameters include at least the number of edge devices. , number of communication rounds Local training steps Learning rate Termination conditions; initialization of each heterogeneous edge device. Local model parameters .
3. The adaptive quantization decentralized learning method for heterogeneous edge devices according to claim 2, characterized in that, The communication relationships between heterogeneous edge devices can be represented as an undirected graph. ,in For the collection of edge devices, This is the corresponding aggregate weight matrix; the weight matrix is a real symmetric double random matrix, and satisfies... , , Among them, when the edge device With edge devices In the When the wheels are able to communicate, there are When edge devices With edge devices In the When the wheel cannot communicate, there is Edge devices In the The neighbor set of the wheel for: ； in, Indicates edge device Weights for retaining information from one's own local model. for A vector of all 1s in dimension.
4. The adaptive quantization decentralized learning method for heterogeneous edge devices according to claim 1, characterized in that, The low-precision local training in step S2 includes: first performing initial training quantization on the local model in the t-th round, to obtain... Then execute The local stochastic gradient update is performed step by step, with each step updating the model parameters within the quantization domain. The update formula is as follows: ； in, Indicates edge device In the Round The data obtained during local training Represents the loss function Find the gradient.
5. The adaptive quantization decentralized learning method for heterogeneous edge devices according to claim 1, characterized in that, The communication quantization in step S3 uses a random unbiased quantization function. For any d-dimensional vector ,have: ； ； Among them, scalar quantizer According to Quantization is performed using random rules for rounding up and down the quantization scale to ensure that the quantization result remains unbiased relative to the original value.
6. The adaptive quantization decentralized learning method for heterogeneous edge devices according to claim 1 or 5, characterized in that, The communication quantization scheme in step S3 includes a monolithic communication quantization scheme: Will The transmission vector is treated as a single parameter block and is distributed across each edge device. Total communication bit budget Configure a unified communication quantization scale under constraints If edge devices To be transmitted to its adjacent edge devices The weighted model information can be represented as a vector. The unified communication quantization scale is then: ； ； in, Indicates the first Wheel edge device To be sent to neighboring devices The weighted model information in the first Values on the dimension, Let j represent the set of neighbors that communicate with edge device j in round t. Indicates edge device In the The available communication bit budget for the round, Represents the complete set of coordinates. This represents the set of coordinate indices with a value of 0 in the weighted model information.
7. The adaptive quantization decentralized learning method for heterogeneous edge devices according to claim 2 or 6, characterized in that, The communication quantization scheme in step S3 also includes a multi-block communication quantization scheme: The set of coordinates of the vector to be transmitted is divided into Non-overlapping parameter sub-blocks And configure the communication quantization scale for each parameter sub-block. If edge devices To be transmitted to its adjacent edge devices The weighted model information can be represented as a vector. The communication quantization scale is then: ； in, , Indicates the first Wheel edge device To be sent to neighboring devices The weighted model information in the first The value can be taken on the dimension; and ； in, , .
8. An adaptive quantization decentralized learning system for heterogeneous edge devices, characterized in that, The method for implementing any one of claims 1-7 includes a heterogeneous edge device, an initialization module, a training quantization module, a low-precision training module, a communication quantization module, a model exchange module, an aggregation update module, and an output module; wherein: The heterogeneous edge device is used to perform low-precision local training, quantization model information exchange, and aggregation update based on local data, local model parameters, and current communication relationships. The initialization module is used to set training control parameters, initialize the communication relationships between heterogeneous edge devices, and initialize the local model parameters of each heterogeneous edge device. The training quantization module is used to adaptively determine the training quantization scale based on the currently available computing resources of the device; The low-precision training module is used to perform low-precision training on the local model according to the training quantization scale. The communication quantization module is used to adaptively determine the communication quantization scale based on the currently available communication resources of the device, and to quantize and encode the information of the exchange model. The model exchange module is used to complete the sending and receiving of quantized model information between adjacent edge devices; The aggregation update module is used to perform weighted aggregation of the received quantized model information and local model information according to the aggregation weight, and update the local model parameters. The output module is used to output the model parameters corresponding to each heterogeneous edge device after the termination condition is met.