Quantized Federated Learning for Heterogeneous Bit-Width Nodes

The patent applies local quality by customizing the quantization bit-width for each node based on its specific capabilities. Nodes with higher bit-width capabilities (e.g., 32-bit) receive higher precision quantized models, while nodes with lower capabilities (e.g., 8-bit) receive appropriately quantized models. This allows each node to operate at its optimal performance level, preventing slower nodes from becoming bottlenecks while maintaining compatibility across heterogeneous devices.

The patent changes the quantization parameters (bit-width) of the global model to match the capabilities of different nodes. The aggregator determines the bit-width capability of each node and generates quantized versions of the global model with appropriate precision levels. This parameter adaptation enables nodes with lower computational capabilities to complete training faster, reducing overall training time and improving fairness without sacrificing the ability to utilize heterogeneous hardware resources.

The patent applies preliminary action by having nodes with lower bit-width capabilities perform training in advance using pre-quantized models tailored to their capabilities. These nodes can complete their training iterations faster and submit their updates earlier to the aggregator. The aggregator then accumulates these updates over time rather than waiting for all nodes to complete simultaneously, thus maintaining aggregation accuracy while improving overall training throughput.

The patent introduces dynamics by allowing the aggregation process to be asynchronous and adaptive. The aggregator dynamically accumulates model updates from nodes as they complete training, rather than enforcing a rigid synchronous schedule. This dynamic approach allows faster nodes to contribute their updates without waiting for slower nodes, improving productivity while the aggregator maintains reliability by continuously integrating available updates to form an accurate global model.

The patent applies segmentation by dividing the model generation process into distinct quantization versions for different bit-width capabilities. Instead of creating one complex universal model, the system segments the global model into multiple quantized versions (e.g., 8-bit, 16-bit, 32-bit) and assigns appropriate versions to nodes based on their capabilities. This segmentation simplifies the generation process for each node while maintaining overall system compatibility.

The patent applies partial action by generating quantized model versions only for the specific bit-width capabilities that are actually needed by the nodes in the federated learning system. Rather than exhaustively creating all possible quantization variants, the aggregator identifies the relevant capability levels present in the node population and generates quantized models only for those levels. This reduces model generation complexity while still providing adequate support for the heterogeneous capabilities in the system.