Decentralized federated learning framework based on heterogeneous computing power perception and modeling method

Abstract

The invention discloses a decentralized federated learning framework based on heterogeneous computing power perception. The decentralized federated learning framework comprises a cloud coordinator and a plurality of equipment ends. The cloud coordinator is used for management, training and parameter updating scheme generation during operation and regular model backup. The equipment end is used for transmitting equipment information to the cloud coordinator, operating a model locally and updating equipment end parameters. The cloud coordinator acquires the least common multiple of the one-time training time of the equipment end as a super cycle, the equipment end calculates different step lengths in the super cycle, and the model is aggregated in the integral multiple of the super cycle. Different local steps are operated according to different computing capabilities of equipment, and in the model aggregation process, negative effects of slow nodes are reduced; and a distributed point-to-point communication mode is adopted, and the communication pressure of the central server in the distributed training process can be eliminated under the condition that the overall communication traffic is not increased.

Description

technical field [0001] The invention belongs to the field of big data data aggregation technology, and in particular relates to a decentralized federated learning framework and modeling method based on heterogeneous computing power perception. Background technique [0002] Artificial intelligence is increasingly used in all aspects of human life. However, traditional artificial intelligence learning faces two outstanding problems. [0003] 1) Data island problem [0004] An AI project may involve multiple fields and needs to integrate data from various companies and departments. (For example, research on online consumption of residents requires data from various consumer platforms, and may also require bank data, etc.) However, in reality, it is almost impossible to integrate data scattered in various places and institutions. [0005] 2) Data Privacy Issues [0006] The promulgation of GDPR has made all parties pay more and more attention to data ownership and privacy, an...

AI Technical Summary

Problems solved by technology

[0005] 2) Data Privacy Issues

The imbalance of node computing power will exacerbate the lagging problem and cause some slow nodes to fall seriously behind

[0012] 2) The federated learning framework has a huge amount of communication

FedAvg's centralized model aggregation strategy will put a lot of communication and computing pressure on the central server, resulting in poor scalability and communication bottlenecks

[0013] 3) The equipment is widely distributed, which can easily cause unreliable communication and reduce performance

[0015] 1) Model aggregation is performed in an asynchronous manner, however, stale lagging node parameters can lead to incorrect convergence or increased number of iterations

[0016] 2) Adopt centralized model synchronization and aggregation methods, however, in the case of massive devices, the communication pressure increases sharply

[0017] 3) Adopt a distributed design federated learning framework. However, this framework assumes that the devices are isomorphic and the synchronous aggregation model is not suitable for training models on heterogeneous devices.

Images