Data processing method, device and system and computer readable storage medium

Abstract

The invention discloses a data processing method, device and system and a computer readable storage medium. The method comprises the following steps: encoding each computing task according to a presetencoding rule, and generating encoding information for recording the type of the current computing task, the number of the current computing task in a type task and an affiliated user; carrying out task division processing based on the coding information of each calculation task; obtaining a user task set of each type of calculation tasks according with a preset coding rule, and based on the usertask sets, the actual power consumption values of the FPGA accelerators of the FPGA cluster and the allowable number of the FPGA accelerators with the computing capability, calling a path optimization algorithm to allocate the computing tasks to the cloud server cluster and the FPGA cluster according to the principle of minimum energy consumption; analyzing the user to which each calculation taskbelongs according to a preset coding rule, and sending the corresponding calculation result to the user side, so that the advantages of low energy consumption and parallel calculation of the FPGA accelerator are brought into full play while the calculation pressure of the cloud data center is reduced.

Description

technical field [0001] The present application relates to the field of data processing, and in particular to a data processing method, device, system and computer-readable storage medium. Background technique [0002] With the advent of the 5G era, smart devices are developing rapidly, and the demand for data computing is increasing day by day. However, Moore's Law has gradually reached the bottleneck, and the x86 architecture represented by the CPU (Central Processing Unit) and the data center are gradually unable to meet the increasingly large and diverse data processing needs in terms of computing performance and energy efficiency. Compared with the traditional x86 architecture, FPGA (Field-Programmable Gate Array, Field-Programmable Gate Array) has the advantages of high performance and low power consumption. FPGAs can improve the performance of many computationally intensive tasks, such as matrix calculations, encryption, and signal processing, compared to CPUs. Compa...

AI Technical Summary

Problems solved by technology

However, the existing technical solutions are to lease the accelerators in the FPGA accelerator cluster to some developers. Such methods lead to the inability to reflect the heterogeneous computing capabilities of the FPGA accelerator cluster

Moreover, the existing technical solutions have not yet formed a perception of the operating status of the accelerator, resulting in the problem that the FPGA accelerator in the cluster cannot be used effectively

Images