Time estimation cooperative processing method based on ARM-FPGA coprocessor heterogeneous platform

Abstract

The invention discloses a time estimation cooperative processing method based on an ARM-FPGA coprocessor heterogeneous platform. The time estimation cooperative processing method is applied to a scenethat a big data storage unit and a computing unit are separated. A high-performance hardware FPGA coprocessor is deployed in a storage unit through an AXI protocol interface so as to enhance the dataprocessing capacity at the node of the storage unit and provide computing power for preprocessing of mass data. When redundant information in source data is too little, if source data preprocessing is carried out again, the data interaction time and the data preprocessing time of a storage unit are increased, and the time delay of the whole process cannot be reasonably reduced. Therefore, the preprocessing amount of the source data is estimated in a probabilistic mode, and whether data preprocessing needs to be conducted or not is decided. Estimating the time overhead in the whole process through a time estimation method, wherein the time overhead comprises the sum of time delays of source data extraction, transmission communication, calculation and the like. And finally, a proper processor is selected for data processing, so that the efficiency is maximized, and the overall performance of the system is improved.

Description

technical field [0001] The invention belongs to the technical field of distributed computing for heterogeneous platform cooperative processing, in particular to a time-predictable collaborative processing method based on ARM-FPGA coprocessor heterogeneous system. Background technique [0002] The explosive growth of data volume in the era of big data has led to a lot of redundant and invalid information in massive big data. In the big data system scenario where the data storage unit and the computing unit are separated, how to efficiently perform data preprocessing in the face of massive data and remove redundant data, so as to save data storage nodes, reduce data communication delay and improve data availability , is a problem worthy of attention. In traditional data processing, high-performance processors or multiple processors are used for online processing to meet the needs of big data processing. With the constraints of Moore's Law, the improvement of the main frequen...

AI Technical Summary

Problems solved by technology

However, during data preprocessing, there may be little redundancy in preprocessing data information, but data preprocessing will process the entire data, which cannot effectively reduce data internal communication traffic, network transmission delay, and calculation processing time.

Images