A mapreduce-based k-means clustering algorithm fpga acceleration system

Abstract

The present invention proposes a K-means clustering algorithm FPGA acceleration system based on MapReduce, the acceleration system mainly includes a Map task data transceiver subsystem, a Map task acceleration subsystem, a Reduce task data transceiver subsystem and a Reduce task acceleration subsystem, The Map task data sending and receiving subsystem transmits the corresponding data from the PCIe end to the Map task acceleration subsystem, and returns the final calculation result of the Map task acceleration subsystem to the PCIe end; the Reduce task data sending and receiving subsystem sends the corresponding data from the PCIe end to the Reduce The task acceleration subsystem returns the final calculation result of the Reduce task acceleration subsystem to the PCIe end. According to the acceleration system realized in the present invention, the required time-consuming calculation process is separated from the upper layer, and a dedicated hardware system is used to perform corresponding calculations, and each module in the system adopts a pipeline design and a parallel processing method, which greatly improves computing speed.

Description

technical field [0001] The invention belongs to a complex algorithm hardware acceleration system, in particular to a MapReduce-based K-means clustering algorithm FPGA acceleration system under a large amount of data. Background technique [0002] With the advent of the era of big data, the application of big data has increasingly demonstrated its advantages, and the fields it occupies are becoming larger and larger; however, the era of big data still faces some problems that need to be solved urgently, such as large data volume and value density. Low processing speed, high timing requirements, etc., so machine learning and data mining technologies are increasingly gaining attention in the computing field. [0003] Cluster analysis is an important content in data mining, and it is playing an increasingly important role in the fields of industry, commerce and scientific research. The K-means algorithm belongs to a basic division method in cluster analysis, and the error sum o...

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a K-means clustering algorithm FPGA acceleration system based on MapReduce under a large amount of data. Expensive bugs for faster computational processing

Images