A query task communication method and system

Abstract

The invention provides a task communication querying method and a system thereof. The method comprises the following steps: the server of the received query request analyzing the query request, and decomposing the query sentence in the query request into a plurality of sub-steps, wherein the related information of the sub-steps belongs to the metadata of the query task; processing the query request step by step from the first sub-step of the plurality of sub-steps to obtain an intermediate result of the query; if the next sub-step relies on data at the remote server, then, the query intermediate result and the metadata of the query task being sent to the remote server through GPUDirect RDMA and RDMA respectively, and the remote server continuing to process the sub-steps of the query request according to the received query intermediate result and the metadata of the query task. The invention reduces the cost of the whole communication process, avoids the contention of network resourcesand improves the performance of the whole inquiry system.

Description

technical field [0001] The present invention relates to the technical field of communication, in particular to a query task communication method based on GPUDirect RDMA. Background technique [0002] In the era of big data, the scale of data is getting larger and larger. For example, the number of web pages on the Internet reaches billions. Huge data is often divided into multiple parts and stored in multiple machines. In order to find interesting data in a huge data set, software that provides query services usually runs in a distributed environment consisting of multiple machines. [0003] With the continuous development of hardware technology, servers equipped with high-performance graphics processing units (GPUs) have gradually appeared in data centers. Compared with CPUs, GPUs have stronger computing performance and higher memory bandwidth, so they are often used as computing tasks. Accelerator, as a supplement to the CPU. Nvidia GPUs, which are widely used in data ce...

AI Technical Summary

Problems solved by technology

It is obvious that the frequent data copying between GPU and CPU in the above communication process will significantly increase the time consumption of the query task, which will cause a bad user experience for tasks with low delay tolerance such as query.

Images