Computing transmission pipeline parallel method and system suitable for graph index
By adopting the parallel method of computing transmission pipeline in graph index query, using the parallel processing of asynchronous I/O requests and distance calculations, the CPU idle time problem caused by the dependence of computing tasks and transmission tasks in the prior art is solved, and the query efficiency and Throughput, suitable for a variety of storage media.
Patent Information
- Application Number
- CN202410982368.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-08
AI Technical Summary
During the query process, the existing large-scale graph indexes have increased the CPU idle time due to the interdependence between computing tasks and transmission tasks, which affects the query efficiency, especially when the number of I/O requests is limited under massive vector data.
The calculation transmission pipeline parallel method is adopted to read the complete vector and neighbors of the storage layer through asynchronous I/O requests, and perform distance calculations while waiting for the IO to complete, reducing the CPU idle time and optimizing the graph index query pipeline.
It effectively reduces CPU idle time, improves the throughput of graph index query, avoids repeated calculation and transmission tasks, and is suitable for a variety of storage media including SSD and remote memory, with high performance and zero additional resource overhead.
Smart Images

Figure CN118916525A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of parallel optimization of vector index query, and in particular to a parallel method and system for computing and transmitting pipelines suitable for graph indexes. Background Art
[0002] Similarity queries for unstructured data are becoming a mainstream query method in the real world. To meet such special query requirements, existing systems usually use machine learning models to extract the features of unstructured data into vectors and filter neighbors based on vector distance calculations. Vector indexing is one of the important technologies used to reduce the total number of vectors that need to be checked for queries. Vector indexing is a special unstructured data indexing technology that accelerates similarity queries and can return a set of approximate neighbor vectors of the target vector based on distance. In recent years, with the advancement of algorithms and system implementations, vector indexing has been widely used to support applications such as search engines, recommendation systems, and generative services.
[0003] From the perspective of indexing algorithm type, vector indexes can be divided into cluster-based coarse-grained indexes and neighbor graph-based fine-grained indexes, referred to as cluster indexes and graph indexes. Cluster indexes cluster vectors in advance, and search for neighbors by selecting clusters first and then traversing vectors during queries. The advantage is that the vector traversal stage can be parallelized. Graph indexes use directed edges to record the neighbor relationships between vectors, and establish a neighbor graph consisting of all vectors in the system in advance. When querying the graph index service, it starts from a fixed vector and traverses the graph along the directed edges based on a local greedy strategy until the path reaches the end or exceeds the maximum vector access limit. The system regards the neighbor set on the search path as a global neighbor set and returns it to the application. Compared with cluster indexes, the advantage of graph indexes is that they reduce the amount of calculation, but the disadvantage is that the search algorithm is not friendly to parallel computing.
[0004] At present, due to the large scale of vector data sets and the trend of further growth, large-scale graph indexes store vectors and their neighboring edges in large-capacity, low-speed storage media such as SSDs. In order to avoid the need to read vectors from low-speed storage for all distance calculations, existing large-scale graph indexes store lossy compressed versions of all vectors and the vectors and neighboring edges corresponding to the starting point in memory. Since there are two versions of vectors, the graph index maintains a candidate set sorted based on approximate distance and a candidate set sorted based on true distance, which are referred to as approximate candidate sets and true candidate sets in the following text. During the query process, the system calculates the approximate distances between all neighbors of the current point and the target vector based on the compressed vector, filters the approximate candidate set, and then reads the complete vector and neighboring edges of the next candidate point in the approximate candidate set. The system then calculates the true distance between the current candidate point and the target vector and updates the true candidate set. The system repeats the search process until there is no next unvisited candidate point in the approximate candidate set or the number of visits exceeds the maximum limit. Finally, the system returns the true candidate set, which is the set of neighbor vectors for the query. Due to the natural random access characteristics of graph indexes, queries of graph indexes inevitably require sending a large number of I / O requests. Although some points can be cached in local memory to avoid transmission overhead. However, faced with massive amounts of vector data, the number of I / O requests that memory can reduce is limited. Another optimization method is to start traversing from multiple points of the graph index at the same time. Although this can improve parallelism to a certain extent, it also introduces a large number of repeated calculations and transmission tasks.
[0005] In the existing implementation, there is mutual dependence between computing tasks and transmission tasks, which causes the CPU to idle during the I / O transmission process. Therefore, the key to optimizing graph index queries is to design a more efficient computing and transmission pipeline to reduce the CPU idle time. Summary of the invention
[0006] In view of the defects in the prior art, the object of the present invention is to provide a parallel method and system for computing and transmitting pipelines suitable for graph indexing.
[0007] A single query computing transmission pipeline parallel method applicable to graph indexes provided by the present invention includes:
[0008] Step S1: Starting from the current candidate point, the approximate distance between the neighbors of the current candidate point and the target vector is calculated based on the compressed vector in the memory, and the approximate candidate set is updated;
[0009] Step S2: Check the approximate candidate set. When all points in the approximate candidate set have been visited or the total access count exceeds the preset maximum access limit, end the query and return the real candidate set. Otherwise, select the first unvisited candidate point in the approximate candidate set as the next candidate point, send an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and mark the candidate point as visited. While waiting for the IO to complete, calculate the real distance between the current candidate point and the target vector and update the real candidate set.
[0010] Step S3: upon receiving the I / O completion notification, retain the complete vector of the next candidate point; update the current candidate point based on the next candidate point, and repeatedly trigger steps S1 to S3.
[0011] Preferably, sending an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer includes: sending an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and saving the read complete vector and adjacent edges corresponding to the next candidate point in a buffer of the memory layer;
[0012] Release the memory layer buffer when the distance calculation is completed.
[0013] Preferably, the storage layer includes an SSD or a remote memory connected based on a high-speed memory interconnect technology.
[0014] A single query computing transmission pipeline parallel system suitable for graph indexing provided by the present invention includes:
[0015] Module M1: Starting from the current candidate point, the approximate distance between the neighbors of the current candidate point and the target vector is calculated based on the compressed vector in the memory, and the approximate candidate set is updated;
[0016] Module M2: Check the approximate candidate set. When all points in the approximate candidate set have been visited or the total access count exceeds the preset maximum access limit, end the query and return the real candidate set. Otherwise, select the first unvisited candidate point in the approximate candidate set as the next candidate point, send an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and mark the candidate point as visited. While waiting for the IO to complete, calculate the real distance between the current candidate point and the target vector and update the real candidate set.
[0017] Module M3: receives the I / O completion notification, retains the complete vector of the next candidate point, updates the current candidate point based on the next candidate point, and repeatedly triggers modules M1 to M3.
[0018] Preferably, sending an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer includes: sending an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and saving the read complete vector and adjacent edges corresponding to the next candidate point in a buffer of the memory layer;
[0019] Release the memory layer buffer when the distance calculation is completed.
[0020] Preferably, the storage layer includes an SSD or a remote memory connected based on a high-speed memory interconnect technology.
[0021] According to a batch query calculation transmission pipeline parallel method applicable to graph index provided by the present invention, each query in the batch query is processed in parallel simultaneously using the single query calculation transmission pipeline parallel method applicable to graph index described above.
[0022] Preferably, the approximate candidate sets and true candidate sets for different queries in the same batch are isolated by means of queues.
[0023] According to a batch query computing and transmission pipeline parallel system applicable to graph index provided by the present invention, for each query in the batch query, the above-mentioned single query computing and transmission pipeline parallel system applicable to graph index is used to trigger execution simultaneously.
[0024] Preferably, the approximate candidate sets and true candidate sets for different queries in the same batch are isolated by means of queues.
[0025] Compared with the prior art, the present invention has the following beneficial effects:
[0026] 1. The present invention does not use the method of caching complete vectors and adjacent edges to accelerate index query, thus saving memory;
[0027] 2. The batch query of the present invention performs pipeline setting optimization without algorithm modification, so it can be better compatible with different graph query algorithms;
[0028] 3. Compared with the graph query method that traverses multiple points simultaneously, the present invention does not introduce repeated calculation or transmission tasks;
[0029] 4. The present invention well identifies the computation and transmission tasks that do not have dependencies in the graph index query, and based on this, designs the pipeline to reduce the CPU idle time, thereby accelerating the throughput of the system service similarity query and accelerating the vector query of the graph index;
[0030] 5. The present invention takes into account both the optimization for a single query and the optimization for a batch query, has the characteristics of high performance and zero additional resource overhead, is suitable for a variety of storage media including SSD and remote memory, and has good applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0032] Figure 1 This is a flow chart of the parallel method for computing and transmitting according to the present invention.
[0033] Figure 2 This is a diagram of the computing transmission parallel system architecture of the present invention. DETAILED DESCRIPTION
[0034] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0035] Example 1
[0036] The parallel method for computing and transmitting pipelines applicable to graph indexes provided by the present invention includes:
[0037] The vector data structure of the present invention includes a compressed vector, a complete vector, and a vector neighbor edge set; the compressed vector of all data is stored in the memory, and the remaining data is stored in the external storage. When searching, the user inputs the target vector, and the neighbor set of the target vector in the database is returned. The distance calculated based on the compressed vector is called the approximate distance, and the distance calculated based on the complete vector is called the true distance. Since the two distances cannot be directly compared, two independent candidate sets need to be maintained in the calculation, namely the approximate candidate set and the true candidate set.
[0038] Specifically, the computational transmission pipeline parallel method applicable to graph indexes first identifies computational tasks and transmission tasks that do not have dependencies in graph index queries, and then implements computational transmission pipelines to reduce CPU idle time, thereby improving the throughput of graph index queries. The present invention considers both optimization for single queries and optimization for batch queries, has the characteristics of high performance and zero additional resource overhead, is applicable to a variety of storage media including SSDs and remote memories, and has good applicability.
[0039] Specifically, for a single query, the present invention provides a single query calculation transmission pipeline parallel method applicable to graph index, including:
[0040] Step S1: Starting from the current candidate point, the approximate distance between the neighbors of the current candidate point and the target vector is calculated based on the compressed vector in the memory, and the approximate candidate set is updated;
[0041] Step S2: Check the approximate candidate set. When all points in the approximate candidate set have been visited or the total access count exceeds the preset maximum access limit, end the query and return the real candidate set. Otherwise, select the first unvisited candidate point in the approximate candidate set as the next candidate point, send an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and mark the candidate point as visited. While waiting for the IO to complete, calculate the real distance between the current candidate point and the target vector and update the real candidate set.
[0042] Step S3: upon receiving the I / O completion notification, retain the complete vector of the next candidate point; update the current candidate point based on the next candidate point, and repeatedly trigger steps S1 to S3.
[0043] The candidate sets described in the present invention are all priority lists based on distance, in which each element can only be accessed once. The present invention selects the first unvisited vector in the approximate candidate set in each round of query. When updating the approximate and true candidate sets, the priorities of the vectors in the set are rearranged.
[0044] In this embodiment, the complete vector and edges are not stored in the memory for a long time.
[0045] The maximum access limit can be set according to user needs.
[0046] Send an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and save the read complete vector and adjacent edges corresponding to the next candidate point in the buffer of the memory layer;
[0047] Release the memory layer buffer when the distance calculation is completed.
[0048] The I / O request buffer is dynamically maintained while the system is running, and each query only requires a buffer that can accommodate a complete vector and its edge set.
[0049] The storage layer is not limited to a specific storage medium. It can use SSD or remote memory connected based on high-speed memory interconnect technology (RDMA or CXL).
[0050] The present invention also provides a single query computing and transmission pipeline parallel system applicable to graph indexes. The single query computing and transmission pipeline parallel system applicable to graph indexes can be implemented by executing the process steps of the single query computing and transmission pipeline parallel method applicable to graph indexes, that is, those skilled in the art can understand the single query computing and transmission pipeline parallel method applicable to graph indexes as a preferred implementation of the single query computing and transmission pipeline parallel system applicable to graph indexes.
[0051] For batch queries, the present invention provides a batch query calculation and transmission pipeline parallel method suitable for graph index, including using the above-mentioned single query calculation and transmission pipeline parallel method suitable for graph index for simultaneous parallel processing.
[0052] In this embodiment, the approximate candidate sets and the true candidate sets of different queries in the same batch are isolated from each other. The batch size is generally specified by the user. In this embodiment, the batch size is not higher than 10.
[0053] The present invention also provides a batch query calculation transmission pipeline parallel system suitable for graph index. The batch query calculation transmission pipeline parallel system suitable for graph index can be implemented by executing the process steps of the batch query calculation transmission pipeline parallel method suitable for graph index, that is, those skilled in the art can understand the batch query calculation transmission pipeline parallel method suitable for graph index as a preferred implementation of the batch query calculation transmission pipeline parallel system suitable for graph index.
[0054] Example 2
[0055] A parallel method for computing and transmitting pipelines applicable to graph indexes provided by the present invention includes:
[0056] like Figure 2 As shown below, we take the query with the starting point S as an example, where S's neighbors are vectors A, B, and C, and the target vector is Q. Figure 1 The query process of computing and transmitting in parallel is described in detail:
[0057] In step (1), the approximate distance between the compressed vectors of A, B, and C and Q is calculated and the approximate candidate set is updated. In ascending order of distance, the three vectors in the candidate set are arranged as C, B, and A.
[0058] In step (2), check the approximate candidate set to determine whether there is a next unvisited point. If not or the number of access vectors exceeds the maximum limit, execute step (3). Assuming that this condition is not met in the example, execute step (4).
[0059] In step (3), the present invention returns the true candidate set of Q as a neighbor vector set to the application, and ends the query.
[0060] In step (4), the first unvisited point C in the candidate set is selected and an I / O request is sent to read its complete vector and edge set.
[0061] In step (5), if there are other query requests in the batch, the present invention repeats the above steps for them.
[0062] In step (6), the present invention calculates the true distance between N and Q based on the complete vector and updates the true candidate set. If there are other queries in the batch, the same operation is performed.
[0063] In step (7), the present invention waits for the I / O request of query Q to be completed, then updates the current point to C, and iterates from step (1) again until the execution condition of step (3) is triggered. The same operation is performed for other queries in the batch.
[0064] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.
[0065] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.
Claims
1. A single query computing and transmission pipeline parallel method suitable for graph index, characterized in that: include: Step S1: Starting from the current candidate point, the approximate distance between the neighbors of the current candidate point and the target vector is calculated based on the compressed vector in the memory, and the approximate candidate set is updated; Step S2: Check the approximate candidate set. When all points in the approximate candidate set have been visited or the total visit count exceeds the preset maximum visit limit, end the query and return the true candidate set. Otherwise, select the first unvisited candidate point in the approximate candidate set as the next candidate point, send an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and mark the candidate point as visited; while waiting for the IO to complete, calculate the true distance between the current candidate point and the target vector, and update the true candidate set; Step S3: upon receiving the I / O completion notification, retain the complete vector of the next candidate point; update the current candidate point based on the next candidate point, and repeatedly trigger steps S1 to S3.
2. According to claim 1, the single query calculation transmission pipeline parallel method suitable for graph index is characterized in that: The sending of an asynchronous I / O request to read a complete vector and an adjacent edge corresponding to a next candidate point in the storage layer includes: sending an asynchronous I / O request to read a complete vector and an adjacent edge corresponding to a next candidate point in the storage layer, and saving the read complete vector and the adjacent edge corresponding to the next candidate point in a buffer of the memory layer; Release the memory layer buffer when the distance calculation is completed.
3. The single query calculation transmission pipeline parallel method applicable to graph index according to claim 1, characterized in that: The storage layer includes an SSD or a remote memory connected based on a high-speed memory interconnect technology.
4. A single query computing and transmission pipeline parallel system suitable for graph indexing, characterized in that: include: Module M1: Starting from the current candidate point, the approximate distance between the neighbors of the current candidate point and the target vector is calculated based on the compressed vector in the memory, and the approximate candidate set is updated; Module M2: Check the approximate candidate set. When all points in the approximate candidate set have been visited or the total visit count exceeds the preset maximum visit limit, the query ends and the real candidate set is returned. Otherwise, select the first unvisited candidate point in the approximate candidate set as the next candidate point, send an asynchronous I / O request to read the complete vector and adjacent edges corresponding to the next candidate point in the storage layer, and mark the candidate point as visited; while waiting for the IO to complete, calculate the true distance between the current candidate point and the target vector, and update the true candidate set; Module M3: receives the I / O completion notification, retains the complete vector of the next candidate point, updates the current candidate point based on the next candidate point, and repeatedly triggers modules M1 to M3.
5. The single query computing and transmission pipeline parallel system suitable for graph indexing according to claim 4, characterized in that: The sending of an asynchronous I / O request to read a complete vector and an adjacent edge corresponding to a next candidate point in the storage layer includes: sending an asynchronous I / O request to read a complete vector and an adjacent edge corresponding to a next candidate point in the storage layer, and saving the read complete vector and the adjacent edge corresponding to the next candidate point in a buffer of the memory layer; Release the memory layer buffer when the distance calculation is completed.
6. The single query computing and transmission pipeline parallel system suitable for graph indexing according to claim 4, characterized in that: The storage layer includes an SSD or a remote memory connected based on a high-speed memory interconnect technology.
7. A batch query calculation and transmission pipeline parallel method suitable for graph index, characterized in that: For each query in the batch query, the single query calculation transmission pipeline parallel method applicable to graph index described in any one of claims 1 to 3 is used to process them in parallel at the same time.
8. The batch query calculation and transmission pipeline parallel method applicable to graph index according to claim 7, characterized in that: The approximate candidate sets and true candidate sets for different queries in the same batch are isolated by queues.
9. A batch query computing and transmission pipeline parallel system suitable for graph indexing, characterized in that: For each query in the batch query, the single query calculation transmission pipeline parallel system suitable for graph indexing described in any one of claims 4 to 6 is used to trigger execution simultaneously.
10. The batch query computing and transmission pipeline parallel system suitable for graph index according to claim 9, characterized in that: The approximate candidate sets and true candidate sets for different queries in the same batch are isolated by queues.
