A vector search system, method, and apparatus

Abstract

A vector search system, method and device, in the application, a client sends a vector search instruction to a server, the vector search instruction is used for requesting to obtain K vectors similar to a target vector in a vector set, the target vector can be carried in the vector search instruction. After receiving the vector search instruction, the server determines K vectors similar to the target vector from the vector set based on the global index of the vector set, feeds back the K vectors to the client, the vector set is distributed in a plurality of storage nodes in a storage cluster, the global index for the entire vector set is saved on the server side, the global index is used to realize vector search, and it is no longer necessary to use the indexes of the subsets of the vector set to respectively search the vectors, thereby effectively ensuring the efficiency of the vector search.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a vector search system, method, and device. Background Technology

[0002] Vector search technology is an information search method based on the vector space model. It represents unstructured data such as text, voice, and images as vectors and evaluates the relevance between unstructured data by calculating the similarity between vectors.

[0003] With the generation of massive amounts of data, traditional vector search methods have encountered bottlenecks when processing large-scale data. In other words, with the generation of massive amounts of data, the search scope of vector search will be significantly expanded. The search scope can be understood as the data set. The goal of vector search is to find K data similar to the data to be searched from the data set. The expansion of the search scope will increase the difficulty of similarity calculation and the complexity of similarity evaluation, thus affecting the efficiency of vector search.

[0004] To improve the efficiency of vector search, the dataset is divided into multiple subsets, with each node storing one subset and constructing a vector index for that subset. During vector search, each node searches based on the vector index of its subset, obtaining search results for each subset. These search results are then combined to obtain the final search result. However, this approach requires each node to participate in the vector search, which involves searching the vector index within each node, resulting in significant computational consumption and limited improvement in efficiency. Summary of the Invention

[0005] This application provides a vector search system, method, and device to improve vector search efficiency.

[0006] In a first aspect, embodiments of this application provide a vector search method. This method is applied to a vector search system, which includes a client and a server. In this method, the client sends a vector search instruction to the server. The vector search instruction is used to request the acquisition of K vectors in the vector set that are similar to the target vector. The vector search instruction may carry the target vector.

[0007] After receiving a vector search command, the server determines K vectors similar to the target vector from the vector set based on the global index of the vector set, and sends the K vectors back to the client. The vector set is distributed across multiple storage nodes in the storage cluster.

[0008] Using the above method, a global index for the entire vector set is stored on the server side. This global index can be used to perform a global search of vectors, eliminating the need to perform vector searches separately using the indices of subsets of the vector set, thus effectively ensuring the efficiency of vector search.

[0009] In one possible implementation, the global index is a single index built upon the set of vectors. A "global" index means that it is built over the entire set of vectors, unlike indexes built over subsets of the vector set.

[0010] Using the above method, only one global index needs to be maintained on the server side, eliminating the need to maintain indexes built for each subset of the vector set.

[0011] In one possible implementation, the server includes multiple search nodes, and the client can send a vector search instruction to the target search node among the multiple search nodes; after receiving the vector search instruction, the target search node determines K vectors similar to the target vector from the vector set based on the global index.

[0012] Using the above method, the client can specify a target search node to handle the vector search instruction. Based on this, the client can distribute different vector search instructions to different search nodes, thereby achieving load balancing. Furthermore, compared to the method of completing the vector search together with multiple nodes, in this application, the client can specify a single target search node to perform the vector search, eliminating the need for multiple nodes to participate. In this approach, a single target search node can independently complete the vector search, making the process simpler and avoiding interactions between multiple nodes (such as the interactions that occur when aggregating search results from various nodes), further ensuring vector search efficiency.

[0013] In one possible implementation, the global index includes multiple index layers. The client stores the upper-level index in the global index, which includes the first M index layers in the global index. The client performs a pre-search based on the upper-level index, and the vector search instruction includes the results of the pre-search. The server determines K vectors similar to the target vector from the vector set based on the results of the pre-search and the global index.

[0014] Using the above method, the client can perform part of the vector search operation, while the server performs the other part. This reduces the computational burden of vector search on the server while ensuring the efficiency of vector search execution on the server side.

[0015] In one possible implementation, the client determines the target search node from multiple search nodes based on a routing policy, which specifies the rules for selecting the target search node. After determining the target search node, the client sends a vector search instruction to that target search node from the multiple search nodes.

[0016] Using the above method, the client distributes vector search instructions according to the routing strategy, thereby avoiding uneven load distribution among search nodes on the server side.

[0017] In one possible implementation, the routing strategy includes round-robin or pre-search rules. The pre-search rules are rules for selecting the target search node based on the results of the pre-search. The results of the pre-search are the results obtained by the client performing a pre-search based on the upper-level index, which includes the first M index layers in the global index.

[0018] The routing strategy described above offers considerable flexibility, allowing for customization to suit various scenarios. Furthermore, the rule of selecting the target search node based on pre-search results ensures that for different vector search instructions, the same pre-search result will correspond to the same target search node. This enables the target search node to cache relevant index data, and the pre-caching of index data allows for efficient vector search execution.

[0019] In one possible implementation, the target search node caches the first index data in the global index, and the target search node performs a vector search based on the first index data to determine K vectors.

[0020] By using the above method, the target search node does not need to retrieve the first index data from the storage cluster because it has cached the first index data, which effectively improves the efficiency of vector search.

[0021] In one possible implementation, when the target search node performs a vector search to determine K vectors based on the first index data, it retrieves second index data from the global index, which is different from the first index data, from the storage cluster. The storage cluster stores the global index.

[0022] The target search node performs a vector search based on the first index data and the second index data to determine K vectors.

[0023] Using the above method, the target search node obtains the second index data required for vector search by interacting with the storage cluster, thus ensuring that vector search can be performed.

[0024] In one possible implementation, when the target search node retrieves the second index data from the global index of the storage cluster, the target search node sends a prefetch request to the storage cluster. The prefetch request is used to request the reading of the second index data and the prefetching of the third index data adjacent to the second index data in the global index.

[0025] The target search node obtains the second and third index data from the storage cluster.

[0026] Using the above method, the target search node can prefetch the third index data while acquiring the second index data, so as to improve the efficiency of subsequent vector search execution by the target search node, especially for vector searches that require the use of the third index data.

[0027] In one possible implementation, the target search node sends a first instruction to the storage cluster, which instructs the storage cluster to perform a vector search based on the fourth index data in the global index.

[0028] The target search node obtains the results of the vector search from the storage cluster, and the results of the vector search indicate K vectors.

[0029] Using the above method, the target search node can complete the vector search with the help of the computing power of the storage cluster, realizing "search pushdown", which effectively saves the computing power of the target search node and also improves the utilization rate of computing power in the storage cluster.

[0030] In one possible implementation, the storage cluster receives a first instruction and, based on a near-data strategy, determines a target computing unit within the storage cluster. The target computing unit is used to perform a vector search based on the fourth index data in the global index to obtain the result of the vector search.

[0031] Using the above method, a target computing unit suitable for performing vector search can be selected within the storage cluster to complete the vector search.

[0032] In one possible implementation, the near-data strategy includes selecting the target computing unit based on the storage location of the fourth index data.

[0033] The essence of the near-data strategy, as described above, is to select the computing unit closest to the fourth index data. This reduces the latency for the computing unit to acquire the fourth index data and ensures the efficiency of vector search.

[0034] In one possible implementation, the target computing unit includes some or all of the following:

[0035] Storage array controller, computing accelerator card, CPU, hard disk controller.

[0036] The above method allows for a wide variety of target computing units, making it suitable for storage clusters of different types and effectively expanding the application scenarios of this vector search method.

[0037] Secondly, this application also provides a vector search system, the relevant details of which can be found in the foregoing description and will not be repeated here. This vector search system includes a client and a server:

[0038] The client is used to send vector search instructions to the server, which carry the target vector.

[0039] On the server side, a global index is used to determine K vectors similar to the target vector from the vector set, and then feeds back the K vectors to the client. The vector set is distributed across multiple storage nodes in the storage cluster.

[0040] In one possible implementation, the global index is a single index built upon a set of vectors.

[0041] In one possible implementation, the server includes multiple search nodes, and the client sends a vector search instruction to the target search node among the multiple search nodes; the target search node in the server receives the vector search instruction and determines K vectors similar to the target vector from the vector set based on the global index.

[0042] In one possible implementation, the global index includes multiple index layers. The client stores the upper-level index in the global index, which includes the first M index layers in the global index. The client performs a pre-search based on the upper-level index, and the vector search instruction includes the results of the pre-search.

[0043] The server determines K vectors similar to the target vector from the vector set based on the results of the pre-search and the global index.

[0044] In one possible implementation, the client determines the target search node from multiple search nodes based on a routing policy, which indicates the rules for selecting the target search node.

[0045] In one possible implementation, the routing strategy includes round-robin or pre-search rules. The pre-search rules are rules for selecting target search nodes based on the results of the pre-search. The results of the pre-search are obtained by the client based on the upper-level index and are included in the vector search instructions. The upper-level index includes the first M index layers in the global index.

[0046] In one possible implementation, the target search node caches the first index data in the global index, and performs a vector search based on the first index data to determine K vectors.

[0047] In one possible implementation, the target search node retrieves second index data from the global index, which is different from the first index data, from the storage cluster. The storage cluster stores the global index. The target search node performs a vector search based on the first index data and the second index data to determine K vectors.

[0048] In one possible implementation, the target search node sends a prefetch request to the storage cluster. The prefetch request is used to request the reading of second index data and the prefetching of third index data adjacent to the second index data in the global index. The target search node then obtains the second index data and the third index data returned by the storage cluster.

[0049] In one possible implementation, the target search node sends a first instruction to the storage cluster, which instructs the storage cluster to perform a vector search based on the fourth index data in the global index; the target search node obtains the results of the vector search from the storage cluster, which indicate K vectors.

[0050] In one possible implementation, the vector search system further includes a storage cluster that receives a first instruction and, based on a near-data strategy, determines a target computing unit within the storage cluster. The target computing unit is used to perform a vector search based on the fourth index data in the global index to obtain the result of the vector search.

[0051] In one possible implementation, the near-data strategy includes selecting the target computing unit based on the storage location of the fourth index data.

[0052] In one possible implementation, the target computing unit includes some or all of the following:

[0053] Storage array controller, computing accelerator card, CPU, hard disk controller.

[0054] Thirdly, this application also provides a vector search method, which is executed by a client. Relevant details can be found in the foregoing description and will not be repeated here. The client stores the upper-level index in the global index. The global index is a single index constructed based on a vector set, and includes multiple index layers. The upper-level index includes the first M index layers of the global index. In this method:

[0055] The client performs a pre-search based on the upper-level index and obtains the pre-search results.

[0056] The client sends a vector search command to the server. The vector search command is used to request the K vectors in the vector set that are similar to the target vector. The vector search command includes the results of the pre-search.

[0057] In one possible implementation, the client determines the target search node from multiple search nodes according to a routing policy. The routing policy indicates the rules for selecting the target search node. After determining the target search node, the client sends a vector search instruction to the target search node.

[0058] In one possible implementation, the routing strategy includes round-robin or pre-search rules, where the pre-search rules are rules for selecting target search nodes based on the results of the pre-search. The results of the pre-search are the results obtained by the client device performing a pre-search based on the upper-level index, and the results of the pre-search are included in the vector search instructions. The upper-level index includes the first M index layers in the global index.

[0059] Fourthly, this application also provides a vector search method, which is executed by a server. Related details can be found in the foregoing description and will not be repeated here. In this method:

[0060] The server receives a vector search command from the client, which is used to request the K vectors in the vector set that are similar to the target vector.

[0061] The server determines K vectors similar to the target vector from the vector set based on the global index of the vector set, and feeds back the K vectors to the client. The vector set is distributed across multiple storage nodes in the storage cluster.

[0062] In one possible implementation, the global index is a single index built upon a set of vectors.

[0063] In one possible implementation, the server includes multiple search nodes, among which the target search node receives vector search instructions; the target search node determines K vectors similar to the target vector from the vector set based on a global index.

[0064] In one possible implementation, the global index includes multiple index layers, and the vector search instruction includes the results of the pre-search, which are the results of vector search using the upper-level index. The upper-level index includes the first M index layers in the global index. When the server determines the K vectors similar to the target vector, it determines the K vectors similar to the target vector from the vector set based on the results of the pre-search and the global index.

[0065] In one possible implementation, the target search node caches the first index data in the global index. When the target search node determines K vectors similar to the target vector, it performs a vector search based on the first index data to determine the K vectors.

[0066] In one possible implementation, when the target search node performs a vector search to determine K vectors based on the first index data, it retrieves second index data from the global index, which is different from the first index data, from the storage cluster. The storage cluster stores the global index.

[0067] The target search node performs a vector search based on the first index data and the second index data to determine K vectors.

[0068] In one possible implementation, the target search node can send a prefetch request to the storage cluster. The prefetch request is used to request the reading of the second index data and the prefetching of the third index data adjacent to the second index data in the global index. The target search node then obtains the third index data and the second index data returned by the storage cluster.

[0069] In one possible implementation, when the target search node determines K vectors similar to the target vector from the vector set based on the global index, the target search node sends a first instruction to the storage cluster, which instructs the storage cluster to perform a vector search based on the fourth index data in the global index.

[0070] The target search node obtains the results of the vector search from the storage cluster, and the results of the vector search indicate K vectors.

[0071] In one possible implementation, the storage cluster receives a first instruction and, based on a near-data strategy, determines a target computing unit within the storage cluster. The target computing unit is used to perform a vector search based on the fourth index data in the global index to obtain the result of the vector search.

[0072] In one possible implementation, the near-data strategy includes selecting the target computing unit based on the storage location of the fourth index data.

[0073] In one possible implementation, the target computing unit includes some or all of the following: a storage array controller, a computing accelerator card, a CPU, and a hard disk controller.

[0074] Fifthly, embodiments of this application also provide a vector search method, which is executed by a storage cluster. Relevant details can be found in the foregoing description and will not be repeated here. In this method:

[0075] The storage cluster receives a first instruction, which instructs the storage cluster to perform a vector search based on the fourth index data in the global index.

[0076] The storage cluster determines the target computing power unit in the storage cluster based on the near data strategy. The target computing power unit is used to perform vector search based on the fourth index data in the global index to obtain the vector search result.

[0077] In one possible implementation, the near-data strategy includes selecting the target computing unit based on the storage location of the fourth index data.

[0078] In one possible implementation, the target computing unit includes some or all of the following: a storage array controller, a computing accelerator card, a CPU, and a hard disk controller.

[0079] Sixthly, embodiments of this application also provide a client device that has the function of implementing the client behavior in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the client device includes a pre-search module and a sending module. These modules can perform the corresponding functions in the method example of the third aspect described above, as detailed in the method example description, and will not be repeated here.

[0080] Seventhly, embodiments of this application also provide a server-side device that has the function of implementing the server-side behavior in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the server-side device includes a receiving module and a searching module. These modules can perform the corresponding functions in the method example of the fourth aspect described above, as detailed in the method example description and will not be repeated here.

[0081] Eighthly, embodiments of this application also provide a storage device that has the function of implementing the storage cluster behavior in the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the storage device includes a storage management module, which can perform the corresponding functions in the method example of the fifth aspect described above. For details, please refer to the detailed description in the method example, which will not be repeated here.

[0082] Ninthly, this application also provides a computing device, which includes a processor and a memory, and may further include a communication interface. The processor executes computer program instructions stored in the memory to perform the method provided by the first aspect or any possible implementation thereof, or to perform the method provided by the third aspect or any possible implementation thereof, or to perform the method provided by the fourth aspect or any possible implementation thereof, or to perform the method provided by the fifth aspect or any possible implementation thereof. The memory is coupled to the processor and stores computer program instructions and data necessary for vector search access. The communication interface is used for communicating with other devices.

[0083] Tenthly, this application provides a computing device system including at least one computing device. Each computing device includes a memory and a processor. The processor of the at least one computing device is configured to access computer program instructions in the memory to execute the method provided by the first aspect or any possible implementation thereof, or to execute the method provided by the third aspect or any possible implementation thereof, or to execute the method provided by the fourth aspect or any possible implementation thereof, or to execute the method provided by the fifth aspect or any possible implementation thereof.

[0084] In one aspect, this application provides a computer-readable storage medium, which, when executed by a computing device, enables the computing device to execute the method provided in the first aspect or any possible implementation thereof, or to execute the method provided in the third aspect or any possible implementation thereof, or to execute the method provided in the fourth aspect or any possible implementation thereof, or to execute the method provided in the fifth aspect or any possible implementation thereof.

[0085] The storage medium stores computer program instructions. This storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid state drive (SSD).

[0086] In a twelfth aspect, this application provides a computing device program product, which includes computer program instructions that, when executed by a computing device, enable the computing device to perform the method provided in the first aspect or any possible implementation thereof, or to perform the method provided in the third aspect or any possible implementation thereof, or to perform the method provided in the fourth aspect or any possible implementation thereof, or to perform the method provided in the fifth aspect or any possible implementation thereof.

[0087] The computer program product may be a software installation package, which may be downloaded and executed on a computing device when it is necessary to use the methods provided in the first aspect or any possible implementation of the first aspect, the third aspect or any possible implementation of the third aspect, the fourth aspect or any possible implementation of the fourth aspect, or the fifth aspect or any possible implementation of the fifth aspect.

[0088] In a thirteenth aspect, this application also provides a computer chip connected to a memory, the chip being used to read and execute computer program instructions stored in the memory, to execute the methods in the first aspect and various possible implementations of the first aspect, or to execute the methods in the third aspect and various possible implementations of the third aspect, or the methods provided in the fourth aspect or any possible implementation of the fourth aspect, or to execute the methods provided in the fifth aspect or any possible implementation of the fifth aspect.

[0089] For the technical effects that can be achieved in aspects two through thirteen above, please refer to the description of the technical effects that can be achieved by the corresponding design scheme in aspect one above. This application will not repeat them here. Attached Figure Description

[0090] Figure 1 This is a schematic diagram of the structure of a vector index provided in an embodiment of this application;

[0091] Figure 2 This is a schematic diagram of the structure of a vector search system provided in an embodiment of this application;

[0092] Figure 3 This application provides a schematic diagram illustrating the functional division of a vector search system according to an embodiment of the present application.

[0093] Figures 4-7 , Figure 8A , Figure 8B The figure is a schematic diagram of the deployment of a server and a storage cluster provided in an embodiment of this application;

[0094] Figure 9 A schematic diagram of a vector search method provided in this application;

[0095] Figure 10 A visual interface diagram provided for this application;

[0096] Figure 11 A schematic diagram of the structure of a client device provided in this application;

[0097] Figure 12 A schematic diagram of the structure of a server device provided in this application;

[0098] Figures 13-14 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0099] Before introducing the vector search system, method, and device mentioned in the embodiments of this application, let's first introduce an important concept involved in the embodiments of this application—vector search:

[0100] For unstructured data such as images, documents, videos, and audio, there is a common search scenario: searching for data similar to or identical to the unstructured data to be searched within a dataset. To improve search speed, the unstructured data is converted into vectors, and the search for unstructured data is transformed into "vector search".

[0101] Vector search involves converting unstructured data into vectors and then searching for vectors similar to or identical to the target vector within a multidimensional space. These multidimensional vectors are the vectors obtained after transforming the unstructured data contained in the dataset. For ease of explanation, the set of vectors within this multidimensional space is called the vector set. Common applications of vector search include recommendation systems, image search, natural language processing, and voiceprint matching.

[0102] Vector search essentially involves calculating the similarity between vectors and identifying the top K vectors with the highest similarity to the vector being searched (i.e., the top K vectors in the vector set sorted by similarity from highest to lowest). There are many ways to calculate vector similarity. For example, for floating-point vectors, similarity can be obtained by calculating the inner product, Euclidean distance, or cosine distance. For binary vectors, similarity can be obtained by calculating Hamming distance, Jaccard distance, or Tanimoto distance.

[0103] Vector search can be understood as finding K similar vectors from a set of vectors to be searched. To accelerate vector search, the approximate nearest neighbor (ANN) algorithm is used to reduce the search range and improve speed. The implementation of the ANN algorithm relies on a vector index, which is a vector index constructed for vectors (i.e., the set of vectors) in a multidimensional space. Different ANN algorithms have different vector index structures.

[0104] The following section introduces several common approximate nearest neighbor search algorithms and vector indexing.

[0105] 1. Tree-based approximate nearest neighbor search algorithm.

[0106] In tree-based approximate nearest neighbor search algorithms, the multidimensional space is divided multiple times to form multiple subspaces. Based on the belonging relationships between these subspaces, a tree-like index similar to a binary tree is formed, and vector search is achieved with the help of the tree-like index.

[0107] Tree-based approximate nearest neighbor search algorithms mainly include the K-Dimensional Tree (KD) algorithm and the Annoy (Approximate Nearest Neighbors Oh Yeah) algorithm.

[0108] Let's take the KD-tree algorithm as an example. Before using the KD-tree algorithm for vector search, we need to construct a KD-tree (also known as a tree index).

[0109] The KD-tree construction process is as follows:

[0110] Step 1: For a set of vectors in K-dimensional space, calculate the variance of the coordinates of all vectors in the set in each dimension. Select the dimension with the largest variance as the splitting axis. Use the median of all coordinates of all vectors in the set as the splitting point to divide the hyperrectangular region into two sub-regions. Take this splitting point as the root node. From the root node, generate two nodes with a depth of 1. This depth represents the node's position in the tree index. The root node's depth can be considered as 0, representing the top-level node in the tree index. The nodes with a depth of 1 are the nodes at the second level of the tree index. One of these nodes with a depth of 1 is the node to the left of the root node (referred to as the left node), whose coordinates are less than the root node's; the other is the node to the right of the root node, whose coordinates are greater than the root node's. These two nodes with a depth of 1 represent the two sub-regions formed after the aforementioned split.

[0111] Step 2: For the node at depth i, select a splitting coordinate axis (the selection method is similar to that in Step 1, the difference being that here we refer to the variance of the coordinates of the vectors within the sub-region represented by the node in each dimension). Use the median of the coordinates of the vectors belonging to the sub-region represented by the node in the vector set as the splitting point, dividing the region into two sub-regions, generating a left node and a right node at depth i+1. The coordinates of the left node are less than the splitting point, and the coordinates of the right node are greater than the splitting point. Repeat Step 2 until no vectors from the vector set exist in the two split sub-regions.

[0112] Steps 1 and 2 can be used to form a tree-like index—a KD tree.

[0113] In implementing the KD-tree algorithm, starting from the root node of the KD-tree, the coordinates of the vector to be searched are compared with the nodes in the KD-tree, and the search is carried out layer by layer until the bottom node of the KD-tree is reached.

[0114] The Annoy algorithm uses a hyperplane to divide a high-dimensional space into multiple subspaces and arranges these subspaces into a tree structure to form a tree index, which is then used to perform vector search.

[0115] 2. Graph-based approximate nearest neighbor search algorithm.

[0116] In graph-based approximate nearest neighbor search algorithms, a graph is constructed for the vector set. Each node in the graph represents a vector in the set. The search range is narrowed by the connections between nodes in the graph, and similar or identical vectors are found within this narrowed search range. This graph serves as the vector index constructed by the approximate nearest neighbor search algorithm.

[0117] Graph-based approximate nearest neighbor search algorithms include: navigating small world (NSW) algorithm and hierarchical navigating small world (NSW) algorithm.

[0118] The implementation process of the graph-based approximate nearest neighbor search algorithm can be summarized as follows: Starting from a node in the graph, find the node with the smallest distance to the node corresponding to the vector to be searched from among the M nodes connected to that node. Then, starting from the determined nearest node, continue to search for the node with the smallest distance to the vector corresponding to the vector to be searched from among the M nodes connected to that nearest node, and repeat this process until no node with the smallest distance to the vector corresponding to the vector to be searched is found among the M nodes (that is, the nearest node is the node with the smallest distance to the vector to be searched among the nearest node and the M nodes connected to that node).

[0119] In the NSW algorithm, the graph construction process is as follows: Insert nodes into the graph one by one (a node represents a vector in the vector set). When inserting a new node into the graph, find the M nodes that are closest to the new node in the existing nodes of the graph (the value of M can be set by yourself), and connect the new node with the M nodes.

[0120] The process of vector search using the NSW algorithm is as follows: Select a node in the graph as the base node. Among the M nodes that are close to the base node, determine the nearest node and update the nearest node as the base node. Repeat the above process until the distance between the M nodes and the vector to be queried is no greater than the distance between the base node and the vector to be queried.

[0121] The HNSW algorithm is an improvement on the NSW algorithm. The HNSW algorithm uses a multi-level proximity graph. In the NSW algorithm, a graph is formed into a multi-level proximity graph according to preset rules. In this multi-level proximity graph, the nodes are sparser and farther apart in higher levels, and denser and closer together in lower levels. In adjacent levels, a point in the higher level corresponds to a node in the lower level, and these two nodes are connected by a line. For ease of explanation, the graph used in the HNSW algorithm can be referred to as a multi-level graph index.

[0122] The process of vector search using the NSW algorithm is as follows: Starting from the top-level graph, find the node in the current graph that is closest to the vector to be searched. This closest node serves as the entry point for the next level graph. Jump to the node corresponding to this closest node in the next level graph and continue searching for the closest node in the next level graph. Iterate in this way until the bottom-level graph is reached. In the bottom-level graph, find the K nodes that are closest to the vector to be searched.

[0123] 3. Implement approximate nearest neighbor search based on inverted file (IVF) index.

[0124] When performing approximate nearest neighbor search in a multidimensional space, an inverted index is constructed for the vector set, and the vectors in the set are clustered (by calling a clustering algorithm) to form multiple clusters. Each cluster covers a portion of the vectors in the set, and there is no overlap between the clusters.

[0125] The process of constructing an inverted file index is abstracted as follows: Each vector in the vector set is mapped to a two-dimensional space, and multiple additional points are set within this space. Any additional point can be the centroid of a cell in the two-dimensional space. Each additional point is called a centroid. Each centroid extends outward by an equal radius, and the point where the circumference of each circle contacts the circumference of the circles formed by the extensions of the other centroids forms the cell edge. The points formed by mapping each vector in the vector set to the two-dimensional space are distributed across these multiple cells. These cells are the aforementioned clusters, where the distance between each vector within a cluster and the vector of the centroid is less than a specific value. The above explanation uses a single-level index in the inverted file index as an example. For any cell within this level, multiple additional points are set within that cell. Any additional point can be the centroid of a cell within that cell. For ease of distinction, the cells within this cell are called sub-cells of that cell. The formation of sub-cells is similar to the form of cells; see the previous explanation for details. Each cell's sub-cells constitute the lower-level index within that level of the index. In an inverted index with a multi-level structure, the radius of a cell is larger in higher-level indexes and smaller in lower-level indexes. The points formed by mapping each vector in a vector set to two-dimensional space will be distributed in the lowest-level cells.

[0126] As can be seen, during the construction of the inverted file index, vectors that are close to each other in the vector set are grouped into one cluster, while those that are far apart are distributed in different clusters. Therefore, if we want to find vectors similar to the vector to be searched in the vector set, we can first locate the clusters, and then find similar vectors from the clusters.

[0127] After constructing the inverted index file, for a single-level inverted index file, the vector to be searched is mapped to a point in two-dimensional space (referred to as the search point). The search point will be located within a cell (i.e., a cluster). In this case, the search range will be limited to the vectors represented by the points within that cell (or cluster). The search point is then compared with each point within that cell, and the distance between the search point and each point in that cell is calculated. The vectors represented by the K points within the cell with the smallest distance to the search point are the K nearest similar vectors to the search point. For a multi-level inverted index file, the search point will still be located within a cell (i.e., a cluster). In this case, the search range will be limited to that cell (or cluster). After locating the cell, the search point will be located in a sub-cell of that cell, and the search range will be further limited to that sub-cell (or cluster). Repeat the above process until the bottom cell is located. In the located cell, compare the point to be searched with each point in the cell and calculate the distance between the point to be searched and each point in the cell. The vectors represented by the K points in the cell with the smallest distance to the point to be searched are the K nearest similar vectors to the point to be searched.

[0128] As can be seen from the foregoing, the vector index used in the approximate nearest neighbor search algorithm can have a layered structure, such as KD tree, multi-layer graph index, or IVF index. The more layers the vector index has, the higher the vector search efficiency of the approximate nearest neighbor search algorithm.

[0129] This section further explains the structure of vector indices with a layered structure, such as... Figure 1 The diagram shows the structure of a layered index, which includes multiple index layers, each containing multiple index nodes. For any two adjacent index layers, an index node in the upper layer is the parent node of one or more index nodes in the lower layer; and one or more index nodes in the lower layer are the child nodes of an index node in the upper layer.

[0130] The meaning of an index node varies depending on the type of vector index. For example, in a KD-tree, the index node is a node within the KD-tree; in a multi-level graph index, the index node is a node within the graph; and in an IVF index, the index node is a cell (or the centroid of a cell).

[0131] Based on the foregoing descriptions of various approximate nearest neighbor search algorithms, the search process based on vector indexes can be summarized as follows: starting from the topmost index layer of the vector index, the search proceeds layer by layer, performing a search operation at each layer. The search operation performed in the i-th index layer involves searching within the set of target index nodes in that layer using the vector to be searched. These target index nodes include one or more index nodes, and the target index node is determined from this set. The child nodes of this target index node in the (i+1)-th index layer constitute the set of target index nodes to be searched in that (i+1)-th index layer.

[0132] like Figure 2 The diagram shown illustrates a vector search system architecture applicable to an embodiment of this application. The vector search system includes a client 10 and a server 20. Optionally, the server 20 is connected to a storage cluster 220.

[0133] The vector search system provides vector search services, capable of retrieving K vectors similar to the target vector (for simplicity, the target vector) based on a vector index. Client 10, deployed close to the user, receives vector search requests, which request the K vectors in the vector set that are similar to the target vector. Server 20 stores the vector index and the vector set, and can retrieve the K vectors similar to the target vector based on the vector index.

[0134] In this embodiment, the vector index stored on the server side 20 is a global index, which is a single index built for the entire vector set. That is, only one vector index exists on the server side 20, and when performing a vector search, only this single global index is needed to determine the K vectors. For ease of explanation, the vector index on the server side 20 is referred to as the global index.

[0135] A global index is an index built for each vector in a vector set. This global index has a layered structure, such as the KD-tree, IVF index, and multi-level graph index mentioned above. Therefore, the server 20 can determine K vectors similar to the target vector from the vector set based on the global index.

[0136] 1) Client 10.

[0137] In this embodiment, the client 10 stores a multi-layered index in the global index, which is the first M layers of the global index, where M is a positive integer. For ease of explanation, the M layers of index stored on the client 10 are referred to as the upper-layer index.

[0138] Since client 10 stores the upper-level index, when client 10 receives a vector search request, client 10 can perform a pre-search based on the upper-level index to obtain the pre-search results. These pre-search results indicate the target index node located after the search based on the upper-level index. Client 10 sends a vector search instruction to the search node 210 on server 20. This vector search instruction is used to instruct the search for K vectors in the vector set that are similar to the target vector. Optionally, this vector search instruction carries the pre-search results.

[0139] In this embodiment, from the perspective of the execution entity of the vector search, the search process based on the global index is divided into two types: pre-search and vector search executed on the client 10 side. For ease of explanation, the vector search executed on the client 10 side is referred to as pre-search. Pre-search is a search process based on the upper-level index of the global index. The other type is vector search executed on the server 20 side, which is referred to as formal search. After the formal search is executed by the server 20, K vectors in the vector set that are similar to the target vector can be determined.

[0140] The results of the pre-search obtained after the pre-search is performed by client 10 can be used in part or in all of the following ways:

[0141] First, the client 10 informs the server 20 of the results of the preliminary search, such as the search node 210 in the server 20 that performs the formal search. In this embodiment, the search node 210 that performs the formal search is called the target search node.

[0142] Since the client 10 stores the upper-level index of the global index, the client 10 can perform some vector search operations (i.e., pre-search). When the search node 210 on the server 20 performs the formal search, it does not need to perform all vector search operations. That is, the search on the server 20 does not need to start the vector search from the top level of the global index. It only needs to start the search from the target index node in the global index that is the result of the pre-search. This can effectively improve the vector search efficiency of the search node 210.

[0143] Second, client 10 determines the target search node from multiple search nodes 210 in server 20 according to the pre-search rules. The pre-search rules are the rules for selecting the target search node based on the results of the pre-search.

[0144] Client 10 can use the results of the pre-search to specify the target search node from multiple search nodes 210 in server 20 to perform the formal search, achieving load balancing. That is, when client 10 receives multiple vector search requests, it can perform a pre-search for each request. For subsequent formal searches, client 10 can specify different search nodes 210 to perform those searches.

[0145] In practical applications, client 10 can also use other methods to determine the target search node. For example, client 10 can use a polling method to determine the target search result from the multiple search nodes 210.

[0146] 2) Server 20.

[0147] The server 20 includes multiple search nodes 210, each with vector search capabilities. Upon receiving a vector search instruction from the client 10, any search node 210 performs a search based on the global index to determine the K vectors in the vector set that are similar to the target vector. Since the client 10 has already performed a search using the upper-level index, the search node 210 can then perform a search based on the lower-level index of the global index to determine the K vectors in the vector set that are similar to the target vector. Of course, the search node 210 can also perform a search based on the global index during the actual search, that is, starting the search from the top level of the global index.

[0148] In this embodiment of the application, the server 20 has the following characteristics:

[0149] (1) Index caching feature.

[0150] For any search node 210 of the server 20, a memory space is configured for the search node 210, and the search node 210 can cache data in the memory space.

[0151] In this embodiment, the search node 210 can cache index data from the global index whose access frequency exceeds a frequency threshold in the memory space. This index data can be a portion of the index layers or a portion of the index nodes in the global index. This embodiment does not limit the specific form of the index data. For ease of explanation, the index data cached in the memory space is referred to as the first index data. That is, the search node 210 can cache the first index data from the global index in the memory space.

[0152] This introduces the concept of "index data required to perform a formal search." Index data refers to the target index node and its set located during the search process based on the global index. For example, when using HNSW for vector search, the global index is one or more layers in a multi-level graph. The index data required for the formal search consists of the m nodes in each layer of the lower-level index whose similarity needs to be determined. Similarly, when the global index is an IVF index, the lower-level index is the index layer in the IVF index file. The index data required for the formal search consists of the cells to be searched in each layer of the lower-level index.

[0153] When the target search node receives a vector search instruction from client 10, it executes the formal search. During the formal search, if all the index data required for the formal search is cached in memory, the target search node can directly search based on the cached index data. If some of the index data required for the formal search is cached in memory, the target search node searches based on the first index data cached in memory. For the index data required for the formal search that is not cached in memory (for ease of explanation, this index data required for the formal search that is not cached in memory is referred to as the second index data), the target search node can obtain the second index data from storage cluster 220 and execute the formal search based on the second index data (and the first index data).

[0154] Furthermore, the search node 210 on the server side 20 can perform a data prefetching operation, meaning that the search node 210 can cache index data that may be used later in advance in the memory space. For ease of explanation, in this embodiment, the index data that the search node 210 obtains from the storage node in advance is referred to as prefetched index data. This embodiment does not limit the timing of the search node 210 performing the data prefetching operation. For example, taking the target search node as an example, the target search node can obtain the prefetched index data that may be used in the next vector search from the storage cluster 220 in advance during the execution of the formal search, and cache the prefetched index data. As another example, the target search node can use the index data adjacent to the cached first index data in the global index as prefetched index data, and obtain the prefetchable index data from the storage cluster 220. For example, when the target search node retrieves the second index data from the storage node, it can also retrieve the index data adjacent to the second index data in the global index (in this embodiment, the index data adjacent to the second index data in the global index is referred to as the third index data). The third index data is the pre-fetched index data, and the target index node can cache the third index data in the memory space.

[0155] When the target search node retrieves prefetched index data (such as index data adjacent to the first index data, or the third index data), it sends a prefetch request to the storage node. This prefetch request is used to request the retrieval of the prefetched index data.

[0156] Taking the acquisition of third index data as an example, the target index node sends an expected request to the storage node. This expected request is used to request the acquisition of both the third index data and the second index data. In other words, the target index node can acquire both the second index data and the third index data with a single request.

[0157] This application does not limit the specific form of the search node 210. The search node 210 can be a hardware device, or a single computing device. The search node 210 can be a hardware component within the computing device, such as a processor (e.g., a central processing unit (CPU), data processing unit (DPU)), offloading card, accelerator card, etc. The search node 210 can also be a software device, such as a container, virtual machine, or other computing instance running on the computing device.

[0158] (2) Storage sharing feature.

[0159] The server side is also connected to a storage cluster 220, which has storage capabilities and can store a global index. In this embodiment, any search node 210 on the server side 20 can access the storage cluster 220 to retrieve the global index (such as retrieving second index data or first index data). In other words, multiple search nodes 210 on the server side 20 share the storage space provided by the storage cluster 220, and each search node 210 has access permissions to the storage cluster 220.

[0160] From the perspective of storage cluster 220, the storage cluster 220 also includes a storage management module, which can feed back the index data requested by search node 210 to the storage node according to the request of search node 210.

[0161] Taking the target search node as an example, when the target search node retrieves the second index data from the storage cluster 220, it can send a data read request to the storage cluster 220. This data read request is used to request the reading of the second index data. After receiving the data read request, the storage management module retrieves the second index data from the storage device and sends the second index data back to the target search node.

[0162] While the target search node is prefetching third index data adjacent to the second index data from the global index in storage cluster 220, for example, the target search node can prefetch the third index data simultaneously with the second index data. The target search node can send a prefetch request to storage cluster 220, requesting to read the second index data and prefetch the third index data. Upon receiving the prefetch request, the storage management module retrieves the second and third index data from the storage device and returns the second and third index data to the target search node.

[0163] This application does not limit the specific form of the storage management module. The storage management module can be a hardware module, such as a smart network interface card, computing accelerator card, central processing unit, hard disk controller, or other computing power unit in a storage node. The storage management module can also be a software module, which can be an application program running on the computing power unit.

[0164] Given this shared storage feature, a single global index needs to be stored in storage cluster 220 to support the search function of each search node 210, which can effectively save storage space in storage cluster 220.

[0165] 3) Storage cluster 220.

[0166] Storage cluster 220 includes multiple storage nodes to support persistent storage. Storage nodes in storage cluster 220 refer to nodes with storage capabilities. This application embodiment does not limit the specific form of the storage node; it can be a computing device with storage capabilities, a storage array, or a storage device such as a hard disk. In this application embodiment, the global index and vector set can be distributed and stored across these multiple storage nodes.

[0167] This application does not limit the specific form of the storage cluster 220. For example, the storage cluster 220 can be a single computing device with storage capabilities, which includes multiple external persistent storage devices (hereinafter referred to as storage devices). The persistent storage devices can be storage arrays or hard disks and other storage devices. In this form, the storage node can be the storage device. Alternatively, the storage cluster 220 can be distributed across multiple computing devices with storage capabilities, with each computing device serving as a storage node.

[0168] As described above, the storage cluster 220 also includes a storage management module. The functions of this storage node management module can be found in the aforementioned description and will not be repeated here.

[0169] In this embodiment of the application, in addition to providing storage space to the server 20, the storage cluster 220 can also provide computing power to the server 20. Since the computing power in the storage cluster 220 is closer to the data, this computing power can also be called near-memory computing power. The meaning of "near-memory computing power" will be explained below.

[0170] The storage cluster 220 includes one or more computing units, which are units within the storage cluster 220 that have data processing capabilities. This application embodiment does not limit the specific type of the computing unit, such as the storage array controller, computing accelerator card, central processing unit, hard disk controller, or smart network card. Different specific forms of the storage cluster 220 will result in different numbers and types of computing units included within it.

[0171] Storage cluster 220 can not only provide storage space, but also provide computing power to search node 210 to assist search node 210 in performing formal searches.

[0172] In this embodiment of the application, the target search node may instruct the storage cluster 220 to perform search operations during the formal search process. The search operations that the target search node instructs the storage cluster 220 to perform may be part or all of the operations in the formal search.

[0173] Here, taking the example where the target search node instructs storage cluster 220 to perform all operations in the formal search, the target search node can provide the index data required for the formal search (referred to as the fourth index data). Storage cluster 220 uses this fourth index data to perform the formal search and then returns the vector search results (i.e., K vectors similar to the target vector) to the target search node. From a message interaction perspective, the target search node can send a first instruction to storage cluster 220, which instructs to perform a vector search based on the fourth index data in the global index, and then obtain the vector search results returned by storage cluster 220.

[0174] Within the storage cluster 220, after receiving the first instruction, the storage management module can select a target computing unit from one or more included computing units. This target computing unit can then perform a vector search based on the fourth index data and obtain the search result. The storage management unit can then report the vector search result back to the target search node.

[0175] This application embodiment does not limit the method of selecting target computing power units within the storage cluster 220. For example, the storage cluster 220 can determine the target computing power unit according to the near data strategy. The near data strategy includes selecting the target computing power unit based on the storage location of the fourth index data. That is, the storage cluster 220 can select the computing power unit closest to the fourth index data as the target computing power unit.

[0176] As can be seen from the above description of client 10 and server 20, client 10 and server 20 perform different functions. In order to more clearly understand the functions of client 10 and server 20, the vector search system is divided from a logical perspective.

[0177] like Figure 3 The diagram shown illustrates the hierarchical division of a vector search system according to an embodiment of this application. The vector search system is divided into four layers: a client view layer (first layer), a cache layer (second layer), a storage management layer (third layer), and an index storage layer (fourth layer).

[0178] The client-side view layer corresponds to client 10 in the vector search system. When a vector search request arrives at the client-side view layer, client 10 performs a pre-search and obtains the pre-search results. Client 10 then determines the target search node from among the multiple search nodes 210 in server 20 to perform the formal search. Client 10 then sends a vector search instruction to this target search node.

[0179] The cache layer corresponds to the search node 210 in the vector search system and the memory space of the search node 210. The vector search instruction is transmitted to the target search node in the cache layer. The target search node performs a vector search based on the cached first index data in the memory space to execute the formal search.

[0180] If the first index data contains all the index data required to perform the formal search, the target search node obtains the formal search results based on the first index data. The formal search results indicate the K vectors in the vector set that are similar to the target vector.

[0181] If the first index data is only a portion of the index data required to perform the formal search, the target search node can send a data read instruction to the storage management layer (i.e., storage cluster 220). This instruction requests the reading of the second index data, which the target search node then retrieves. Based on this first and second index data, the target search node obtains the formal search results. Alternatively, the target search node can instruct the storage management layer (i.e., storage cluster 220) to perform part or all of the formal search operations. In other words, the target search node can utilize the computing power of storage cluster layer 220 to complete the formal search.

[0182] The storage management layer and index storage layer correspond to storage cluster 220 in the vector search system. The storage management layer represents the management functions within storage cluster 220, and it is supported by the processors within storage cluster 220. Within this storage management layer, the storage management layer can process data read commands from the target search node, retrieve second index data from the index storage layer, transmit the second index data to the target search node, and, based on the instructions of the target search node, execute part or all of the formal search operations, and provide feedback on the execution results to the target search node.

[0183] The index storage layer represents the storage functionality within the storage cluster 220, and it is supported by the storage devices within the storage cluster 220. This index storage layer stores a global index and can provide index data (such as second index data and first index data) to the storage management layer.

[0184] The above has introduced the client 10 and server 20 from a functional perspective. The following describes the form of the client 10 and server 20.

[0185] I. The form of client 10.

[0186] Client 10 is deployed close to the user. Client 10 can be a hardware device, such as a server, mobile terminal, desktop computer, portable computer, or other computing device. Client 10 can also be a software module, such as a software program running on the user's device, or a computing instance formed through virtualization technology.

[0187] Virtualization is a resource management technology that abstracts and transforms various physical resources of a host, such as processors, memory, and interfaces, and presents them in a more accessible form. Virtualization is a resource allocation method from a logical perspective; it is a logical abstraction of physical resources.

[0188] With the help of virtualization technology, a host can form a software module with an independent running environment. In this embodiment, the software module with an independent running environment formed on the host is referred to as a computing instance. This computing instance can be a virtual machine or a container.

[0189] A virtual machine (VM) is a "complete computer" simulated using virtualization technology, possessing full hardware system functionality and running in a completely isolated environment. Any task that can be performed on a physical computer can also be performed in a virtual machine. A virtual machine has components such as a processor (also called a virtual processor) and memory (also called virtual memory), which are virtualized from the host machine's processor and memory. An operating system is installed on the virtual machine, and this operating system is independent of the host machine's operating system. To distinguish between these two different operating systems, the host operating system is usually called the host operating system (hostOS), and the virtual machine operating system is usually called the guest operating system (guestOS).

[0190] A container is an independent runtime environment simulated using virtualization technology. A container is similar to a lightweight sandbox, shielding external software and hardware. Containers implement virtualization at the operating system level, directly reusing the host's operating system. Similar to virtual machines, containers also possess virtual hardware components such as virtual processors and virtual memory, which are virtualized from the host's processor, memory, and other components.

[0191] From the host's perspective, a compute instance is viewed as a special kind of "process." This "process" performs computational tasks and consumes the host's resources such as processor, memory, and hard drive.

[0192] For example, the host operating system will configure a dedicated memory space for the compute instance, and the compute instance will occupy this memory space to support the computing tasks that the compute instance needs to perform.

[0193] II. The form of server 20.

[0194] For server 20, it needs to have both vector search function and storage access function. Server 20 can be a single computing device or a computing device cluster containing multiple computing devices.

[0195] The following are some examples of server-side 20 configurations.

[0196] In the first scenario, the server 20 exists as a host. Multiple computing instances are created on this host using virtualization technology. Each computing instance can act as a search node 210. The host is connected to at least one storage device.

[0197] like Figure 4 The diagram shown illustrates the virtualization architecture of a host 300 according to an embodiment of this application. The virtualization architecture of the host 300 includes underlying hardware 340, a host operating system 330, a computing instance management unit 320, and at least one computing instance 310. The host 300 is a computing device. The host 300 can be a server, mobile terminal, tablet computer, or other computing device.

[0198] The underlying hardware 340 refers to some hardware components in the host 300, such as the processor 301, memory 302, input / output (I / O) interfaces, etc. The underlying hardware 340 can be understood as the physical resources on the host 300, which are objects that need to be virtualized in virtualization technology.

[0199] Based on the underlying hardware 340, the host 300 runs software modules such as the host operating system 330, the computing instance management unit 320, and the computing instance 310.

[0200] The host operating system 330 runs on the processor 301 and is used to implement the basic functions of the host 300. These basic functions include, but are not limited to: management functions for the underlying hardware 340, control functions for input / output devices (such as monitors, keyboards, mice, etc.) connected to the host 300, and management of processes within the host 300. This embodiment mainly involves the management functions for the underlying hardware 340. Taking the management of the host operating system 330's memory 302 on the host 300 as an example, the host operating system 330 can monitor the memory occupancy of 302 and allocate memory 302 space for the computing instance 310.

[0201] The compute instance management unit 320 manages the compute instance 310. This unit can virtualize the underlying hardware 340 to provide a runtime environment for the compute instance 310. There are various ways the compute instance management unit 320 can provide a runtime environment to the compute instance 310. For example, it can simulate a runtime environment for the compute instance 310 through software. Alternatively, the compute instance 310 can directly access the underlying hardware 340 (such as memory 302).

[0202] When compute instance 310 is a virtual machine, the compute instance management unit 320 includes QEMU (quick emulator) and a virtual machine monitor (VMM). When compute instance 310 is a container, the compute instance management unit 320 can be a container engine.

[0203] In some possible scenarios, the compute instance management unit 320 can be built into the host operating system 330. In this scenario, the host operating system 330 possesses the functionality of the compute instance management unit 320.

[0204] Computing instance 310 has an independent operating environment. It occupies the physical resources of host 300 and can perform various operations (such as pre-search) and implement related services (such as vector search services) based on the occupied physical resources. Computing instances 310 are independent of each other and do not affect one another. In this embodiment, the computing instance 310 can be a virtual machine, a container, or other modules formed by virtualization using the physical resources of host 300.

[0205] In this embodiment, the computing instance 310 has a vector search function, and the computing instance 310 can be understood as the search node 210 on the server side 20 mentioned above. Each computing instance 310 has its own memory space, which may cache the first index data. After receiving a vector search instruction from the client 10, the computing instance 310 performs a formal search according to the vector search instruction. If the first index data contains all the index data required to perform the formal search, the computing instance 310 performs the formal search according to the first index data, obtains the vector search result, and feeds back the vector search result to the client 10.

[0206] If the first index data does not represent all the index data required to perform the formal search, compute instance 310 can obtain second index data from storage device 304 in storage cluster 220 through compute instance management unit 320, perform a formal search based on the first and second index data, obtain the vector search result, and report the vector search result back to client 10. Compute instance 310 can instruct processor 301 (such as the storage management module in processor 301) in host 300 to perform part or all of the formal search operations through compute instance management unit 320 to complete the formal search, obtain the vector search result, and report the vector search result back to client 10.

[0207] The hardware components included in the host 300 are described below. As shown in Figure 5, which is a structural schematic diagram of a host 300 provided in an embodiment of this application, the host 300 includes a network card 303, a processor 301, and a memory 302. The host 300 is also connected to a persistent storage device 304 (in this embodiment, the persistent storage device 304 can be simply referred to as storage device 304). The network card 303, the processor 301, the memory 302, and the storage device 304 can be connected via a system bus. This system bus can be a peripheral component interconnect express (PCIe) bus, or a compute express link (CXL), universal serial bus (USB) protocol, or a bus of other protocols.

[0208] The network interface card 303 is used to communicate with devices located outside the host 300. For example, it can receive data sent by devices outside the host 300 or send data to devices outside the host 300 through the network interface card 303.

[0209] Processor 301 is the computing and control core of host 300. It can be a central processing unit (CPU) or other specific integrated circuits. Processor 301 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. This application embodiment does not limit the number of processors 301 included in host 300.

[0210] Memory 302 is typically used to store computer program instructions related to the host operating system 330, the computing instance management unit 320, and the computing instance 310, as well as data generated by the host operating system 330, the computing instance management unit 320, and the computing instance 310 during operation (such as first index data). Memory 302 has the advantage of high access speed. Memory 302 is typically a dynamic random access memory (DRAM). In addition to DRAM, memory 302 can also be other random access memories, such as static random access memory (SRAM) and storage class memory (SCM). Alternatively, memory 302 can also be a read-only memory (ROM). For example, a read-only memory could be a programmable read-only memory (PROM) or an erasable programmable read-only memory (EPROM). Memory 302 can also be a dual in-line memory module (DIMM), flash memory, hard disk drive (HDD), or solid state disk (SSD), etc.

[0211] Processor 301 is connected to memory 302 via a double data rate (DDR) bus or other type of bus. Processor 301 can call computer program instructions in memory 302 to form software modules such as host operating system 330, computing instance management unit 320, and computing instance 310 in host 300. Processor 301, as the computing and control core of host 300, supports the formal search performed by computing instance 310 in the embodiment shown in the figure.

[0212] In this embodiment, the processor 301 is also capable of managing the storage device 304. A storage management module is deployed on the processor 301. This storage management module can be circuit logic within the processor 301 or a software module running on the processor 301 for managing the storage device 304. This software module can be part of the host operating system 330 or a separate software module from the host operating system 330. The processor 301 can read second index data from the storage device 304 under the instruction of the computing instance 310 and provide the second index data to the computing instance 310. The processor 301 can also select a target computing unit under the instruction of the computing instance 310, perform part or all of the formal search operations, and provide the execution results back to the computing instance 310.

[0213] Storage device 304 refers to the persistent storage device 304 in host 300. This storage device 304 stores a global index and a vector set, and is connected to host 300 via the system bus. This storage device 304 can be a non-volatile memory such as ROM, flash memory, HDD, or SSD.

[0214] Although not shown, the host 300 may also include a computing accelerator card, which is a hardware device deployed within the host 300 to provide additional computing power. Furthermore, as mentioned above, the network interface card 303 can be a smart network interface card (NIC). A "smart NIC" refers to a NIC with data processing capabilities; a smart NIC not only has the functions of a NIC but can also provide a certain amount of computing power support.

[0215] In this hardware configuration, corresponding to Figure 2 The storage cluster 220 shown is... Figure 5 In the host 300 shown, the storage cluster 220 may include the processor 301 (such as the storage management module in the processor 301) and the storage device 304 of the host 300, and may also include a computing accelerator card and a smart network card. The storage device 304 can be regarded as a storage node in the storage cluster 220.

[0216] That is to say, in Figure 5 In the host 300 shown, the computing units in the storage cluster 220 include some or all of the processor 301, smart network card, computing acceleration card, and controller (such as hard disk controller) in the storage device 304.

[0217] In the second configuration, the server 20 exists as a single computing device, which includes multiple processors based on a non-uniform memory access architecture (NUMA). Each of the aforementioned search nodes 210 includes a processor and its local memory. The computing device is connected to multiple storage devices.

[0218] NUMA is a multiprocessor computer architecture. In a NUMA-based computing device, each processor is equipped with its own memory. Besides accessing its own dedicated memory, each processor can also access the memory of other processors. When the computing device starts up, the memory closest to the processor (i.e., the memory allocated to the processor) is designated as local memory, while memory farther away (such as the memory of other processors) is designated as remote memory. In NUMA, processors and local memory have affinity, and because local memory is closer to the processor, accessing local memory is faster than accessing remote memory.

[0219] like Figure 6 The diagram shown is a structural schematic of a computing device 400 provided in an embodiment of this application. In terms of hardware, the computing device 400 includes at least a network interface card (NIC) 403, multiple processors 401, and local memory 402 for each processor 401. For any processor 401, the processor 401, its local memory 402, the NIC 403, and the storage device 404 are connected via a bus. The computing device 400 is also connected to the storage device 404.

[0220] The computing device 400 includes a plurality of processors 401, each processor 401 being configured with local memory 402. The embodiments of this application do not limit the number of local memory 402 for each processor 401. Each processor 401 may be configured with one local memory 402 or multiple local memory 402.

[0221] The specific types of processor 401, local memory 402, network card 403, and storage device 404 are... Figure 5 The specific types of processor 301, memory 302, network card 303 and storage device 304 are similar, and can be found in the foregoing description, which will not be repeated here.

[0222] Although not shown, the computing device 400 may also include a computing accelerator card, and the aforementioned network card 403 may be a smart network card 403.

[0223] In this computing device 400, each processor 401 and its local memory 402 can perform the functions of the search node 210. That is, the processor 401 can receive vector search instructions from the client 10 and execute a formal search according to the vector search instructions. The local memory 402 of the processor 401 can store first index data. If the first index data contains all the index data required to execute the formal search, the processor 401 executes the formal search according to the first index data, obtains the vector search result, and feeds back the vector search result to the client 10.

[0224] If the first index data does not contain all the index data required to perform the formal search, the processor 401 can obtain the second index data from the storage device 404, perform the formal search based on the first and second index data, obtain the vector search result, and feed the vector search result back to the client 10. The processor 401 can instruct the storage device 404 to perform part or all of the formal search operations to obtain the vector search result and feed the vector search result back to the client 10.

[0225] In this computing device 400, each processor 401 can access the storage device 404 to achieve shared storage. Each processor 401 can also have management functions for the storage device 404, such as monitoring the storage space in the storage device 404, writing data to the storage device 404, or reading data from the storage device 404. The storage device 404 has storage space for storing a global index. Corresponding to... Figure 2 The storage cluster 220 shown is... Figure 5 In the computing device 400 shown, the storage cluster 220 may include a processor 401 and a storage device 404, wherein the processor 401 is used to perform storage management functions, and the storage device 404 performs storage functions. Optionally, the storage cluster 220 may also include a computing accelerator card and a smart network card. The storage device 404 can be regarded as a node in the storage cluster 220.

[0226] Of course, in practical applications, a storage management module independent of the processor 401 can also be deployed in the computing device 400. This storage management module undertakes the management functions for the storage device 404. When the processor 401 needs to access the storage device 404, it sends a data access command to the storage management module. This data access command is used to access data (such as first index data or second index data) in the storage device 404. After receiving the data access command, the storage management module accesses the data in the storage device 404 according to the data access command, such as retrieving the first index data or the second index data from the storage device 404. Corresponding to Figure 2 The storage cluster 220 shown is... Figure 6In the computing device 400 shown, the storage cluster 220 may include a storage management module and a storage device 404. The storage management module is used to perform storage management functions, and the storage device 404 performs storage functions. Optionally, the storage cluster 220 may also include a computing accelerator card and a smart network card. The storage device 404 can be regarded as a node in the storage cluster 220.

[0227] exist Figure 6 In the computing device 400 shown, the computing units in the storage cluster 220 include some or all of the processor 401, smart network card, computing accelerator card, and controller (such as hard disk controller) in the storage device 404.

[0228] Form 3: Server 20 is located in a distributed storage system. Search node 210 can be a compute node in the distributed storage system, a compute instance running on the compute node, or a processor in the compute node. Storage cluster 220 is the cluster of storage nodes in the distributed storage system.

[0229] like Figure 7 The diagram shown is a schematic of the system architecture of a distributed storage system provided in an embodiment of this application. In this distributed storage system, the computing and storage functions required by the distributed storage system are deployed on different nodes. The node with computing function is called computing node 500, and the node with storage function is called storage node 600.

[0230] exist Figure 7 In this distributed storage system, there are compute node clusters and storage node clusters. The compute node cluster includes one or more compute nodes (500...). Figure 7 The diagram shows three compute nodes 500 (but is not limited to two compute nodes 500), and the compute nodes 500 can communicate with each other. A compute node 500 is a computing device, such as a server, desktop computer, or controller of a storage array.

[0231] Compute node 500 has computing capabilities and can receive and process data requests sent from outside the storage system. If the data request is for accessing a vector in a vector set, compute node 500 can send a data access request to storage node 600 to request the vector to be retrieved from storage node 600 and written into storage node 600.

[0232] In this embodiment, the computing node 500 can receive vector search instructions from the client 10 and perform a formal search according to the instructions. The memory of the computing node 500 can store first index data. If the first index data contains all the index data required to perform the formal search, the computing node 500 performs the formal search based on the first index data, obtains the vector search result, and feeds back the vector search result to the client 10.

[0233] If the first index data does not contain all the index data required to perform the formal search, the compute node 500 can obtain the second index data from the storage node 600, perform the formal search based on the first and second index data, obtain the vector search result, and report the vector search result back to the client 10. The compute node 500 can instruct the storage node 600 to perform part or all of the formal search operations to obtain the vector search result and report the vector search result back to the client 10.

[0234] Specifically regarding the internal structure of compute node 500, two internal structures of compute node 500 are listed here.

[0235] The first type, such as Figure 8A The diagram shown is a virtualization architecture diagram of a computing node 500 provided in an embodiment of this application. The virtualization architecture of the computing node 500 includes underlying hardware 540, host operating system 530, computing instance management unit 520, and at least one computing instance 510.

[0236] Regarding the functions of the underlying hardware 540, host operating system 530, computing instance management unit 520, and at least one computing instance 510... Figure 4 The underlying hardware 340, host operating system 330, computing instance management unit 320, and at least one computing instance 310 shown are similar, as detailed in the foregoing description, and will not be repeated here.

[0237] Figure 7 and Figure 8A The combined deployment form of server 20 and its relationship with Figure 4 In the deployment configuration of server 20 shown, the search node 210 on the server side exists as a compute instance. The deployment of storage cluster 220 is different; the difference lies in... Figure 7 and Figure 8A In the combined deployment configuration of server 20, storage cluster 220 is deployed within a storage node cluster. In this configuration, storage cluster 220 includes a storage node cluster. The computing units within storage cluster 220 include some or all of the processor, smart network interface card, computing accelerator card, and controllers (such as hard disk controllers) within the storage devices.

[0238] Figure 4 In the deployment configuration of server 20 shown, storage cluster 220 is deployed on the host, with its storage management and data storage functions respectively handled by the processor and storage devices. In this configuration, storage cluster 220 includes the host's processor (or storage management module) and storage devices, and may also include computing accelerator cards and smart network interface cards (NICs). The computing units in storage cluster 220 include some or all of the processor, smart NIC, computing accelerator cards, and controllers (such as hard disk controllers) within the storage devices.

[0239] The second type, such as Figure 8B The diagram shown is a structural schematic of a computing node 500 provided in an embodiment of this application. The computing device includes multiple NUMA processors 501, local memory 502 for each processor, network interface card 503, and storage device 504.

[0240] The specific types of processor 501, local memory 502, network card 503, and storage device 504 are... Figure 6 The specific types of the processor 401, local memory 402, network card 403, and storage device 404 are similar, and can be found in the foregoing description, so they will not be repeated here.

[0241] Figure 7 and Figure 8B The combined deployment form of server 20 and its relationship with Figure 4 In the deployment configuration of server 20 shown, the search node 210 on the server side exists as a processor and its local memory. The deployment of storage cluster 220 is different; the difference lies in... Figure 7 and Figure 8A In the combined deployment configuration of server 20, storage cluster 220 is deployed within a storage node cluster. In this configuration, storage cluster 220 includes a storage node cluster. The computing units within storage cluster 220 include some or all of the processor, smart network interface card, computing accelerator card, and controllers (such as hard disk controllers) within the storage devices. Figure 4 In the deployment configuration of server 20 shown, storage cluster 220 is deployed on computing devices, with its storage management and storage functions respectively housed in the processor and storage devices. In this configuration, storage cluster 220 includes the host's processor (or storage management module) and storage devices, and may also include computing accelerator cards and smart network interface cards (NICs). The computing units in storage cluster 220 include some or all of the processor, smart NIC, computing accelerator cards, and controllers (such as hard disk controllers) within the storage devices.

[0242] The following is an introduction to the 600 storage node cluster, which consists of multiple 600 storage nodes. Figure 7The diagram shows three storage nodes 600, but is not limited to three storage nodes 600. Each storage node 600 includes one or more control units 601, a network interface card (NIC) 602, and multiple storage devices 603. The NIC 602 is used to communicate with the computing node. The storage devices 603 are used to store data and can be disks or other types of storage media, such as solid-state drives (SSDs) or shingled magnetic recording hard drives (SMADs).

[0243] The specific types of the control unit 601 (equivalent to the processor in the storage node 600), the network card 602 and the storage device 603 are similar to the specific types of the processor 301, network card 303 and storage device 304 mentioned above. For details, please refer to the above description and it will not be repeated here.

[0244] The control unit 601 is the control core in the storage node 600. The control unit 601 can manage each storage device 603 in the storage node 600, monitor the storage space of each storage device 603, and access the storage device 603 to read or write data from the storage device 603.

[0245] In this embodiment, the control unit 601 can acquire second index data under the instruction of a computing node (such as a computing instance or processor on a computing node) and feed the second index data back to the computing node. The control unit 601 can also select a target computing unit under the instruction of a computing node (such as a computing instance or processor on a computing node) to perform part or all of the formal search operation and feed the execution result back to the computing node.

[0246] Figure 7 The storage node cluster shown is merely an example. In practical applications, this storage node cluster can also exist in the form of a storage array, that is, the storage node cluster includes a storage array and a storage array controller for managing the storage array. The storage array includes one or more storage devices. Optionally, the storage node cluster may also include a network interface card (for communicating with devices outside the storage node cluster) and a computing accelerator card (for providing computing power support to the storage array controller to assist the storage array controller in performing certain operations). The storage array can serve as a storage node in storage cluster 220.

[0247] The computing units in storage cluster 220 include some or all of the storage array controller, smart network card, computing accelerator card, and controllers (such as hard disk controllers) within the storage device.

[0248] The vector search method provided in this application embodiment is described below. This vector search method comprises two parts. The first part is the pre-search execution process, which is performed by the client 10. See steps 901 to 904 for details. The second part is the formal search execution process, which is performed by the search node 210 on the server 20 side. See steps 905 to 906 for details.

[0249] Step 901: Client 10 receives a vector search request, which requests K vectors in the vector set that are similar to the target vector. It is worth noting that these K vectors can be the K vectors in the vector set that are most similar to the target vector, or they can be the K vectors whose similarity to the target vector is greater than a similarity threshold.

[0250] This application does not limit the scenario in which the client 10 receives the vector search request. Several possible scenarios are listed below:

[0251] Scenario 1: The user can interact with client 10 to trigger the vector search request.

[0252] Client 10 provides a vector search interface to users, through which users can send vector search requests to client 10. In this embodiment, the vector search interface represents the functionality that client 10 can provide to users, and is not limited to the specific presentation of the vector search interface.

[0253] For example, this vector search interface exists in the form of a visual interface, such as... Figure 10 The diagram illustrates a visual interface provided in an embodiment of this application. In this interface, a user can select a target vector from candidate vectors or define a custom target vector. The user can also select a search range, i.e., select a set of vectors. The user clicks the "Vector Search" option in the visual interface; unless a vector search request is generated, the client 10 obtains the vector search request.

[0254] For example, the vector search interface exists in a defined command format. A user triggers the vector search request by typing a command that conforms to this format into the client 10. Exemplarily, this command format includes a first field indicating the target vector and a second field indicating the vector set. The user defines the value of the first field to indicate the target vector and the value of the second field to indicate the vector set. After detecting the user's entered command, the client 10 determines the target vector by parsing the first field and the vector set by parsing the second field, thus obtaining the vector search request.

[0255] Scenario 2: Client 10 interacts with the demand conversion device and receives a vector search request from the demand conversion device.

[0256] Users can interact with the demand conversion device, informing it of their search needs. The conversion device then generates a vector search request based on the user's search needs and sends the vector search request to the user.

[0257] Generally speaking, users do not need to truly understand the process of vector search. That is, users do not need to directly provide the target vector and / or vector set, but only need to express their search needs in natural language, such as searching for pictures similar to picture A in the album, or searching for sentences similar to sentence A in the document.

[0258] The demand conversion device can both identify the user's search demand and perform vector conversion to transform the user's search demand into a vector search request that the client 10 can parse. In other words, the demand conversion device can convert the objects to be searched (such as the aforementioned image A and statement A) involved in the search demand into target vectors and determine the vector set corresponding to the search scope involved in the search demand (such as the aforementioned image set and file).

[0259] Step 902: Client 10 performs a pre-search and obtains the results of the pre-search. The pre-search is based on the upper-level index stored on the client 10 side, and the results of the pre-search indicate the target index node determined by the pre-search.

[0260] After receiving a vector search request, client 10 first performs a pre-search. The specific method by which client 10 performs the pre-search depends on the specific type of the approximate nearest neighbor search algorithm invoked. For details on using the approximate nearest neighbor search algorithm for vector search, please refer to the aforementioned explanation of vector search; it will not be repeated here. Since client 10 only stores the upper-level index of the global index, when performing the pre-search, client 10 only performs the vector search based on the upper-level index.

[0261] After performing a pre-search, client 10 obtains the pre-search result, which indicates the target index node determined by the pre-search. For details regarding the target index node, please refer to the aforementioned explanation of global indexes; it will not be repeated here.

[0262] Step 903: Client 10 determines the target search node from multiple search nodes 210 in server 20.

[0263] By executing step 903, client 10 can determine the target search node for the formal search. This embodiment does not limit the manner in which client 10 executes step 903. Several execution methods are listed below:

[0264] Method 1: Client 10 determines the target search node through polling.

[0265] The client 10 can obtain information about each search node 210 in the server 20. The client 10 determines a search node 210 from each search node 210 in a preset order as the target search node for which a formal search needs to be performed.

[0266] Method 2: Client 10 determines the target search node based on a pre-search rule, which is a rule for selecting the target search node based on the pre-search results. This application embodiment does not limit the specific content of the pre-search rule; any method that uses the pre-search results to determine the target search node is applicable to this application embodiment. Two pre-search rules are listed here.

[0267] Rule 1, the pre-search rule is reflected in the correspondence between each index node at the lowest level of the upper-level index and the search node 210.

[0268] Client 10 stores the correspondence between each index node at the lowest level of the upper-level index and the search node 210. Client 10 determines the target search node based on this correspondence.

[0269] After obtaining the pre-search results, client 10 determines the target index node in the lowest level of the upper-level index. Based on this correspondence, client 10 determines the search node 210 corresponding to the target index node, which is the target search node.

[0270] Rule 2: The pre-search rule indicates that the hash value of the pre-search result points to the search node 210 as the target search node.

[0271] After obtaining the pre-search result, client 10 hashes the result to obtain a hash value, which points to a search node 210 in server 20. The search node 210 pointed to by the hash value is the target search node.

[0272] Since client 10 has the ability to determine the target search node, it can determine different target search nodes each time a formal search needs to be performed, thus diverting different formal searches to different search nodes 210.

[0273] Step 904: Client 10 sends a vector search instruction to the target search node. This vector search instruction is used to indicate the acquisition of K vectors in the vector set that are similar to the target vector. Optionally, the vector search instruction carries the results of the pre-search.

[0274] Since some operations (i.e., pre-search) have already been performed on the client 10 side, the target search node can continue the vector search based on the pre-search to complete the formal search.

[0275] In order to enable the target search node to determine the starting index node for the formal search, the vector search instruction can carry the results of the pre-search, so that the target search node can determine which index node in the global index to start the search from.

[0276] In practical applications, the global index may change at any time due to changes in the vector set (such as the addition, deletion, or modification of vectors in the vector set), and the index nodes in the global index will also change. Therefore, each index node in the global index records version information, and each time an index node is updated, its version information will also change. The results of this pre-search also indicate the version information of the target index node.

[0277] Of course, the target search node can also re-execute the pre-search, meaning the target search node starts the formal search from the top level of the global index. In this case, the vector search instruction does not need to carry the results of the pre-search.

[0278] Step 905: The target search node receives the vector search instruction, performs the formal search according to the vector search instruction, and obtains the vector search result, which indicates the K vectors in the vector set that are similar to the target vector.

[0279] After receiving the vector search instruction, the target search node can begin its search from the target index node indicated by the pre-search result. The specific method by which the target search node executes the formal search depends on the specific type of approximate nearest neighbor search algorithm invoked. For details on using the approximate nearest neighbor search algorithm for vector search, please refer to the aforementioned explanation of vector search; it will not be repeated here. Since the vector search instruction specifies the target index node, the target search node does not need to start the vector search from the top level of the global index, but rather searches from the child nodes of the target index node.

[0280] This application provides two methods for the target search node to perform a formal search, which are described below:

[0281] In the first execution method, the target search node directly executes the formal search.

[0282] For the target search node, it has accessible memory space in which the first index data can be cached.

[0283] If the first index data includes all the index data required to perform the formal search, that is, if the first index data includes the set of target index nodes in each index layer of the global index (or lower-level indexes, where lower-level indexes are the remaining indexes in the global index excluding the upper-level indexes), then the target search node can directly perform the formal search based on this first index data. (For an explanation of the target index node set, please refer to the relevant explanation of the global index; it will not be repeated here.)

[0284] If the first index data only includes the partial index data required to perform the formal search, that is, the first index data does not include the set of target index nodes in each index layer of the global index (or lower index), the set of target index nodes in each index layer of the global index (or lower index) that is not included in the first index data is referred to as the second index data.

[0285] The target search node retrieves the second index data from storage cluster 220. For example, the target search node sends a data read request to storage cluster 220, which requests to read the second index data. After receiving the data read instruction, storage cluster 220 (i.e., the storage management module) retrieves the second index data from the storage device and sends the second index data back to the target index node.

[0286] The target search node performs a formal search based on the first index data and the second index data. It should be noted that the process of the target search node performing the formal search and the process of obtaining the second index data can be carried out simultaneously. That is, the target search node can determine whether the required second index data is cached in the memory space during the formal search. If it is determined that the index data is not cached, the second index data is obtained from the storage cluster 220.

[0287] In the first execution method, the target search node is the main executor of the formal search, and the storage cluster 220 only needs to provide the second index data.

[0288] The second execution method involves the target search node cooperating with the storage cluster 220 to perform the formal search.

[0289] Typically, storage cluster 220 has a certain computing power, and since it is closer to the global index, the target search node can use the computing power of storage cluster 220 to complete the formal search.

[0290] In the second execution mode, the target search node instructs the storage cluster 220 to perform a formal search, or it can perform a partial operation of the formal search by the storage node.

[0291] (1) The target search node instructs storage cluster 220 to perform part of the formal search operation.

[0292] Since the target search node has accessible memory space, the first index data can be cached in this memory space. If the first index data only includes the portion of the index data required to perform the actual search,

[0293] The target search node can perform part of the formal search based on the first index data. For the remaining operations of the formal search (such as operations that need to be performed based on the second index data), the target search node instructs the storage cluster 220 to complete them.

[0294] The target search node sends a first instruction to the storage cluster 220, which instructs to perform a vector search based on the second index data.

[0295] The target search node informs the storage cluster 220 via a first instruction that the index data required for the operation is the second index data. This first instruction may specify the storage address of the second index data, which is the storage address of the second index data within the storage cluster 220. The target search node also specifies the operation to be performed for the vector search via a first instruction. This first instruction may include an operation operator that indicates the steps to perform the operation.

[0296] The steps indicated by this operation operator include, but are not limited to: calculating similarity (e.g., calculating the similarity between the target vector and other vectors), comparing similarity (e.g., comparing the similarity between the target vector and different vectors), and sorting based on similarity. These steps can be viewed as the operations required by invoking the approximate nearest neighbor search algorithm based on the second index data.

[0297] After receiving the first instruction, the storage cluster 220 performs operations on the second index data according to the first instruction, and the storage cluster 220 feeds back the execution result to the target search node.

[0298] When storage cluster 220 performs an operation based on the second index data, it retrieves the second index data according to the storage address of the second index data. After retrieving the second index data, it performs the operation on the second index data according to the operation operator and obtains the execution result.

[0299] After obtaining the execution result, the target search node determines the result of the vector search based on the execution result. When the target search node informs the storage cluster 220 through the first instruction that the operation to be performed is all the operations required based on the second index data, then the execution result is the result of the vector search (that is, the K vectors similar to the target vector).

[0300] It should be noted that the foregoing description is based on the example that the target search node informs the storage cluster 220 of the index data required for the operation through the first instruction, which is the second index data, and the operation performed is all the operations required based on the second index data. This application embodiment does not limit the specific content of the index data required by the target search node informing the storage cluster 220 of the operation through the first instruction, nor the specific content of the operation performed.

[0301] (2) The target search node instructs storage cluster 220 to perform a formal search. Here, we will take the fourth index data as an example to illustrate the process.

[0302] The target search node sends a first instruction to storage cluster 220, which instructs the execution of a vector search based on the fourth index data. This first instruction includes the address of the fourth index data and the various operation operators for performing the actual search.

[0303] When storage cluster 220 performs an operation based on the fourth index data, it obtains the fourth index data according to the storage address of the fourth index data. After obtaining the fourth index data, it performs the operation on the fourth index data according to the operation operator in the first instruction to obtain the result of vector search (that is, K vectors similar to the target vector).

[0304] From the perspective of storage cluster 220, after receiving the first instruction, storage cluster 220 (storage management module) can select a target computing unit from one or more computing units within the storage cluster 220. The target computing unit then performs a vector search operation using index data (such as second index data and fourth index data) according to the instructions of the first instruction. The storage management module can select the target computing unit from one or more computing units within the storage cluster 220 according to a proximity strategy. The proximity strategy indicates that the target computing unit is determined based on the storage location of the index data.

[0305] For example, such as Figure 5 In the server 20 shown, the storage cluster 220 includes a central processing unit, storage devices, computing accelerator cards, etc. If the index data indicated by the first instruction is located in memory, then the central processing unit and computing accelerator cards are closer to the memory, and the storage management module can select the central processing unit or computing accelerator card as the target computing unit.

[0306] In this embodiment, the target search node supports index data prefetching. That is, when the target search node retrieves index data (such as the aforementioned second index data) from the storage cluster 220, it can also retrieve adjacent index data. This adjacent index data refers to the index data in the global index that is adjacent to the index data required by the target search node from the storage cluster 220. Taking the second index data as an example, the second index data includes one or more index nodes. For ease of explanation, the index nodes included in the second index data are referred to as index nodes S. That is, the second index data can include one or more index nodes S. For one of the index nodes S, the adjacent index data of the second index data includes the index nodes adjacent to index node S. The index nodes adjacent to index node S can be simply referred to as the neighboring index nodes of index node S. The adjacent index data of the second index data can include all the neighboring index nodes of index node S, or it can include some of the neighboring index nodes of index node S. By prefetching index data, the target search node avoids retrieving index data from the storage cluster 220 when performing other vector searches subsequently, thus improving the efficiency of subsequent vector search execution.

[0307] It should be noted that if the pre-search result also indicates the version information of the target index node, after receiving the vector search instruction, the target search node can first determine whether the version information of the target index node is consistent with the version information of the target index node in the global index of storage cluster 220. If they are consistent, the target search node starts the formal search from the target index node; otherwise, the target search node starts the vector search from the top level of the global index, identifying K vectors in the vector set that are similar to the target vector.

[0308] The preceding explanation regarding the target search node performing the formal search was based on the example of the target search node starting the formal search from the target search node itself. In practical applications, the target search node can also start the formal search from the top level of the global index. That is, the target search node needs to perform a pre-search again.

[0309] Step 906: The target search node sends the vector search results back to client 10.

[0310] Scenario 1: The user can interact with client 10 and triggers the vector search request.

[0311] After obtaining the results of the vector search, client 10 returns the results to the user through the vector search interface.

[0312] Scenario 2: Client 10 interacts with the demand conversion device, and the vector search request comes from the demand conversion device.

[0313] After obtaining the vector search results, client 10 sends the results back to the demand conversion device. The demand conversion device converts the vector search results into natural language that the user can understand, and then transmits the vector search results to the user in natural language.

[0314] Based on the same inventive concept as the method embodiments, this application also provides a client device 1100, which is used to execute the method executed by the client 10 in the above method embodiments. For example... Figure 11 As shown, the client device 1100 includes a pre-search module 1101 and a sending module 1102. The client device 1100 stores the upper-level index in the global index. The global index is a single index device constructed based on a vector set, and includes multiple index layers. The upper-level index includes the first M index layers of the global index. Specifically, in the client device 1000, the modules establish connections through communication paths.

[0315] The pre-search module 1101 is used to perform a pre-search based on the upper-level index and obtain the results of the pre-search.

[0316] The sending module 1102 is used to send a vector search instruction to the server. The vector search instruction is used to request the K vectors in the vector set that are similar to the target vector. The vector search instruction includes the results of the pre-search.

[0317] As one possible implementation, the sending module 1102 determines the target search node from multiple search nodes according to a routing policy, the routing policy indicating the rules for selecting the target search node.

[0318] As one possible implementation, the routing strategy includes round-robin or pre-search rules. The pre-search rules are rules for selecting target search nodes based on the results of the pre-search. The results of the pre-search are the results obtained by the sending module 1102 based on the upper-level index, and the results of the pre-search are included in the vector search instruction. The upper-level index includes the first M index layers in the global index.

[0319] Based on the same inventive concept as the method embodiments, this application also provides a server-side device 1200, which is used to execute the method executed by the server 20 in the above method embodiments. For example... Figure 12 As shown, the server device 1200 includes a receiving module 1201 and a searching module 1202. Specifically, in the server device 1200, the modules establish connections with each other through a communication path.

[0320] The receiving module 1201 is used to receive a vector search instruction from the client. The vector search instruction is used to request the acquisition of K vectors in the vector set that are similar to the target vector.

[0321] The search module 1202 is used to determine K vectors similar to the target vector from the vector set based on the global index of the vector set, and to return the K vectors to the client. The vector set is distributed across multiple storage nodes in the storage cluster.

[0322] As one possible implementation, a global index is a single index built upon a set of vectors.

[0323] In one possible implementation, the server device includes multiple search nodes, a search module 1202, and a target search node among the multiple search nodes.

[0324] As one possible implementation, the global index includes multiple index layers, the vector search instruction includes the results of the pre-search, which are the results of vector search using the upper-level index, the upper-level index includes the first M index layers in the global index, and the search module 1202 determines K vectors similar to the target vector from the vector set based on the results of the pre-search and the global index.

[0325] As one possible implementation, the search module 1202 caches the first index data in the global index. When the search module 1202 determines K vectors similar to the target vector from the vector set based on the global index of the vector set, it performs a vector search based on the first index data to determine the K vectors.

[0326] As one possible implementation, when the search module 1202 performs a vector search to determine K vectors based on the first index data, it obtains second index data from the global index, which is different from the first index data, from the storage cluster. The storage cluster stores the global index. The search module 1202 performs a vector search based on the first index data and the second index data to determine K vectors.

[0327] In one possible implementation, the search module 1202 can retrieve other index data besides the second index data in the global index obtained by the storage cluster. For example, the search module 1202 sends a prefetch request to the storage cluster, which requests to read the second index data and prefetch the third index data adjacent to the second index data in the global index; the search module 1202 then retrieves the second index data and the third index data returned by the storage cluster.

[0328] In one possible implementation, when the search module 1202 determines K vectors similar to the target vector from the vector set based on the global index, it sends a first instruction to the storage cluster. This first instruction instructs the storage cluster to perform a vector search based on the fourth index data in the global index. The search module 1202 then obtains the vector search results from the storage cluster, which indicate the K vectors.

[0329] The module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0330] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a personal computer, mobile phone, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0331] This application also provides, for example Figure 13 The computing device 1300 shown includes a bus 1301, a processor 1302, a communication interface 1303, and a memory 1304. The processor 1302, the memory 1304, and the communication interface 1303 communicate with each other via the bus 1301.

[0332] The processor 1302 can be a central processing unit (CPU) or other specific integrated circuits. The processor 132 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0333] Memory 1304 can be DRAM. Besides DRAM, memory 1304 can also be other random access memory (such as SRAM). Additionally, memory 1302 can also be ROM. For read-only memory, for example, it could be PROM, EPROM, etc. Memory 1304 can also be flash memory, HDD, or SSD.

[0334] The memory 1304 stores computer program instructions, and the processor 1302 executes these computer program instructions to perform the aforementioned tasks. Figure 9 The steps executed by client 10. The memory 1304 may also include software modules required for other running processes, such as the operating system (e.g., multiple modules in client device 1100). The operating system may be LINUX. TM UNIX TM WINDOWS TM wait.

[0335] or

[0336] The memory 1304 stores computer program instructions, and the processor 1302 executes these computer program instructions to perform the aforementioned tasks. Figure 9 The steps executed by the server 20. The memory 1304 may also include software modules required for other running processes, such as the operating system (e.g., multiple modules in the server device 1200). The operating system may be LINUX. TM UNIX TM WINDOWS TM wait.

[0337] This application also provides a computing device system, the computing device system including at least one such as Figure 14 The computing device 1400 shown includes a bus 1401, a processor 1402, a communication interface 1403, and a memory 1404. The processor 1402, the memory 1404, and the communication interface 1403 communicate with each other via the bus 1401. At least one computing device 1400 in the computing device system communicates with each other via a communication path.

[0338] The specific types of processor 1402 and memory 1404 can be found in the descriptions of processor 1302 and memory 1304, and will not be repeated here. Processor 1402 executes the computer program instructions stored in memory 1404 to perform the aforementioned tasks. Figure 9 The method described may involve client 20 performing some or all of the steps, or performing the aforementioned steps. Figure 9 The described method includes some or all of the steps executed by server 20. The memory may also include other software modules required for running processes, such as the operating system. The operating system may be Linux.TM UNIX TM WINDOWS TM wait.

[0339] At least one computing device 1400 in the computing device system establishes communication with each other through a communication network, and each computing device 1400 runs any one or any multiple modules of a server device 1200 or a client device 1100.

[0340] The descriptions of the processes corresponding to the above-mentioned figures each have their own emphasis. For parts of a process that are not described in detail, please refer to the relevant descriptions of other processes.

[0341] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes computer program instructions, which, when loaded and executed on a computer, generate, in whole or in part, the product according to the embodiments of the present invention. Figure 9 The process or function described.

[0342] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD).

[0343] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A vector search system, characterized in that, The vector search system includes a client and a server: The client is used to send a vector search instruction to the server, the vector search instruction carrying a target vector; The server is used to determine K vectors similar to the target vector from the vector set based on the global index of the vector set, and to feed back the K vectors to the client. The vector set is distributed across multiple storage nodes in the storage cluster.

2. The system as described in claim 1, characterized in that, The global index is a single index built based on the set of vectors.

3. The system according to any one of claims 1 to 2, characterized in that, The server includes multiple search nodes, and the client is used for: Send the vector search instruction to the target search node among the plurality of search nodes; The target search node in the server is used for: Based on the global index, determine K vectors from the vector set that are similar to the target vector.

4. The system according to any one of claims 1 to 3, characterized in that, The global index includes multiple index layers. The client stores the upper-level index in the global index. The upper-level index includes the first M index layers in the global index. The client is further configured to: A pre-search is performed based on the upper-level index, and the vector search instruction includes the results of the pre-search. The server is used for: Based on the results of the pre-search and the global index, K vectors similar to the target vector are determined from the vector set.

5. The system as described in claim 3, characterized in that, The client is also used for: The target search node is determined from the plurality of search nodes according to a routing policy, wherein the routing policy indicates the rules for selecting the target search node.

6. The system as described in claim 5, characterized in that, The routing strategy includes round-robin or pre-search rules. The pre-search rules are rules for selecting the target search node based on the pre-search results. The pre-search results are the results obtained by the client performing a pre-search based on the upper-level index, and the pre-search results are included in the vector search instruction. The upper-level index includes the first M index layers in the global index.

7. The system as described in claim 3, characterized in that, The target search node caches the first index data in the global index, and the target search node is used for: A vector search is performed based on the first index data to determine the K vectors.

8. The system as described in claim 7, characterized in that, The target search node is used for: Retrieve second index data, which is different from the first index data, from the global index in the storage cluster, wherein the global index is stored in the storage cluster; A vector search is performed based on the first index data and the second index data to determine the K vectors.

9. The system as described in claim 8, characterized in that, The target search node is used for: A prefetch request is sent to the storage cluster. The prefetch request is used to request the reading of the second index data and the prefetching of the third index data adjacent to the second index data in the global index. Obtain the second index data and the third index data fed back by the storage cluster.

10. The system as described in claim 3, characterized in that, The target search node is used for: Send a first instruction to the storage cluster, the first instruction being used to instruct the storage cluster to perform a vector search based on the fourth index data in the global index; Obtain the results of the vector search fed back by the storage cluster, the results of the vector search indicating the K vectors.

11. The system as claimed in claim 10, characterized in that, The vector search system further includes the storage cluster, which is used for: Upon receiving the first instruction, and based on the near-data strategy, a target computing unit in the storage cluster is determined. The target computing unit is used to perform a vector search based on the fourth index data in the global index to obtain the result of the vector search.

12. A vector search method, characterized in that, The method is applied to a vector search system, which includes a client and a server, and the method includes: The client sends a vector search instruction to the server, the vector search instruction carrying the target vector; The server determines K vectors similar to the target vector from the vector set based on the global index of the vector set, and feeds back the K vectors to the client. The vector set is distributed across multiple storage nodes in the storage cluster.

13. The method as described in claim 12, characterized in that, The global index is a single index built based on the set of vectors.

14. The method according to any one of claims 12-13, characterized in that, The server includes multiple search nodes, and the client sends vector search instructions to the server, including: The client sends the vector search instruction to the target search node among the plurality of search nodes; The server determines K vectors similar to the target vector from the vector set based on a global index of the vector set, including: The target search node determines K vectors similar to the target vector from the vector set based on the global index.

15. The method according to any one of claims 12 to 14, characterized in that, The global index includes multiple index layers, and the client stores the upper-level index in the global index. The upper-level index includes the first M index layers in the global index. The method further includes: The client performs a pre-search based on the upper-level index, and the vector search instruction includes the results of the pre-search. The server determines K vectors similar to the target vector from the vector set based on a global index of the vector set, including: The server determines K vectors similar to the target vector from the vector set based on the results of the pre-search and the global index.

16. The method as described in claim 14, characterized in that, Before the client sends the vector search instruction to the target search node among the plurality of search nodes, it also includes: The client determines the target search node from the plurality of search nodes according to a routing policy, wherein the routing policy indicates the rules for selecting the target search node.

17. The method as described in claim 16, characterized in that, The routing strategy includes round-robin or pre-search rules; the pre-search rules are rules for selecting the target search node based on the pre-search results, and the pre-search results are the results obtained by the client performing a pre-search based on the upper-level index, and the upper-level index includes the first M index layers in the global index.

18. The method as described in claim 14, characterized in that, The target search node caches the first index data in the global index. Based on the global index of the vector set, the target search node determines K vectors similar to the target vector from the vector set, including: The target search node performs a vector search based on the first index data to determine the K vectors.

19. The method as described in claim 18, characterized in that, The target search node determines the K vectors by performing a vector search based on the first index data, including: The target search node obtains second index data from the global index, which is different from the first index data, from the storage cluster, where the global index is stored; The target search node performs a vector search based on the first index data and the second index data to determine the K vectors.

20. The method as described in claim 19, characterized in that, The target search node obtains the second index data from the storage cluster, including: The target search node sends a prefetch request to the storage cluster. The prefetch request is used to request the reading of the second index data and to prefetch the third index data adjacent to the second index data in the global index. The target search node obtains the second index data and the third index data fed back by the storage cluster.

21. The method as described in claim 14, characterized in that, The target search node determines K vectors similar to the target vector from the vector set based on the global index, including: The target search node sends a first instruction to the storage cluster, the first instruction being used to instruct the storage cluster to perform a vector search based on the fourth index data in the global index; The target search node obtains the results of the vector search fed back by the storage cluster, and the results of the vector search indicate the K vectors.

22. The method as described in claim 21, characterized in that, The method further includes: The storage cluster receives the first instruction and, according to the near data strategy, determines the target computing unit in the storage cluster. The target computing unit is used to perform a vector search based on the fourth index data in the global index to obtain the result of the vector search.

23. A computing device, characterized in that, The computing device includes a memory and a processor; The memory stores metadata indexes; The processor is used to perform the method as described in any one of claims 12 to 22.

Images