Approximate nearest neighbor search method and approximate nearest neighbor search system

Abstract

The present disclosure relates to an approximate nearest neighbor search method and an approximate nearest neighbor search system. According to the embodiment, the approximate nearest neighbor search method manages graph index information for defining an inter-cluster graph. The approximate nearest neighbor search method searches for a vector closest to a query from vectors belonging to a search start cluster having a reference position closest to the query among a plurality of clusters. The approximate nearest neighbor search method selects one or more search target clusters close to the search start cluster while advancing along the inter-cluster graph, and searches for a vector closest to the query from vectors belonging to each of the one or more search target clusters. Thus, the approximate nearest neighbor search method capable of reducing the amount of data required to be rewritten accompanying update of the graph index and capable of obtaining sufficient search accuracy is provided.

Description

Technical Field

[0001] The embodiments of the present invention relate to Approximate Nearest Neighbor Search (ANNS). Background Technology

[0002] Vector databases are used in various fields such as machine learning and data analysis. In a vector database, each data point is stored as a high-dimensional vector, containing numerous feature values ​​corresponding to many attributes. During the search stage, a nearest neighbor search is performed to find the vector (nearest neighbor vector) that is closest to the query vector (query vector) through an exhaustive search. The query vector has the same dimension as the vectors in the vector database.

[0003] Furthermore, approximate nearest neighbor search is currently being used instead of direct nearest neighbor search. Unlike direct nearest neighbor search, which strictly searches for the nearest neighbor vector through exhaustive search, approximate nearest neighbor search is a method for quickly searching for vectors sufficiently close to the query (approximate nearest neighbor vectors). Algorithms using graph-based indexes are known to perform approximate nearest neighbor searches with high accuracy and speed even for high-dimensional vector sets.

[0004] To build a large-scale vector database capable of searching over a billion vectors, the data volume of the vector set and the graph index would increase significantly, making it impossible to configure them on the computer's main storage device. Therefore, approximate nearest neighbor search algorithms, such as configuring the graph index on secondary storage devices like solid-state drives (SSDs), are currently being developed.

[0005] Additionally, many graph indexes use inter-vector graphs, which are structures where closely spaced vectors are connected by edges. Each edge is represented by edge information within the graph index.

[0006] When constructing a graph for a large-scale vector database with over a billion vectors, the overall data volume of the graph index increases significantly due to the increased number of edge information required. As a result, a large storage area is needed to store the graph index.

[0007] Furthermore, in the vector graph, whenever a vector is added to the vector database, the added vector needs to be registered as a new neighbor vector to many other vectors that are close to it. Therefore, whenever a vector is added, the edge information corresponding to many other vectors also needs to be rewritten.

[0008] Secondary storage devices such as SSDs and hard disk drives (HDDs) have higher access latency compared to primary storage devices like DRAM and SRAM. Furthermore, SSDs have limitations, such as a cap on the number of times they can be overwritten. Therefore, in algorithms using inter-vector graphs, there are issues such as increased time required to register a vector, and the increased number of overwrites due to edge information changes causing the secondary storage device to quickly reach its lifespan.

[0009] On the other hand, algorithms using cluster-based indexes do not require rewriting much of the edge information. However, algorithms using cluster-based indexes struggle to search for vectors located near inter-cluster boundaries, sometimes resulting in insufficient search accuracy. Summary of the Invention

[0010] Provides an approximate nearest neighbor search method and system that can reduce the amount of data that needs to be rewritten as the graph index is updated and can perform approximate nearest neighbor search with sufficient search accuracy.

[0011] According to an implementation, an approximate nearest neighbor search method for a vector database manages cluster index information for defining multiple clusters. The vector database is configured to store multiple vectors, each containing multiple feature values ​​corresponding to multiple dimensions. Each cluster has a reference position, and for each cluster, a group of vectors whose reference positions are close to that cluster belongs to that cluster. The approximate nearest neighbor search method also manages graph index information for defining an inter-cluster graph, which includes multiple nodes and multiple edges. Each node corresponds to one of the multiple clusters, and each edge connects nodes corresponding to clusters with mutually close reference positions. The approximate nearest neighbor search method receives a query vector, which contains feature values ​​for each of the multiple dimensions. The approximate nearest neighbor search method performs a first process of determining the cluster with the reference position closest to the query vector as the starting cluster for the search. The approximate nearest neighbor search method then searches from the vectors belonging to the starting cluster for the vector closest to the query vector as the nearest neighbor vector within the starting cluster. The approximate nearest neighbor search method traverses the inter-cluster graph while selecting one or more search object clusters that are close to the search start cluster. It then searches for the vector closest to the query vector from the vectors belonging to each of these search object clusters, using this vector as the nearest neighbor vector within each search object cluster. The approximate nearest neighbor search method outputs the nearest neighbor vectors found in the search start cluster and the vectors closest to the query vector from the nearest neighbor vectors found in each of the one or more search object clusters, as the approximate nearest neighbor vectors of the query vector. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating an example of the configuration of an approximate nearest neighbor search system involved in the implementation.

[0013] Figure 2 This is a diagram illustrating an example of a hybrid index structure for an approximate nearest neighbor search system involved in an implementation.

[0014] Figure 3 This is a diagram illustrating an example of search processing performed in the approximate nearest neighbor search system involved in the implementation using a hybrid index structure.

[0015] Figure 4 This is a diagram illustrating an example of the structure of an inter-cluster graph used in the approximate nearest neighbor search system described in the implementation.

[0016] Figure 5It is a graph used to illustrate cluster search using hierarchical clusters and approximate nearest neighbor vector search using inter-cluster graphs.

[0017] Figure 6A This is a graph representing the search process for the starting cluster.

[0018] Figure 6B It is a graph representing the search process for the first search object cluster that is closest to the search starting cluster.

[0019] Figure 6C This is a graph representing the search process for the second search object cluster.

[0020] Figure 6D It is a graph representing the search process for more than one neighboring cluster of the second search target cluster, excluding the clusters that have already been searched.

[0021] Figure 7A This is a graph representing the search process for the starting cluster.

[0022] Figure 7B It is a graph representing the search process for the cluster of search objects that has the closest orientation to the query orientation from the reference position of the search starting cluster.

[0023] Figure 7C It is a graph representing the search process for one or more neighboring clusters of the search target cluster that has the closest orientation from the reference position of the search starting cluster to the orientation of the query, excluding the clusters that have already been searched.

[0024] Figure 8 This is a graph representing the index update process used to append new clusters.

[0025] Figure 9 It is a graph representing the index update process used to delete clusters whose vector count becomes zero.

[0026] Figure 10 This is a diagram showing an example of upper-level cluster index information corresponding to each upper-level (the layer above) cluster.

[0027] Figure 11 This is a diagram showing an example of the index information of the lowest-level clusters corresponding to each lowest-level (bottom-level) cluster.

[0028] Figure 12A This is a diagram used to illustrate part of the principle of approximate nearest neighbor vector search using the distance between vectors.

[0029] Figure 12B This is another part of the diagram used to illustrate the principle of approximate nearest neighbor vector search using the distance between vectors.

[0030] Figure 12CThis is another part of the diagram used to illustrate the principle of approximate nearest neighbor vector search using the distance between vectors.

[0031] Figure 13A This is a diagram used to illustrate the principle of approximate nearest neighbor vector search using distance and orientation between vectors.

[0032] Figure 13B This is another part of the diagram used to illustrate the principle of approximate nearest neighbor vector search using the distance and orientation between vectors.

[0033] Figure 13C This is another part of the diagram used to illustrate the principle of approximate nearest neighbor vector search using the distance and orientation between vectors.

[0034] Figure 13D This is another part of the diagram used to illustrate the principle of approximate nearest neighbor vector search using the distance and orientation between vectors.

[0035] Figure 14 This is a flowchart illustrating an example of the process (procedure, steps) for constructing a hybrid index structure that includes hierarchical clusters and inter-cluster graphs.

[0036] Figure 15 This is a flowchart illustrating an example of the process of searching for the approximate nearest neighbor cluster (search start cluster).

[0037] Figure 16 This is a flowchart illustrating another example of the process for searching the approximate nearest neighbor cluster (search start cluster).

[0038] Figure 17 This is a flowchart illustrating an example of the process of approximate nearest neighbor vector search.

[0039] Figure 18 This is a flowchart illustrating an example of the first index update process performed based on vectors appended to a vector database.

[0040] Figure 19 This is a flowchart illustrating an example of the intra-cluster nearest neighbor vector search process performed when the cluster being processed during append processing is determined to be the starting cluster for the search.

[0041] Figure 20 This is a flowchart illustrating an example of the second index update process performed when a vector is deleted from a dataset in a vector database.

[0042] Figure 21 This is a diagram representing the first variation of the search process for the approximate nearest neighbor cluster (search start cluster).

[0043] Figure 22 This is a diagram representing the second variation of the search process used for the approximate nearest neighbor cluster (search start cluster).

[0044] Figure 23 This is a diagram representing the third variation of the search process used for the approximate nearest neighbor cluster (search start cluster).

[0045] Explanation of reference numerals in the attached figures

[0046] 1. Approximate nearest neighbor search system; 2. External device; 3. Communication path; 11. Processor; 12. Main memory; 13. Communication interface; 14. Secondary storage device; 111. Cluster index creation unit; 112. Graph index creation unit; 113. Search unit; 121. Index creation program; 122. Search program; 141. Storage medium (non-volatile memory); 21. Data set; 22. Hybrid index information; 221. Upper-level cluster index information; 222. Lower-level cluster index information; HC hierarchical cluster. Detailed Implementation

[0047] Hereinafter, the embodiments will be described with reference to the accompanying drawings.

[0048] Figure 1 This is a block diagram illustrating a configuration example of the approximate nearest neighbor search system 1 according to the implementation method. The approximate nearest neighbor search system 1 is a computer system configured to perform approximate nearest neighbor search for a vector database.

[0049] The vector database is a database for storing and managing dataset 21. Dataset 21 includes multiple vectors. Each of these vectors is an uncompressed vector, i.e., a full-precision vector.

[0050] The dataset 21 contains multiple vectors, each comprising multiple feature values ​​corresponding to multiple dimensions. When the dimension of each vector is set to D, each vector (D-dimensional vector) is equivalent to a point (data point) in a D-dimensional space. The D features contained in the D-dimensional vector each represent feature values ​​(real numbers) of D attributes. Each vector is a high-dimensional vector with a dimension D of hundreds or thousands. The dimension D can be, for example, 1024 or 2048. Hereinafter, the D-dimensional space is also referred to as the data space or vector space.

[0051] The approximate nearest neighbor search system 1 receives a query vector based on a query from an external device 2. The query vector represents the target data (target vector) to be searched from a vector database. The query vector has the same dimension D as the vectors in the vector database. That is, like the vectors in the vector database, the query vector also includes D eigenvalues ​​corresponding to each of the D dimensions. Hereinafter, the query vector will be simply referred to as the query.

[0052] The Approximate Nearest Neighbor Search System 1 performs an approximate nearest neighbor search on a vector database based on the received query. Approximate nearest neighbor search is a fast method for searching for vectors that are sufficiently close to the query (approximate nearest neighbor vectors) at a certain distance scale.

[0053] In this embodiment, the distance scale used to represent the distance between vectors can be, for example, Euclidean distance. In this case, basically, the Euclidean distance to the query (query vector) can be calculated for each of the search object vectors in the vector database, and the vector with the shortest Euclidean distance to the query is found among the search object vectors as the approximate nearest neighbor vector of the query.

[0054] Furthermore, the distance scale is not limited to Euclidean distance; any other distance that can represent the distance between vectors can also be used as the distance scale.

[0055] Next, the configuration of the approximate nearest neighbor search system 1 will be described. The approximate nearest neighbor search system 1 includes a processor 11, a main memory 12, a communication interface 13, and a secondary storage device 14. The processor 11, the main memory 12, the communication interface 13, and the secondary storage device 14 are interconnected via a bus 10.

[0056] Processor 11 is, for example, a central processing unit (CPU). Processor 11 is capable of accessing main memory 12 and secondary storage device 14. Processor 11 performs various processes, including the creation of index information 22 (here, hybrid index information 22), the storage of index information 22 to secondary storage device 14, and the searching of dataset 21, by executing computer programs (here, index creation program 121 and search program 122) stored in main memory 12. Hybrid index information 22 is a data structure used to search for target vectors (the approximate nearest neighbor vectors of the query) from dataset 21.

[0057] Main memory 12 is a low-latency storage device such as DRAM. The storage areas of main memory 12 are used to store programs to be executed by processor 11, as well as the working area of ​​processor 11.

[0058] Communication interface 13 is a communication device. Communication interface 13 performs communication with external device 2, for example, via communication path 3 such as a network or bus.

[0059] Secondary storage device 14 is a storage device with a larger capacity than main memory 12 and a slower access speed than main memory 12.

[0060] The approximate nearest neighbor search system 1 aims to realize a trillion-scale vector database capable of storing and managing a dataset 21 containing more than one trillion vectors. When constructing a trillion-scale vector database, the size of the dataset 21 and the index information 22 increase, making it difficult to store the index information 22 in the main memory 12. Therefore, in the approximate nearest neighbor search system 1, the dataset 21 and the index information 22 are each stored in the storage medium 141 of the secondary storage device 14.

[0061] The secondary storage device 14 can be implemented using a hard disk drive (HDD) or a solid-state drive (SSD). Hereinafter, we assume the secondary storage device 14 is implemented using an SSD.

[0062] An SSD is a memory system that includes non-volatile memory and a controller that controls the non-volatile memory.

[0063] Non-volatile memory comprises multiple blocks (also referred to as "memory blocks," "physical blocks," or "flash memory blocks"), each of which is a unit for data erasure operations. Each block comprises multiple pages, each of which is a unit for data write operations and data read operations. Non-volatile memory is, for example, NAND flash memory. NAND flash memory is, for example, a three-dimensional flash memory.

[0064] The controller is a memory controller with circuitry, such as an LSI implemented as a system-on-a-chip (SoC).

[0065] The processor 11 functions as a cluster index creation unit 111, a graph index creation unit 112, and a search unit 113 by executing the index creation program 121 and the search program 122. Furthermore, the cluster index creation unit 111, the graph index creation unit 112, and the search unit 113 can each be implemented using dedicated hardware (circuit) within the near nearest neighbor search system 1.

[0066] The cluster index creation unit 111 creates and manages multiple clusters. Each cluster has a reference position. The reference position of each cluster is represented by a vector with the same dimension as the vectors in dataset 21. In other words, like the vectors in dataset 21, the reference position of a cluster is equivalent to a point in the data space (D-dimensional vector space), represented by a D-dimensional vector. Therefore, the reference position of each cluster will be referred to as the reference vector of each cluster below.

[0067] For each of the multiple clusters, the group of vectors closest to its reference position belongs to that cluster. The number of vectors belonging to a cluster is, for example, N. When building a trillion-vector database, N could be, for example, 256. The relationship between each cluster and the groups of vectors belonging to each cluster is determined as follows.

[0068] For example, consider the scenario of managing cluster X with reference position x, cluster Y with reference position y, and cluster Z with reference position z.

[0069] In this case, vectors close to the reference position x are managed to belong to cluster X, vectors close to the reference position y are managed to belong to cluster Y, and vectors close to the reference position z are managed to belong to cluster Z.

[0070] The distance from each vector belonging to a certain cluster to the reference position of that cluster is shorter than the distance from each of the aforementioned vectors to the reference positions of other clusters. In other words, each vector in dataset 21 belongs to the cluster with the shortest distance from that vector to the reference position.

[0071] As the reference position for each cluster, for example, any vector belonging to that cluster can be used. In this case, the reference position of each cluster is equivalent to a representative point of multiple data points corresponding to multiple vectors belonging to that cluster, and is also called the cluster center.

[0072] The cluster-based index creation unit 111 creates cluster-based index information for multiple clusters that each has a reference position (reference vector), and manages the created cluster-based index information.

[0073] Cluster-based indexing information is for each of multiple clusters and includes a list of vectors belonging to that cluster. The list of vectors belonging to a particular cluster represents the identifiers of the vectors belonging to that cluster.

[0074] In this embodiment, to reduce the computational cost and time required to search for approximate nearest neighbor vectors from more than one trillion vectors, multiple clusters can be managed as hierarchical clusters. In this case, the cluster index information includes information for managing multiple clusters as hierarchical clusters with a hierarchical cluster structure. Details of the hierarchical clusters will be discussed later. Figure 2 To be continued.

[0075] The graph-based index creation unit 112 generates graph-based index information and manages the generated graph-based index information. The graph-based index information contains information used to define the inter-cluster graph.

[0076] An inter-cluster graph is not a graph that connects vectors that are close to each other, but rather a graph that connects clusters that are close to each other. That is, an inter-cluster graph consists of multiple nodes corresponding to multiple clusters, and multiple edges connecting the nodes corresponding to clusters with similar reference positions. Details of the structure of an inter-cluster graph can be found using... Figure 4 Please provide an explanation.

[0077] Search unit 113 receives a query vector based on a query from external device 2 and performs an approximate nearest neighbor search on the vector database. In the approximate nearest neighbor search, search unit 113 uses hybrid index information 22, which includes clustered index information and graph index information, to search for the approximate nearest neighbor vector of the query from the dataset 21.

[0078] Next, the hybrid index structure, which includes hierarchical clusters (HC) and inter-cluster graphs (CG), will be explained. Figure 2 This is a diagram illustrating an example of a hybrid index structure 22S.

[0079] Hierarchical clusters (HC) consist of multiple layers. These layers include a lowest-level layer and multiple higher-level layers. Figure 2 The example shown is a case where the lowest layer is layer L0 and there are multiple upper layers including layers L1, L2 and L3.

[0080] The lowest layer L0 comprises multiple lowest layer clusters. For each of these clusters, the group of vectors closest to its reference position belongs to that cluster. The multiple clusters described above correspond to the multiple lowest layer clusters included in the lowest layer L0.

[0081] exist Figure 2 In the example shown in the lowest level L0, only three lowest level clusters CL1, CL2, and CL3 are illustrated. However, the actual number of lowest level clusters can be calculated by dividing the number of vectors in dataset 21 by N (the number of vectors in each cluster). Additionally, in... Figure 2 The example shown is N = 5, but it can also be a finite number of natural numbers other than 5.

[0082] Vectors V1 to V5 belong to the lowest layer cluster CL1. Each of vectors V1 to V5 is a vector that is close to the reference position B0-1 of the lowest layer cluster CL1. The reference position (reference vector) B0-1 of the lowest layer cluster CL1 can also be set to be consistent with one of the vectors V1 to V5 (in this case, vector V2).

[0083] Vectors V6 to V10 belong to the lowest layer cluster CL2. Each of vectors V6 to V10 is a vector close to the reference position B0-2 of the lowest layer cluster CL2. The reference position (reference vector) B0-2 of the lowest layer cluster CL2 can also be set to be consistent with one of the vectors V6 to V10 (in this case, vector V7).

[0084] Vectors V11 to V15 belong to the lowest layer cluster CL3. Each of vectors V11 to V15 is a vector close to the reference position B0-3 of the lowest layer cluster CL3. The reference position (reference vector) B0-3 of the lowest layer cluster CL3 can also be set to be consistent with one of the vectors V11 to V15 (in this case, vector V14).

[0085] Within these lowest-level clusters, two lowest-level clusters with closely spaced reference positions are connected to each other via edges in the inter-cluster graph CG. Furthermore, in Figure 2 The middle edge is represented by a thick line.

[0086] The N (in this case, 5) lowest-level clusters in the lowest-level cluster are grouped together and belong to one of the multiple upper-level clusters included in layer L1, the layer above the lowest-level cluster.

[0087] exist Figure 2 In the example shown, only two upper-level clusters, L1-CL1 and L1-CL2, from the multiple upper-level clusters included in layer L1 are presented. Upper-level clusters L1-CL1 and L1-CL2 each have reference positions B1-1 and B1-2, respectively.

[0088] For example, the lowest layer clusters CL1 to CL5 belong to the upper layer cluster L1-CL1. These lowest layer clusters CL1 to CL5 are called the lower layer clusters of the upper layer cluster L1-CL1. Each of the lowest layer clusters CL1 to CL5 is a lowest layer cluster with a reference position close to the reference position B1-1 of the upper layer cluster L1-CL1. The reference position (reference vector) B1-1 of the upper layer cluster L1-CL1 can also be set to coincide with the reference position of one of the lowest layer clusters CL1 to CL5 (in this case, the lowest layer cluster CL1).

[0089] N (in this case, 5) of the multiple upper-level clusters included in layer L1 are grouped together and belong to one of the multiple upper-level clusters included in layer L2.

[0090] exist Figure 2 In the example shown, only two upper-level clusters, L2-CL1 and L2-CL2, are shown among the multiple upper-level clusters included in layer L2. Upper-level clusters L2-CL1 and L2-CL2 each have reference positions B2-1 and B2-2, respectively.

[0091] For example, the upper-level clusters L1-CL1 to L1-CL5 of layer L1 are subordinate clusters of upper-level cluster L2-CL1 and belong to upper-level cluster L2-CL1. Each of the upper-level clusters L1-CL1 to L1-CL5 of layer L1 has a reference position close to the reference position B2-1 of the upper-level cluster L2-CL1 of layer L2. The reference position (reference vector) B2-1 of the upper-level cluster L2-CL1 of layer L2 can also be set to be consistent with the reference position of one of the upper-level clusters L1-CL1 to L1-CL5 (in this case, upper-level cluster L1-CL1).

[0092] The N (in this case, 5) super-layer clusters included in layer L2, the layer above layer L1, are grouped together and belong to one super-layer cluster (superior layer cluster) L3-CL included in layer L3 (the uppermost layer).

[0093] The upper-level clusters L2-CL1 to L2-CL5 of layer L2 are subordinate clusters of the uppermost cluster L3-CL and thus belong to the uppermost cluster L3-CL. Each of the upper-level clusters L2-CL1 to L2-CL5 of layer L2 has a reference position close to the reference position B3-1 of the uppermost cluster L3-CL. The reference position (reference vector) B3-1 of the uppermost cluster L3-CL can also be set to be consistent with the reference position of one of the upper-level clusters L2-CL1 to L2-CL5 (in this case, the upper-level cluster L2-CL1).

[0094] Thus, the hierarchical cluster HC has the following structure: For the uppermost layer L3, the uppermost layer cluster (L3-CL) has multiple lowermost layer clusters belonging to it, each with a reference position close to its reference position. Similarly, for each uppermost layer L1 and L2, each uppermost layer cluster has multiple lowermost layer clusters belonging to it, each with a reference position close to its reference position. Furthermore, the multiple clusters belonging to groups with vectors are managed using a hierarchical cluster structure that includes the lowermost layer and multiple uppermost layers.

[0095] Furthermore, the number of lower-level clusters belonging to each higher-level cluster may change due to the addition or deletion of lower-level clusters. A higher-level cluster whose number of lower-level clusters becomes zero can also be deleted. Therefore, more than one lower-level cluster belongs to each higher-level cluster. Additionally, the number of vectors belonging to each lowest-level cluster changes due to the addition of vectors to dataset 21 or the deletion of vectors from dataset 21. A lowest-level cluster whose number of vectors becomes zero can also be deleted. Therefore, more than one vector belongs to each lowest-level cluster.

[0096] Next, consider the scenario of building a trillion-level vector database. In this case, N can also be set to hundreds, such as 256.

[0097] For each lowest-level cluster CL of the lowest-level layer L0, there are 256 vectors V belonging to that lowest-level cluster CL.

[0098] Each of the 256 vectors V belongs to one of the 256 lowest-level clusters CL, which are subordinate clusters of a higher-level cluster L1-CL within layer L1. Therefore, the number of vectors V that can be managed by each higher-level cluster L1-CL within layer L1 is 256. 2 .

[0099] Each of the 256 lower-level clusters (256 lowest-level clusters CL) belongs to one of the 256 upper-level clusters L1-CL, which in turn are lower-level clusters of an upper-level cluster L2-CL within layer L2. Therefore, the number of vectors that can be managed by each upper-level cluster L2-CL within layer L2 becomes 256. 3 .

[0100] Each of the 256 lower-level clusters (256 upper-level clusters L1-CL) belongs to one of the 256 upper-level clusters L2-CL, which in turn are lower-level clusters of an upper-level cluster L3-CL within layer L3. Therefore, the number of vectors that can be managed by an upper-level cluster L3-CL within layer L3 becomes 256. 4 .

[0101] Thus, whenever the layer level increases by 1, the total number of vectors that can be managed increases to 256 (=2^3). 8 Therefore, by setting the total number of layers included in the hierarchical cluster HC to a number corresponding to the size of the vector database being constructed, it is possible to construct a large-scale vector database exceeding one trillion. For example, with the total number of layers set to 8, it can manage a maximum of 2... 64 (=2 8× 8) vectors.

[0102] Figure 2 The hybrid index information 22 corresponding to the hierarchical cluster HC shown in the diagram can also include, for example, the following for each upper-level cluster: (1) a list of lower-level clusters representing the identifiers of their respective lower-level clusters belonging to the upper-level cluster; (2) relative position information (first relative position information) representing the positional relationship between the reference position of the upper-level cluster and the reference positions of each lower-level cluster belonging to the upper-level cluster; and (3) relative position information (second relative position information) representing the positional relationship between the reference position of the upper-level cluster and the reference positions of each co-level cluster corresponding to the upper-level cluster. Here, the co-level clusters corresponding to a certain upper-level cluster are other clusters included in the same layer as the layer that includes the upper-level cluster.

[0103] The first relative position information may include, for example, either or both of distance information, which indicates the distance between the reference position of the upper layer cluster and the reference position of each lower layer cluster, and position information, which indicates the orientation from the reference position of the upper layer cluster to the reference position of each lower layer cluster.

[0104] The second relative position information may include, for example, either or both of distance information, which indicates the distance between the reference position of the upper-level cluster and the reference positions of each cluster in the same level, and position information, which indicates the orientation from the reference position of the upper-level cluster to the reference positions of each cluster in the same level.

[0105] This relative position information (the first relative position information and the second relative position information) is obtained through pre-calculation. By using the relative position information (the first relative position information and the second relative position information), it is possible to efficiently search for the cluster with the closest reference position to the query from multiple lower-level clusters belonging to the higher-level cluster. Details of the search process using the relative position information (the first relative position information and the second relative position information) will be described later.

[0106] Furthermore, the hybrid index information 22 corresponding to the hierarchical cluster HC can also include, for example, the following for each lowest-level cluster: (1) a list of vectors to which the identifiers of the vectors belonging to the lowest-level cluster belong; (2) relative position information (third relative position information) indicating the positional relationship between the reference position of the lowest-level cluster and the reference positions of the corresponding clusters in the same level; (3) relative position information (fourth relative position information) indicating the positional relationship between the reference position of the lowest-level cluster and the vectors belonging to the lowest-level cluster; (4) relative position information (fifth relative position information) indicating the positional relationship between each vector belonging to the lowest-level cluster and other vectors in the lowest-level cluster; (5) an adjacent list of the identifiers of the adjacent clusters connected to the lowest-level cluster by edges; and (6) relative position information (sixth relative position information) indicating the positional relationship between the reference position of the lowest-level cluster and the reference positions of the corresponding clusters in the adjacent clusters.

[0107] The third relative position information may include either or both of the following: distance information, which indicates the distance between the reference position of the lowest layer cluster and the reference positions of each cluster in the same layer, and orientation information, which indicates the orientation from the reference position of the lowest layer cluster to the reference positions of each cluster in the same layer.

[0108] The fourth relative position information may include either or both of the following: distance information representing the distance between the reference position of the lowest layer cluster and each vector, and orientation information representing the orientation from the reference position of the lowest layer cluster to each vector.

[0109] The fifth relative position information is information representing the relative position between vectors belonging to the same lowest-level cluster. It may include either or both of distance information representing the distance between a vector and other vectors and orientation information representing the orientation from a vector to other vectors.

[0110] The sixth relative position information may include either or both of the following: distance information, which represents the distance between the reference position of the lowest layer cluster and the reference positions of each adjacent cluster corresponding to the lowest layer cluster; and orientation information, which represents the orientation from the reference position of the lowest layer cluster to the reference positions of each adjacent cluster corresponding to the lowest layer cluster.

[0111] These relative positional information (the 3rd, 4th, 5th, and 6th relative positional information) are obtained through pre-calculation. By using this relative positional information, it is possible to efficiently search for the lowest-level cluster with the closest reference position to the query from the lowest-level cluster of a certain upper-level cluster belonging to layer L1, and to efficiently search for the vector closest to the query among the vectors of each lowest-level cluster belonging to the search object. Details of the search processing using this relative positional information will be described later.

[0112] Next, an example of search processing using hierarchical clusters (HC) and inter-cluster graphs (CG) will be explained. Figure 3 This is a diagram illustrating an example of the search process performed in the approximate nearest neighbor search system 1 using the hybrid index structure 22S.

[0113] Based on the query vector Q (hereinafter referred to as query Q) received from the external device 2, the processor 11 begins a search process for an approximate nearest neighbor vector of query Q from the dataset 21.

[0114] The search process includes: (1) an approximate nearest neighbor cluster search process for searching the lowest-level cluster (approximate nearest neighbor cluster) that has the closest reference position to the query Q; and (2) an approximate nearest neighbor vector search process for searching the approximate nearest neighbor vector of the query Q from the group of vectors belonging to the approximate nearest neighbor cluster and the group of vectors belonging to one or more lowest-level clusters that are close to the approximate nearest neighbor cluster.

[0115] The near nearest neighbor cluster search process starts from the top layer L3.

[0116] Processor 11 sets the top-level cluster L3-CL as the target cluster for the approximate nearest neighbor cluster search. Processor 11 finds the lower-level cluster with the closest reference position to query Q from the lower-level clusters belonging to the top-level cluster L3-CL (here, the upper-level clusters L2-CL1 to L2-CL5 of layer L2). Figure 3In the above-top cluster L3-CL, the lower-level cluster with the shortest reference position to query Q among the lower-level clusters (the upper-level clusters L2-CL1 to L2-CL5 of layer L2) is the upper-level cluster L2-CL1. Therefore, the upper-level cluster L2-CL1 is found as the lower-level cluster belonging to the above-top cluster L3-CL and having the reference position closest to query Q.

[0117] When the upper-level cluster L2-CL1 is found to be a lower-level cluster belonging to the highest-level cluster L3-CL and having the reference position closest to query Q, the processor 11 sets the upper-level cluster L2-CL1 as the new object cluster. The processor 11 then finds the lower-level cluster with the reference position closest to query Q from the lower-level clusters belonging to the upper-level cluster L2-CL1 (here, the upper-level clusters L1-CL1 to L1-CL5 of layer L1). Figure 3 In the above, the lower-level cluster (L1-CL1 to L1-CL5, the upper-level clusters of layer L1) that has the shortest reference position to query Q is the upper-level cluster L1-CL1. Therefore, the upper-level cluster L1-CL1 is found as the lower-level cluster that belongs to the upper-level cluster L2-CL1 and has the closest reference position to query Q.

[0118] When the upper-level cluster L1-CL1 is found to be a lower-level cluster belonging to the upper-level cluster L2-CL and having the reference position closest to query Q, the processor 11 sets the upper-level cluster L1-CL1 as the new object cluster. The processor 11 then finds the lower-level cluster with the reference position closest to query Q from the lower-level clusters belonging to the upper-level cluster L1-CL1 (here, the lowest-level clusters CL1-CL5 of layer L0). Figure 3 In the above-level clusters L1-CL1, the lowest-level cluster CL3 is the lowest-level cluster that has the shortest reference position to the query Q among the lower-level clusters (CL1-CL5 of layer L0). Therefore, the lowest-level cluster CL3 is found as the lower-level cluster belonging to the upper-level clusters L1-CL1 and having the reference position closest to the query Q. The lowest-level cluster CL3 is determined to be the approximate nearest neighbor cluster, i.e., the search starting cluster used for approximate nearest neighbor vector search processing.

[0119] Thus, in the approximate nearest neighbor cluster search process using hierarchical clusters (HC), the search process for finding the lower-level cluster with the reference position closest to query Q is repeatedly performed at each level until one of the multiple lowest-level clusters (CL) is found as the lower-level cluster with the reference position closest to query Q. Furthermore, the lowest-level cluster (CL) with the reference position closest to query Q among the multiple lowest-level clusters (CL) is determined as the approximate nearest neighbor cluster (search start cluster). In addition, other methods besides those using hierarchical clusters (HC) can be used as methods for determining the search start cluster, such as exhaustive cluster search, LSH (Locality Sensitive Hash), or hierarchical graph methods.

[0120] When the lowest-level cluster CL3 is determined to be the approximate nearest neighbor cluster (search start cluster), the processor 11 performs a search for the search start cluster (here, the lowest-level cluster CL3), finding the vector closest to the query Q among the vectors V11 to V15 belonging to the lowest-level cluster CL3 as the nearest neighbor vector within the search start cluster (lowest-level cluster CL3). In other words, the processor 11 searches for the vector closest to the query Q among the vectors V11 to V15 belonging to the search start cluster (here, the lowest-level cluster CL3) as the nearest neighbor vector within the search start cluster (lowest-level cluster CL3).

[0121] Next, while traversing the inter-cluster graph CG, the processor 11 performs a search for one or more search object clusters (e.g., lower-level clusters CL1, CL2, ...) that are close to the search start cluster (lowest-level cluster CL3). For each of the more than one search object clusters, the processor finds the vector closest to the query Q in the vectors belonging to the search object cluster as the nearest neighbor vector within the search object cluster. In other words, while traversing the inter-cluster graph CG, the processor 11 selects one or more search object clusters and searches for the vector closest to the query Q in the vectors belonging to each of the more than one search object clusters as the nearest neighbor vector within each search object cluster. Furthermore, in Figure 3 In, with Figure 2 Similarly, edges are represented by thick lines.

[0122] Then, processor 11 outputs the nearest neighbor vector found in the search start cluster (lowest-level cluster CL3) and the vector closest to query Q from the nearest neighbor vectors found in more than one search object cluster (e.g., lowest-level clusters CL1, CL2, ...) as the search results (approximate nearest neighbor vectors). In this case, the search results (approximate nearest neighbor vectors) are sent back to external device 2 as a response to query Q.

[0123] Furthermore, the search process for finding the lower-level cluster with the reference position closest to query Q for each upper-level layer can be performed using relative position information obtained through prior calculation (e.g., first relative position information and second relative position information) as follows.

[0124] (1) The processor 11 sets the topmost cluster L3-CL as the object cluster, and uses the first relative position information corresponding to the object cluster (topmost cluster L3-CL) and the second relative position information corresponding to the lower clusters (upper clusters L2-CL1 to L2-CL5 of layer L2) belonging to the object cluster (topmost cluster L3-CL) to find the lower cluster (e.g., cluster L2-CL1) with the reference position closest to the query Q among the lower clusters (upper clusters L2-CL1 to L2-CL5 of layer L2) belonging to the object cluster (topmost cluster L3-CL). In other words, the processor 11 uses the first relative position information corresponding to the object cluster (the topmost layer cluster L3-CL) and the second relative position information corresponding to the lower layer clusters (the upper layer clusters L2-CL1 to L2-CL5 of layer L2) belonging to the object cluster (the topmost layer cluster L3-CL) to search for the lower layer cluster (e.g., cluster L2-CL1) with the reference position closest to the query Q from the lower layer clusters (the upper layer clusters L2-CL1 to L2-CL5 of layer L2) belonging to the object cluster (the topmost layer cluster L3-CL).

[0125] (2) Next, the processor 11 sets the found lower-level cluster (cluster L2-CL1) as the new object cluster, and uses the first relative position information corresponding to the new object cluster (cluster L2-CL1) and the second relative position information corresponding to the lower-level clusters (upper-level clusters L1-CL1 to L1-CL5 of layer L1) belonging to the new object cluster (cluster L2-CL1) to find the lower-level cluster (e.g., cluster L1-CL1) with the closest reference position to the query among the lower-level clusters (upper-level clusters L1-CL1 to L1-CL5 of layer L1) belonging to the new object cluster (cluster L2-CL1). In other words, the processor 11 uses the first relative position information corresponding to the new object cluster (cluster L2-CL1) and the second relative position information corresponding to the lower layer clusters (upper layer clusters L1-CL1 to L1-CL5 of layer L1) belonging to the new object cluster (cluster L2-CL1) to search for the lower layer cluster (e.g., cluster L1-CL1) with the closest reference position to the query from the lower layer clusters (upper layer clusters L1-CL1 to L1-CL5 of layer L1) belonging to the new object cluster (cluster L2-CL1).

[0126] (3) Then, the processor 11 repeatedly performs the process (2) including setting the found lower-level cluster as a new object cluster and finding the lower-level cluster with the closest reference position to the query in the lower-level cluster belonging to the new object cluster, until the found lower-level cluster reaches the lowest level L0, that is, until one of the multiple lowest-level clusters is found as the lower-level cluster with the closest reference position to the query.

[0127] When the found lower-level cluster reaches the lowest level L0, the found lower-level cluster becomes the search start cluster of the lowest level L0. The search process for finding the nearest neighbor vector within the search start cluster and the search process for finding the nearest neighbor vector within each search object cluster can be performed using relative position information obtained through pre-computation (e.g., the fourth relative position information and the fifth relative position information).

[0128] For example, consider the case where the lowest-level cluster CL3 is determined to be the starting cluster for the search. In this case, the processor 11 uses the fourth relative position information, which represents the positional relationship between the reference position B0-3 of the lowest-level cluster CL3 and each of the vectors V11 to V15, and the fifth relative position information, which represents the positional relationship between the vectors V11 to V15, to find the vector closest to the query Q among the vectors V11 to V15 belonging to the lowest-level cluster CL3.

[0129] Next, the structure of the inter-cluster graph (CG) will be explained. Figure 4 This is a diagram representing a structural example of the inter-cluster graph CG used in the approximate nearest neighbor search system 1.

[0130] An inter-cluster graph (CG) is not a graph that connects vectors, but rather a graph that connects clusters to which each vector belongs. This inter-cluster graph (CG) can be implemented, for example, using a hub and spoke (central-radial) structure.

[0131] A hub and spoke structure consists of a hub and several spokes extending from that hub. In an inter-cluster graph CG with a hub and spoke structure, a cluster is represented by a hub. Furthermore, each vector belonging to a cluster is represented by a spoke extending from the hub. A cluster (hub) corresponds to a node in the inter-cluster graph CG, and two clusters (two hubs) that are close to each other are connected by an edge.

[0132] exist Figure 4 In the example, clusters c1 to c5 are shown as the lowest level clusters.

[0133] Cluster c1 has four spokes corresponding to the four vectors v11 to v14 belonging to cluster c1. The four spokes of cluster c1 are represented by a list of vectors belonging to cluster c1 and relative positional information indicating the positional relationship between the reference position of cluster c1 and the respective positions of vectors v11 to v14. Furthermore, in Figure 4 In the diagram, spokes are represented by dashed arrows.

[0134] Cluster c2 has four spokes corresponding to the four vectors v21 to v24 belonging to cluster c2. The four spokes of cluster c2 are represented by a list of vectors belonging to cluster c2 and relative position information indicating the positional relationship between the reference position of cluster c2 and the respective positions of vectors v21 to v24.

[0135] Cluster c3 has four spokes corresponding to the four vectors v31 to v34 belonging to cluster c3. The four spokes of cluster c3 are represented by a list of vectors belonging to cluster c3 and relative position information indicating the positional relationship between the reference position of cluster c3 and the respective positions of vectors v31 to v34.

[0136] Cluster c4 has four spokes corresponding to the four vectors v41 to v44 belonging to cluster c4. The four spokes of cluster c4 are represented by a list of vectors belonging to cluster c4 and relative position information indicating the positional relationship between the reference position of cluster c4 and the respective positions of vectors v41 to v44.

[0137] Cluster c5 has four spokes corresponding to the four vectors v51 to v54 belonging to cluster c5. The four spokes of cluster c5 are represented by a list of vectors belonging to cluster c5 and relative position information indicating the positional relationship between the reference position of cluster c5 and the respective positions of vectors v51 to v54.

[0138] Cluster c1 is connected to cluster c2 via edge e12, to cluster c3 via edge e13, to cluster c4 via edge e14, to cluster c5 via edge e15, and to other clusters (not shown) via edge e16. These edges e12 to e16 are represented by an adjacency list corresponding to cluster c1. Furthermore, in Figure 4 In, with Figure 2 and Figure 3 Similarly, edges are represented by thick lines.

[0139] Cluster c2 is connected to cluster c1 via edge e12, to cluster c3 via edge e23, to cluster c4 via edge e24, to cluster c5 via edge e25, and to other clusters (not shown) via edge e27. These edges e12, e23 to e25, and e27 are represented by the adjacent list corresponding to cluster c2.

[0140] Cluster c3 is connected to cluster c1 via edge e13, to cluster c2 via edge e23, to cluster c4 via edge e34, to cluster c5 via edge e35, and to other clusters (not shown) via edge e37. These edges e13, e23, e34, e35, and e37 are represented by an adjacency list corresponding to cluster c3.

[0141] Cluster c4 is connected to cluster c1 via edge e14, to cluster c2 via edge e24, to cluster c3 via edge e34, and to cluster c5 via edge e45. These edges e14, e24, e34, and e45 are represented by an adjacency list corresponding to cluster c4.

[0142] Cluster c5 is connected to cluster c1 via edge e15, to cluster c2 via edge e25, to cluster c3 via edge e35, to cluster c4 via edge e45, and to other clusters (not shown) via edge e59. These edges e15, e25, e35, e45, and e59 are represented by an adjacency list corresponding to cluster c5.

[0143] Furthermore, another cluster B connected to a cluster A by an edge is called a neighboring cluster of cluster A.

[0144] In this way, by using a graph that connects clusters rather than vectors, the number of edges in the graph can be significantly reduced. For example, if 256 vectors can be registered in a cluster (hub), the number of edges in the graph can be reduced to 1 / 256. Therefore, since the number of adjacency lists used to represent edges in the graph can be reduced, even when building a large-scale vector database of over a billion, the increase in the size of the index information can be kept to a minimum.

[0145] Furthermore, in constructing a graph using vectors, many adjacency lists (edge ​​information) need to be rewritten each time a vector is appended to the vector database. However, in constructing a graph that connects clusters, only a new vector needs to be registered in one cluster (hub), without rewriting the adjacency lists (edge ​​information). This reduces the frequency of graph updates, thereby reducing the amount of data written to the SSD14, extending its lifespan, and shortening the time required to register a vector.

[0146] Furthermore, the number of vectors that can be registered in each cluster (hub) is limited to an upper limit. Therefore, new clusters need to be created infrequently, and the adjacent lists (edge ​​information) accompanying the creation of new clusters need to be rewritten. However, the number of clusters (hubs) is much smaller than the number of vectors, so the adverse impact on the lifespan of the SSD14 is minimal.

[0147] In this embodiment, the dataset (multiple vectors) 21 and the hybrid index information 22 (clustered index information and graph index information) are stored in a secondary storage device 14 such as an SSD. In this case, the adjacency list included in the graph index information is stored in a storage area within a secondary storage device 14 that is different from the storage area within the secondary storage device 14 storing the dataset (multiple vectors) 21.

[0148] For example, in an SSD, the adjacency list and multiple vectors are stored in distinct blocks within non-volatile memory. Even if a new vector is added to dataset 21, there is no need to update the vectors already stored in secondary storage device 14. On the other hand, the adjacency list is updated when clusters need to be added or deleted as vectors are added or deleted. Thus, by storing vectors and adjacency lists in distinct storage areas within non-volatile memory, it is possible to rewrite only the adjacency list without rewriting the vectors, preventing the degradation of SSD write amplification.

[0149] Figure 5 This is a graph illustrating cluster search using hierarchical clusters (HC) and approximate nearest neighbor vector search using inter-cluster graphs (CG).

[0150] Processor 11 sets the topmost hierarchical cluster L3-c of the hierarchical cluster HC as the object cluster and begins the search process from the topmost hierarchical cluster L3-c. Processor 11 finds the hierarchical cluster L2-c with the closest reference position to the query among the multiple hierarchical clusters L2-c belonging to the topmost hierarchical cluster L3-c.

[0151] The processor 11 sets the found upper-level cluster L2-c as the new object cluster, and finds an upper-level cluster L1-c with the closest reference position to the query among the multiple upper-level clusters L1-c1, L1-c2, ... belonging to the found upper-level cluster L2-c.

[0152] The processor 11 sets the found upper-level clusters L1-c as the new object clusters, and selects the lowest-level cluster c with the closest query reference position among the multiple lowest-level clusters c belonging to the found upper-level clusters L1-c as the approximate nearest neighbor cluster (search start cluster). For example, if the new object cluster is the upper-level cluster L1-c1, the processor 11 selects the lowest-level cluster c1, c2, ... belonging to the upper-level clusters L1-c1 with the closest query reference position as the approximate nearest neighbor cluster (search start cluster). Similarly, if the new object cluster is the upper-level cluster L1-c2, the processor 11 selects the lowest-level cluster c4, c5, ... belonging to the upper-level clusters L1-c2 with the closest query reference position as the approximate nearest neighbor cluster (search start cluster).

[0153] Furthermore, in the data space, there may be ranges with a large number of vectors v and ranges with a small number of vectors v. The lowest-level cluster located within the range with a small number of vectors v may skip layer L1 and belong to a higher-level cluster within layer L2. Figure 5 In this context, a reference position for the lowest-level cluster c3 is defined within a range containing a small number of vectors v, and a reference position for the upper-level cluster L2-c of layer L2 is defined within this range. In this case, the lowest-level cluster c3 can also skip layer L1 and directly belong to the upper-level cluster L2-c of layer L2. Furthermore, in Figure 5 In the text, a bold arrow with a dotted line indicates such a membership relationship.

[0154] Thus, in this embodiment, multiple upper layers of hierarchical clusters HC are configured on the upper side of the inter-cluster graph CG instead of the graph itself. Moreover, the processor 11 determines the approximate nearest neighbor cluster (search start cluster) by performing the following processes (1) to (3): (1) setting the uppermost layer cluster as the processing target cluster; (2) finding the lower layer cluster with the closest reference position to the query among the lower layer clusters belonging to the processing target cluster, and setting the found lower layer cluster as the new processing target cluster; (3) repeating the process (2) until the lowermost layer cluster is found as the lower layer cluster with the closest reference position to the query.

[0155] In the lowest level, first, find the nearest neighbor vector within the search start cluster, and set the distance from the nearest neighbor vector to the query as the temporary (provisional) nearest neighbor distance.

[0156] For example, if the lowest-level cluster c1 is determined to be an approximate nearest neighbor cluster (the search starting cluster), the vector closest to the query among vectors v11 to v14 is found as the nearest neighbor vector within the lowest-level cluster c1. Then, the distance from the nearest neighbor vector within the lowest-level cluster c1 to the query is set as the temporary nearest neighbor distance. The distance from the nearest neighbor vector within the lowest-level cluster c1 to the query is, for example, the Euclidean distance between the nearest neighbor vector within the lowest-level cluster c1 and the query.

[0157] Since the lowest-level cluster c1 is the lowest-level cluster with the reference position closest to the query, we also consider using a method that determines the nearest neighbor vector within the lowest-level cluster c1 as the approximate nearest neighbor vector of the query.

[0158] However, typically, the location of the query within the data space differs from the reference location of the lowest-level cluster c1. Although vectors v11–v14 are a group of vectors close to the reference location of the lowest-level cluster c1, there is a discrepancy between the distance from each of these vectors v11–v14 to the query and the distance from each of these vectors v11–v14 to the reference location of the lowest-level cluster c1. Therefore, there may not be a vector among vectors v11–v14 that is sufficiently close to the query; vectors that are sufficiently close to the query sometimes belong to another lowest-level cluster located near the lowest-level cluster c1.

[0159] Therefore, in the approximate nearest neighbor vector search process of this embodiment, while traversing the inter-cluster graph CG, a search process is further performed for one or more lowest-level clusters (search object clusters) that are closest to the lowest-level cluster c1. In this case, for each of the more than one search object cluster, the vector closest to the query is found among the vectors belonging to the search object cluster as the nearest neighbor vector within the search object cluster. Then, the nearest neighbor vector found from the lowest-level cluster c1 and the vector closest to the query vector among the nearest neighbor vectors found from the more than one search object cluster are output as the search result, i.e., the approximate nearest neighbor vector of the query.

[0160] In the search processing for the starting cluster and each search object cluster, the distance from the query vector to all vectors within the cluster can be calculated, and the vector with the smallest distance can be selected as the nearest neighbor vector within the cluster. Alternatively, the search processing for one or more lowest-level clusters (search object clusters) that are close to the lowest-level cluster c1 can be performed using the following steps.

[0161] For example, the lowest-level cluster c1 can be set as the base cluster, and one or more neighboring clusters of c1 can be set as one or more search object clusters. The search process can also end if no vector shorter than the temporary nearest neighbor distance is found in any of the neighboring clusters of the search object clusters. In this case, the temporary nearest neighbor distance is output as the approximate nearest neighbor vector of the query.

[0162] On the other hand, if a vector is found in a search object cluster cx from one or more search object clusters that is shorter than the temporary nearest neighbor distance from the query, the distance (Euclidean distance) from the found vector to the query can be set as the new temporary nearest neighbor distance, and one or more neighboring clusters of one or more neighboring clusters of the search object cluster cx other than the clusters that have been searched can be set as one or more new search object clusters.

[0163] The search process can also end if no vector shorter than the temporary nearest neighbor distance is found in any of the neighboring clusters of a new search object cluster. In this case, the new temporary nearest neighbor distance is output as the approximate nearest neighbor vector of the query.

[0164] Furthermore, if a vector is found in a search object cluster cy from one or more new search object clusters that is shorter than the temporary nearest neighbor distance from the query, the distance (Euclidean distance) from the found vector to the query can be set as a newer temporary nearest neighbor distance. Also, one or more neighboring clusters of the search object cluster cy, excluding the clusters that have already been searched, can be set as one or more newer search object clusters.

[0165] Next, refer to Figures 6A to 6D An example is given to illustrate the process of approximate nearest neighbor vector search. Figure 6A This is a graph representing the search process for the starting cluster (in this case, the lowest-level cluster c1). Figure 6B This is a graph representing the search process for the first search object cluster (in this case, the lowest-level cluster c2) that is closest to the search start cluster. Figure 6C This is a graph representing the search process for the second search object cluster (here, the lowest-level cluster c3). Figure 6D This is a graph representing the search process for one or more neighboring clusters of the second search target cluster (here, the lowest-level cluster c3) that are not already searched.

[0166] (Process 1) such as Figure 6A As shown, the lowest-level cluster c1, which has the reference position closest to query Q1, is determined as the search starting cluster, and the search process begins from the lowest-level cluster c1.

[0167] (Process 2) such as Figure 6BAs shown, the vector closest to query Q1 (in this case, vector v11) is found among vectors v11 to v14 belonging to the lowest-level cluster c1. That is, vector v11 is the nearest neighbor vector within the lowest-level cluster c1. The distance from vector v11 to query Q1 is set as the temporary nearest neighbor distance. It is highly probable that a vector with a shorter distance to query Q1 than the temporary nearest neighbor distance is found among clusters with a reference position close to the reference position of the lowest-level cluster c1. Among the neighboring clusters of the lowest-level cluster c1 (in this case, lowest-level clusters c2 to c5), the closest neighboring cluster to the reference position of the lowest-level cluster c1 is the lowest-level cluster c2. Therefore, by setting the lowest-level cluster c1 as the base cluster and proceeding along the inter-cluster graph CG, the search process for the lowest-level cluster c2 is then performed. Next, the process of finding a vector with a shorter distance to query Q1 than the temporary nearest neighbor distance from vectors v21 to v24 belonging to the lowest-level cluster c2 is performed.

[0168] (Process 3) If no vector is found in the lowest-level cluster c2 whose distance to query Q1 is shorter than the temporary nearest neighbor distance, perform a search process on the next lowest-level cluster. The next nearest neighbor to the second lowest-level cluster c1 is the lowest-level cluster c3. Therefore, as Figure 6C As shown, by setting the lowest-level cluster c1 as the base cluster and traversing along the inter-cluster graph CG, a search process is then performed for the lowest-level cluster c3. Next, a process is performed to find vectors from vectors v31 to v34 belonging to the lowest-level cluster c3 whose distance to query Q1 is shorter than the temporary nearest neighbor distance. Here, the distance from vector v32 to query Q1 is shorter than the temporary nearest neighbor distance. Therefore, vector v32 is found as the vector whose distance to query Q1 is shorter than the temporary nearest neighbor distance. Then, the distance from vector v32 to query Q1 is set as the new temporary nearest neighbor distance.

[0169] (Process 4) Figure 6D As shown, the lowest-level cluster c3 is set as the new base cluster. A search is performed on all neighboring clusters (c1, c2, c4, c5, etc.) of the lowest-level cluster c3, excluding those already searched (c1, c2, etc.). If no vector is found in the neighboring clusters (c4, c5, etc.) whose distance to query Q1 is shorter than the new temporary nearest neighbor distance, the process ends, and vector v32 is determined as the approximate nearest neighbor vector of query Q1.

[0170] Thus, in the approximate nearest neighbor vector search process, not only is the process of finding the nearest neighbor vector within the search start cluster (c1) belonging to the lowest-level cluster c1 (search start cluster) with the reference position closest to query Q1 performed, but also, by traversing the inter-cluster graph CG, a search is performed on more than one lowest-level cluster (search object cluster) close to the search start cluster, finding the nearest neighbor vector within each search object cluster. Therefore, not only is the search start cluster searched, but also each of the neighboring clusters of the search start cluster is searched. As a result, while significantly reducing the number of times edge information is rewritten, the same high search accuracy as the approximate nearest neighbor search algorithm using an inter-vector graph that connects vectors can be achieved.

[0171] Next, refer to Figures 7A to 7C Here is another example illustrating the process of approximate nearest neighbor vector search. Figure 7A This is a graph representing the search process for the starting cluster (in this case, the lowest-level cluster c1). Figure 7B This is a diagram representing the search process for the search object cluster (in this case, the lowest-level cluster c3) that has the closest orientation from the reference position of the search start cluster to the query orientation. Figure 7C This is a graph representing the search process for one or more neighboring clusters (excluding those already searched) of the search target cluster (in this case, the lowest-level cluster c3) that has the closest orientation from the reference position of the search starting cluster to the query orientation.

[0172] exist Figures 7A to 7C In the approximate nearest neighbor vector search process shown, it is assumed that the graph index information for each of the neighboring clusters (in this case, the lowest layer clusters c2 to c5) of the lowest layer cluster c1 includes orientation information representing the orientation from the reference position of the lowest layer cluster c1 to the reference position of the neighboring clusters.

[0173] (Process 1) such as Figure 7A As shown, the lowest-level cluster c1, which has the reference position closest to query Q1, is determined as the starting cluster for the search, and the search process begins from the lowest-level cluster c1. Then, the vector closest to query Q1 (in this case, vector v11) is found among the vectors v11 to v14 belonging to the lowest-level cluster c1. That is, vector v11 is the nearest neighbor vector within the lowest-level cluster c1. The distance from vector v11 to query Q1 is set as the temporary nearest neighbor distance.

[0174] (Process 2) Calculate the orientation from the reference position of the lowest layer cluster c1 to the query Q1.

[0175] This orientation is, for example, equivalent to the direction of the difference vector (vector difference) obtained by subtracting the reference position (reference vector) of the lowest-level cluster c1 from the query (query vector) Q1. Therefore, the orientation from the reference position of the lowest-level cluster c1 to the query Q1 can be obtained by calculating the difference vector obtained by subtracting the reference position (reference vector) of the lowest-level cluster c1 from the query (query vector) Q1. Furthermore, the orientation information obtained by further compressing this difference vector can also be used as the orientation information representing the orientation from the reference position of the lowest-level cluster c1 to the query Q1. As a method of compressing the difference vector, the dimension of the difference vector can be reduced. The orientation information compressed by reducing the dimension is also called an orientation hash. The orientation information contained in the hybrid index information 22 can also be represented by the orientation hash after dimension reduction.

[0176] The vector closer to query Q1 is more likely to exist at a position corresponding to an orientation similar to that of query Q1 as observed from the reference position of the lowest-level cluster c1. Therefore, using the orientation information corresponding to each of the lowest-level clusters c2 to c5, the lowest-level cluster among c2 to c5 with the orientation most similar to that from the reference position of the lowest-level cluster c1 to query Q1 is selected as the next search target cluster over other adjacent clusters. The lowest-level cluster with the orientation most similar to that from the reference position of the lowest-level cluster c1 to query Q1 is the cluster that is in an orientation similar to query Q1 when observed from the reference position of the lowest-level cluster c1.

[0177] For example, the lowest-level cluster corresponding to the azimuth information (azimuth hash) most similar to the azimuth information (azimuth hash) representing the direction from the reference position of the lowest-level cluster c1 to the query Q1 can be selected as the next search target cluster over other neighboring clusters. The similarity (consistency) between azimuth hashes can be calculated by computing the Hamming distance between these azimuth hashes.

[0178] exist Figure 7B In this context, the lowest-level cluster c3 is the one with the orientation most similar to the orientation from the reference position of the lowest-level cluster c1 to the query Q1. Therefore, the lowest-level cluster c3 is selected as the next search target cluster with priority over other neighboring clusters (c2, c4, c5, etc.). Thus, as... Figure 7BAs shown, by setting the lowest-level cluster c1 as the base cluster and traversing along the inter-cluster graph CG, a search process for the lowest-level cluster c3 is then performed. Next, a process is executed to find vectors from vectors v31 to v34 belonging to the lowest-level cluster c3 whose distance to query Q1 is shorter than the temporary nearest neighbor distance. Here, the distance from vector v32 to query Q1 is shorter than the temporary nearest neighbor distance. Therefore, vector v32 is found as a vector whose distance to query Q1 is shorter than the temporary nearest neighbor distance. Then, the distance from vector v32 to query Q1 is set as the new temporary nearest neighbor distance.

[0179] (Process 3) such as Figure 7C As shown, the lowest-level cluster c3 is set as the new base cluster. A search is performed on all neighboring clusters (c1, c2, c4, c5, etc.) of the lowest-level cluster c3 (excluding the already searched cluster (c1)). If no vector is found in the neighboring clusters (c2, c4, c5, etc.) whose distance to query Q1 is shorter than the new temporary nearest neighbor distance, the process ends, and vector v32 is determined as the approximate nearest neighbor vector of query Q1.

[0180] Next, the process of adding new clusters to the inter-cluster graph will be explained.

[0181] Figure 8 This is a graph representing the index update process used to append new clusters. As described above, when appending a new vector to dataset 21 of the vector database, the cluster with the closest reference position to the new vector is determined among the existing clusters (the existing lowest-level clusters), and the new vector is registered in the vector list corresponding to the determined cluster.

[0182] If the number of vectors belonging to the identified cluster exceeds the upper limit due to the registration of new vectors, a new cluster is created, and then the created cluster is appended to the inter-cluster graph CG.

[0183] exist Figure 8 In the middle, it is envisioned that in Figure 5 In the case where the new vector Va is appended to dataset 21 under the given conditions, we can also consider the case where the cluster with the reference position closest to the new vector Va (the lowest-level cluster) is the lowest-level cluster c3.

[0184] exist Figure 5 In this state, there are already 4 vectors v31 to v34 belonging to the lowest-level cluster c3. Therefore, if the upper limit of the number of vectors that can belong to each cluster is, for example, 4, the registration of the new vector Va will cause an overflow of the lowest-level cluster c3.

[0185] Therefore, as Figure 8As shown, a new cluster (new lowest-level cluster) c0 is created with a reference position close to the reference position of the lowest-level cluster c3. Then, a cluster appending process is performed to append this lowest-level cluster c0 to the inter-cluster graph CG. In the cluster appending process, the processor 11 performs the following processes.

[0186] Processor 11 registers the identifier of the lowest-level cluster c3 and the identifiers of its neighboring clusters (lowest-level clusters c1, c2, c4, c5) in the adjacent list corresponding to the new lowest-level cluster c0. The neighboring clusters of the lowest-level cluster c3 can each be determined by obtaining the adjacent list of the lowest-level cluster c3 from the graph index information. Alternatively, the process of registering the identifiers of the neighboring clusters (lowest-level clusters c1, c2, c4, c5) in the adjacent list corresponding to the new lowest-level cluster c0 can be executed sequentially starting from the neighboring clusters with the shortest distance when viewed from the new lowest-level cluster c0. Furthermore, the lowest-level cluster c2 is also a neighboring cluster of the lowest-level cluster c3. However, if the distance from the new lowest-level cluster c0 to the lowest-level cluster c2 exceeds a certain baseline value, and / or if the total number of clusters registered in the neighboring list corresponding to the new lowest-level cluster c0 reaches a certain upper limit value, the identifier of the lowest-level cluster c2 may not be registered in the neighboring list corresponding to the new lowest-level cluster c0.

[0187] Next, the processor 11 registers the identifier of the new lowest-level cluster c0 in the adjacent list corresponding to the lowest-level cluster c3. Thus, the identifier of the lowest-level cluster c3 is registered in the adjacent list corresponding to the new lowest-level cluster c0, and the identifier of the new lowest-level cluster c0 is registered in the adjacent list corresponding to the lowest-level cluster c3. Therefore, the new lowest-level cluster c0 and the lowest-level cluster c3 are connected by edge e03.

[0188] Similarly, the processor 11 registers the identifier of the new lowest-level cluster c0 in the adjacent lists corresponding to the lowest-level cluster c1, the lowest-level cluster c4, and the lowest-level cluster c5, respectively. Thus, the new lowest-level cluster c0 is connected to the lowest-level cluster c1 via edge e01, to the lowest-level cluster c4 via edge e04, and to the lowest-level cluster c5 via edge e05.

[0189] In addition, the existing edge e35 between the lowest-level cluster c3 and the lowest-level cluster c5 can also be deleted as needed.

[0190] For example, if the registration of a new identifier for the lowest-level cluster c0 causes an overflow where the total number of clusters registered in the adjacent list of the lowest-level cluster c3 exceeds the upper limit, the identifier of the cluster with the longest distance from the lowest-level cluster c3 can be deleted from the adjacent list of the lowest-level cluster c3. However, if the cluster with the longest distance is an isolated cluster (no longer having adjacent clusters) due to the deletion of its identifier, the identifier of the second longest-distance cluster is deleted.

[0191] Additionally, the processor 11, for example, registers the identifier of the new lowest-level cluster c0 in the lower-level cluster list of the upper-level cluster L2-c.

[0192] Then, processor 11 determines, from all vectors already belonging to the lowest-level cluster c3 and the new vector Va, one or more vectors that are closer to the reference position of the new lowest-level cluster c0 than the reference position closest to the lowest-level cluster c3. Next, processor 11 registers the identifiers of each of the determined vectors in the vector list corresponding to the new lowest-level cluster c0. Furthermore, processor 11 removes the identifiers of each of the determined vectors that were registered in the vector list of the lowest-level cluster c3 from the vector list of the lowest-level cluster c3.

[0193] exist Figure 8 The example shown is the case where a new vector Va is appended to the new lowest-level cluster c0 and the vector v34 is moved from the lowest-level cluster c3 to the new lowest-level cluster c0.

[0194] Furthermore, the same processing is performed on all neighboring clusters of the new lowest-level cluster c0 as on the lowest-level cluster c3. Thus, for example, if vector v41 is closer to the reference position of the new lowest-level cluster c0 than to the reference position closer to the lowest-level cluster c4, then... Figure 8 As shown, vector v41 moves from the lowest-level cluster c4 to the new lowest-level cluster c0.

[0195] In addition, there are cases where, during the cluster appending process, the new lowest-level cluster c0 is determined as the starting cluster for the search.

[0196] In this case, the vector list belonging to the new lowest-level cluster c0 sometimes does not contain any vectors, or only a subset of vectors that should belong to the new lowest-level cluster c0 are registered. In this situation, it is difficult to correctly search for the approximate nearest neighbor vector. Therefore, processor 11 performs the following processing.

[0197] When it is detected that "a new lowest-level cluster c0 is determined as the search start cluster during the cluster appending process", processor 11 adds all vectors registered in the vector list of the lowest-level cluster c3 as search objects. Then, processor 11 searches for the vector closest to the query from all vectors registered in the vector list of the new lowest-level cluster c0 and all vectors registered in the vector list of the lowest-level cluster c3, and uses it as the nearest neighbor vector within the search start cluster.

[0198] Therefore, even if the movement of the vector from the lowest-level cluster c3 to the new lowest-level cluster c0 is not yet complete, and the new lowest-level cluster c0 is determined as the search start cluster, the nearest neighbor vector within the search start cluster can still be correctly searched.

[0199] Next, the process of deleting a cluster (lowest-level cluster) from the inter-cluster graph CG will be explained. Figure 9 It is a graph representing the index update process used to delete clusters whose vector count becomes zero.

[0200] When deleting a vector from dataset 21 in the vector database, processor 11 determines the cluster (lowest-level cluster) to which the vector belongs and deletes the vector from the list of vectors belonging to the determined cluster.

[0201] Sometimes, due to the deletion of a vector, the number of vectors belonging to the determined cluster will become zero.

[0202] For example, in Figure 9 The example shown is the case where the number of vectors in the vector list of the lowest-level cluster c4 becomes zero due to the deletion of vector v42 of the deleted object.

[0203] In this case, the processor 11 obtains the neighbor list of the lowest-level cluster c4 from the graph index information, and determines one or more clusters registered in the obtained neighbor list (that is, one or more neighboring clusters of the lowest-level cluster c4). Figure 9 In the study, the lowest-level clusters c0, c1, c2, c3, and c5 were identified as the neighboring clusters of the lowest-level cluster c4.

[0204] Next, the processor 11 performs a process of deleting the identifier of the lowest-level cluster c4 from the neighbor list of each neighboring cluster of the lowest-level cluster c4. In this case, the identifier of the lowest-level cluster c4 is deleted from the neighbor list of the lowest-level cluster c0, and similarly, the identifier of the lowest-level cluster c4 is also deleted from the neighbor lists of the lowest-level clusters c1, c2, c3, and c5. As a result, the edge e04 connecting the lowest-level cluster c0 and the lowest-level cluster c4, the edge e14 connecting the lowest-level cluster c1 and the lowest-level cluster c4, the edge e24 connecting the lowest-level cluster c2 and the lowest-level cluster c4, the edge e34 connecting the lowest-level cluster c3 and the lowest-level cluster c4, and the edge e45 connecting the lowest-level cluster c5 and the lowest-level cluster c4 are deleted.

[0205] Then, for each of the neighboring clusters of the lowest-level cluster c4, the processor 11 performs the process of determining the clusters that are not registered in the neighbor list of the neighboring clusters but are registered in the neighbor list of the lowest-level cluster c4, and the process of registering the identifier of the determined cluster in the neighbor list of the neighboring clusters.

[0206] For example, regarding the lowest-level cluster c3, the cluster that is not registered in the neighbor list of the lowest-level cluster c3 but is registered in the neighbor list of the lowest-level cluster c4 is the lowest-level cluster c5.

[0207] Similarly, regarding the lowest-level cluster c5, the cluster that is not registered in the neighbor list of the lowest-level cluster c5 but is registered in the neighbor list of the lowest-level cluster c4 is the lowest-level cluster c3.

[0208] Therefore, the processes of registering the identifier of the lowest-level cluster c5 in the adjacent list of the lowest-level cluster c3 and registering the identifier of the lowest-level cluster c3 in the adjacent list of the lowest-level cluster c5 are performed. As a result, the lowest-level cluster c3 and the lowest-level cluster c5 are connected to each other through edge e35.

[0209] Additionally, for example, regarding the lowest-level cluster c2, the cluster that is not registered in the neighbor list of the lowest-level cluster c2 but is registered in the neighbor list of the lowest-level cluster c4 is the lowest-level cluster c0.

[0210] Similarly, regarding the lowest-level cluster c0, the cluster that is not registered in the neighbor list of the lowest-level cluster c0 but is registered in the neighbor list of the lowest-level cluster c4 is the lowest-level cluster c2.

[0211] Therefore, the processes of registering the identifier of the lowest-level cluster c0 in the adjacent list of the lowest-level cluster c2 and registering the identifier of the lowest-level cluster c2 in the adjacent list of the lowest-level cluster c0 are performed. As a result, the lowest-level cluster c2 and the lowest-level cluster c0 are connected to each other through edge e02.

[0212] Furthermore, if the neighbor list of one of the lowest-level clusters c1, c2, c3, and c5 overflows, the identifier of the cluster that is the longest distance from the lowest-level cluster among the clusters that are not isolated even if deleted will be removed from the neighbor list of the lowest-level cluster.

[0213] Figure 10 This is a diagram representing an example of upper-level cluster index information 221. Upper-level cluster index information 221 is the cluster type index information corresponding to each upper-level cluster.

[0214] The upper-level cluster index information 221 includes multiple entries corresponding to multiple upper-level clusters. Each entry includes, for example, a cluster ID, a reference position, the cluster ID of the lower-level cluster, the relative position information (distance information, orientation information) of the lower-level cluster, the cluster ID of the same-level cluster, and the fields of the relative position information (distance information, orientation information) of the same-level cluster.

[0215] In the entry corresponding to a certain parent-level cluster, the cluster ID field represents the identifier (cluster ID) assigned to that parent-level cluster. The cluster ID is information that can uniquely identify the corresponding cluster.

[0216] The reference position field indicates the reference position (absolute position information) of the corresponding upper-level cluster. As the reference position of a certain upper-level cluster, the reference position of any lower-level cluster belonging to that upper-level cluster can also be used.

[0217] The cluster ID field of a lower-level cluster represents a list of more than one cluster IDs assigned to one or more lower-level clusters belonging to the corresponding upper-level cluster. For example, the cluster ID field of a lower-level cluster lists more than one cluster ID. Lower-level clusters belonging to upper-level clusters are also referred to as their parent lower-level clusters.

[0218] The relative position information field of a lower-level cluster indicates the positional relationship between the reference position of the corresponding upper-level cluster and the reference positions of each of the lower-level clusters belonging to that upper-level cluster. Specifically, the relative position information field of a lower-level cluster includes distance information indicating the distance between the reference position of the upper-level cluster and the reference positions of each of the lower-level clusters. For example, the relative position information field of a lower-level cluster may list one or more distance information corresponding to one or more of its subordinate lower-level clusters.

[0219] The relative position information field of the lower-level clusters may also include azimuth information indicating the orientation from the reference position of the upper-level cluster to the respective reference position of the lower-level clusters.

[0220] The cluster ID field of the lower-level cluster and the relative position information field of the lower-level cluster are used as the lower-level cluster list 221B. The lower-level cluster list 221B is used to maintain information related to each lower-level cluster belonging to each upper-level cluster for each upper-level cluster.

[0221] The cluster ID field of a same-level cluster represents a list of more than one cluster IDs assigned to one or more clusters located at the same level as the corresponding parent cluster (same-level cluster). For example, the cluster ID field of a same-level cluster lists more than one cluster ID.

[0222] The relative position information field of a cluster within the same layer indicates the positional relationship between the reference position of the corresponding parent cluster and the reference positions of each of the other clusters within the same layer. Specifically, the relative position information field includes distance information indicating the distance between the reference position of the parent cluster and the reference positions of each cluster within the same layer. For example, the relative position information field may list more than one distance information corresponding to one or more clusters within the same layer.

[0223] The relative position information field of the same layer cluster may also include orientation information indicating the orientation from the reference position of the upper layer cluster to the reference position of each of the same layer clusters.

[0224] The cluster ID field of the same-level cluster and the relative position information field of the same-level cluster are used as the same-level cluster list 221A. The same-level cluster list 221A is used to maintain information related to each same-level cluster located at the same level as the upper-level cluster for each upper-level cluster.

[0225] Based on the above configuration, the search unit 113 can use the upper-level cluster index information 221 to perform an approximate nearest neighbor cluster search.

[0226] Furthermore, when using the reference position of a lower-level cluster within a higher-level cluster as the reference position of a higher-level cluster, the "relative position information of the same-level cluster" corresponding to the lower-level cluster can be used as the "relative position information of the lower-level cluster" corresponding to the higher-level cluster.

[0227] Figure 11 This is a diagram representing an example of the lowest-level cluster index information 222. The lowest-level cluster index information 222 is the cluster type index information corresponding to each lowest-level cluster.

[0228] The lowest level cluster index information 222 includes cluster type index information 222-1 corresponding to each lowest level cluster and graph type index information 222-2 corresponding to each lowest level cluster.

[0229] First, the cluster type index information 222-1 corresponding to each of the lowest level clusters is explained.

[0230] The cluster index information 222-1 includes multiple entries corresponding to multiple lowest-level clusters. Each entry includes, for example, fields for cluster ID, reference position, cluster ID of the same-level cluster, relative position information (distance information, orientation information) of the same-level cluster, vector ID, relative position information (distance information, orientation information) of the vector relative to the reference position, and relative position information between vectors.

[0231] In the entry corresponding to a specific lowest-level cluster, the cluster ID field represents the identifier (cluster ID) assigned to that lowest-level cluster. The cluster ID is information that can uniquely identify the corresponding cluster.

[0232] The reference position field indicates the reference position (absolute position information) of the corresponding lowest-level cluster. Any vector belonging to that lowest-level cluster can also be used as the reference position for that cluster.

[0233] The Cluster ID field of a same-level cluster represents a list of more than one cluster IDs assigned to one or more clusters located at the same level as the corresponding lowest-level cluster. For example, the Cluster ID field of a same-level cluster may list these more than one cluster IDs.

[0234] The relative position information field for clusters within the same layer indicates the positional relationship between the reference position of the lowest-level cluster and the reference positions of other clusters within the same layer. Specifically, the relative position information field includes distance information indicating the distance between the reference position of the lowest-level cluster and the reference positions of other clusters within the same layer. For example, the relative position information field may list more than one distance information corresponding to one or more clusters within the same layer.

[0235] The relative position information field of the same layer cluster may also include orientation information indicating the orientation from the reference position of the lowest layer cluster to the reference position of each of the same layer clusters.

[0236] The cluster ID field of the same-level cluster and the relative position information field of the same-level cluster are used as the same-level cluster list 222A. The same-level cluster list 222A is used to maintain information related to each same-level cluster located in the same level as the lowest-level cluster for each lowest-level cluster.

[0237] The Vector ID field represents a list of one or more vector IDs assigned to one or more vectors belonging to the corresponding lowest-level cluster. For example, the Vector ID field lists these one or more vector IDs. Vectors belonging to the lowest-level cluster are also referred to as the belonging vectors.

[0238] The relative position information field of a vector relative to a reference position indicates the positional relationship between the reference position of the corresponding lowest-level cluster and each vector belonging to that lowest-level cluster. Specifically, the relative position information field of a vector relative to a reference position includes distance information indicating the distance between the reference position of the lowest-level cluster and each of its respective vectors. For example, the relative position information field of a vector relative to a reference position may list more than one distance information corresponding to each of its more than one belonging vector.

[0239] The relative position information field of a vector with respect to a reference position may also include orientation information representing the orientation from the reference position of the lowest layer cluster to the orientation of each vector.

[0240] The relative position information field between vectors indicates the positional relationship between vectors belonging to the corresponding lowest-level cluster. Specifically, for each belonging vector, the relative position information field includes distance information indicating the distance between that belonging vector and each of its other belonging vectors.

[0241] The relative position information field between vectors can also include orientation information for each belonging vector, indicating the direction from that belonging vector to other belonging vectors.

[0242] The vector ID field, the vector's relative position information field relative to the reference position, and the relative position information field between vectors are used as the belonging vector list 222B. The belonging vector list 222B is used to maintain information related to each vector belonging to the lowest level cluster for each lowest level cluster.

[0243] The graph index information 222-2 corresponding to each of the lowest-level clusters includes multiple entries corresponding to each of the multiple lowest-level clusters. Each entry includes, for example, fields such as the adjacent cluster ID and the relative position information of the adjacent clusters.

[0244] The Adjacent Cluster ID field represents a list of more than one cluster IDs assigned to each of the other lower-level clusters (adjacent clusters) connected to the corresponding lower-level cluster by an edge. For example, this adjacent cluster ID is listed in the Adjacent Cluster ID field.

[0245] The relative position information field of adjacent clusters indicates the positional relationship between the reference position of the corresponding lowest-level cluster and the reference positions of its neighboring clusters. Specifically, the relative position information field of adjacent clusters includes distance information indicating the distance between the reference position of the lowest-level cluster and the reference positions of its neighboring clusters. For example, the relative position information field of adjacent clusters may list more than one distance information corresponding to one or more neighboring clusters.

[0246] The relative position information field of adjacent clusters may also include orientation information indicating the orientation from the reference position of the lowest layer cluster to the respective reference positions of the adjacent clusters.

[0247] The cluster ID field of the adjacent cluster and the relative position information field of the adjacent cluster are used as the adjacent list 222C. The adjacent list 222C is used to maintain information representing the relationship between the lowest-level cluster and each adjacent cluster corresponding to the lowest-level cluster for each lowest-level cluster.

[0248] Furthermore, when using a vector belonging to a certain lowest-level cluster as the reference position (reference vector), the "relative position information between vectors" corresponding to that vector can be used as the "relative position information of the vector relative to the reference position" corresponding to that lowest-level cluster.

[0249] Next, the principle of approximate nearest neighbor vector search using the distance between vectors will be explained. Figure 12A , Figure 12B as well as Figure 12C This is a diagram illustrating the principle of approximate nearest neighbor vector search using the distance between vectors.

[0250] Here, we envision receiving a query Q2 based on a query from external device 2, and obtaining a temporary nearest neighbor and a temporary nearest neighbor distance Ln through a search of any cluster (the lowest-level cluster) based on query Q2. Then, we designate another cluster as the object cluster and further perform an approximate nearest neighbor vector search. The temporary nearest neighbor is the nearest neighbor vector within that lowest-level cluster found through a search of any cluster (the lowest-level cluster) based on query Q2. The temporary nearest neighbor distance Ln is the distance between the temporary nearest neighbor and query Q2. Multiple vectors belong to the object cluster. Figure 12A In this context, reference point B1 (reference vector B1) is the reference position of the object cluster. Within the object cluster, the distances between vectors are calculated beforehand.

[0251] like Figure 12A As shown, the search unit 113 calculates the distance Lc between the reference point B1 of the object cluster and the query Q2. The distance Lc is longer than the temporary nearest neighbor distance Ln. In this case, ideally, it would be desirable to search for vectors within a radius 501 centered on the query Q2. However, the distances between the query Q2 and each vector are unknown before calculation. Therefore, to determine whether each vector is inside or outside the range 501, the distances between the query Q2 and each vector must be calculated, increasing the computational load.

[0252] Therefore, as Figure 12BAs shown, the search unit 113 determines a search range 504 obtained by subtracting range 503 from range 502. Range 502 is a range centered on the reference point B1 with a radius of (Lc+Ln), which is the distance obtained by adding the temporary nearest neighbor distance Ln to the distance Lc. Range 503 is a range centered on the reference point B1 with a radius of (Lc-Ln), which is the distance obtained by subtracting the temporary nearest neighbor distance Ln from the distance Lc. Search range 504 is a range centered on the reference point B1 that includes all vectors whose distance from query Q2 may be less than the temporary nearest neighbor distance Ln.

[0253] Next, as Figure 12C As shown, the search unit 113 selects vector 505 within the search range 504. The search unit 113 calculates the distance Ln2 between vector 505 and query Q2. The search unit 113 determines whether the distance Ln2 is less than the temporary nearest neighbor distance Ln. Here, since the distance Ln2 is less than the temporary nearest neighbor distance Ln, the search unit 113 sets vector 505 as the new temporary nearest neighbor. In addition, the search unit 113 sets the distance Ln2 as the new temporary nearest neighbor distance.

[0254] Search unit 113 determines a search range 509 obtained by removing range 507 from range 508. Range 508 is a range centered on reference point B1 with a radius of (Lc+Ln2), which is the distance obtained by adding the temporary nearest neighbor distance Ln2 to the distance Lc. Range 507 is a range centered on reference point B1 with a radius of (Lc-Ln2), which is the distance obtained by subtracting the temporary nearest neighbor distance Ln2 from the distance Lc. Search range 509 includes all vectors whose distance from query Q2 may be less than the temporary nearest neighbor distance Ln2. Search unit 113 can determine whether each vector is within search range 509 based on the pre-calculated distance between reference point B1 and each vector. Thus, search unit 113 can exclude vectors outside search range 509 from the search targets.

[0255] Furthermore, the search unit 113 determines a search range 510 with a radius of 2Ln2 centered on the temporary nearest neighbor 505. Vectors within the search range 510 may be closer to query Q2 than the temporary nearest neighbor 505. The search unit 113 can determine whether each vector of the search object is within the search range 510 based on the pre-calculated distance between vectors. Thus, the search unit 113 can exclude vectors outside the search range 510 from the search objects.

[0256] Furthermore, the search unit 113 determines the search range 511 where the search range 509 and the search range 510 overlap. Vectors within the search range 511 may be closer to query Q2 than the temporary nearest neighbor 505. The search unit 113 can determine whether each vector of the search object is within the search range 511 based on pre-calculated distances between vectors, i.e., the distance between the reference point B1 and each vector and the distance between the temporary nearest neighbor 505 and each vector. Thus, the search unit 113 can exclude vectors outside the search range 511 from the search objects.

[0257] If the search unit 113 finds a vector within the search range 511 that has not been searched and whose distance from query Q2 is shorter than the temporary nearest neighbor distance Ln, it sets that vector as a new temporary nearest neighbor and further narrows the search range, performing a search for vectors within the search range. The search unit 113 repeats this process, for example, until there are no unsearched vectors within the search range.

[0258] If there are no unsearched vectors within the search range, or if any unsearched vector within the search range is at least a temporary nearest neighbor distance (Ln2) from query Q2, the search unit 113 outputs the vector last set as the temporary nearest neighbor as an approximate nearest neighbor vector. For example, if there are no unsearched vectors within the search range 511, or if any unsearched vector within the search range 511 is at least a temporary nearest neighbor distance (Ln2) from query Q2, the search unit 113 outputs the vector 505 last set as the temporary nearest neighbor as an approximate nearest neighbor vector.

[0259] Based on the above principle of approximate nearest neighbor vector search, the search unit 113 can use the pre-calculated distance between vectors to narrow the search range of vectors and efficiently search for approximate nearest neighbor vectors.

[0260] Next, an example illustrating the principle of approximate nearest neighbor vector search using the distance and orientation between vectors will be given. Figure 13A , Figure 13B , Figure 13C as well as Figure 13D This is a diagram illustrating the principle of approximate nearest neighbor vector search using distance and orientation between vectors.

[0261] Here, we envision receiving a query Q3 based on a query from external device 2, and setting a first temporary nearest neighbor 601 that is assumed to be close to query Q3. The first temporary nearest neighbor 601 is, for example, any vector included in dataset 21 (vector database). This arbitrary vector can be, for example, a vector with a specific ID among multiple vectors included in dataset 21, or a vector randomly selected from these multiple vectors. Furthermore, these multiple vectors each belong to any one of the multiple lowest-level clusters CL of the hierarchical cluster HC. If the approximate nearest neighbor cluster for query Q3 is determined among the multiple lowest-level clusters CL, the reference vector of the approximate nearest neighbor cluster can also be used as the first temporary nearest neighbor 601.

[0262] like Figure 13A As shown, the search unit 113 calculates the distance Ln1 between the first temporary nearest neighbor 601 and the query Q3. The distance between vectors calculated by the search unit 113 is, for example, Euclidean distance. In this case, ideally, it is desirable to search for vectors within a radius 602 centered on the query Q3 with a radius Ln1. In other words, it is desirable to exclude vectors outside the range 602 from the search objects. However, the distance between the query Q3 and each vector is unknown before calculation. Therefore, in order to determine whether each vector is inside or outside the range 602, the distance between the query Q3 and each vector must be calculated, increasing the computational load.

[0263] Therefore, as Figure 13B As shown, the search unit 113 determines a search range 603 with a radius of 2Ln1 centered on the first temporary nearest neighbor 601. Vectors within the search range 603 may be closer to query Q3 than the first temporary nearest neighbor 601.

[0264] The distances between vectors are calculated in advance (i.e., calculated before receiving query Q3). Specifically, for example, the distances between the first temporary nearest neighbor 601 and the respective vectors of the search object are calculated in advance.

[0265] The search unit 113 can determine whether each vector of the search object is within the search range 603 based on the pre-calculated distance between vectors. Therefore, the search unit 113 can exclude vectors outside the search range 603 from the search objects.

[0266] exist Figure 13C In the example shown, the search range 603 includes five vectors 611 to 615. The search unit 113 can determine that all five vectors 611 to 615 are within the search range 603 based on the pre-calculated distance between the vectors.

[0267] In addition, the search unit 113 calculates the orientation information α1, which represents the orientation from the first temporary nearest neighbor 601 to the query Q3. The search unit 113 searches sequentially within the search range 603, focusing on a specific orientation range 604 centered on the orientation information α1, starting from the point closest to the orientation information α1.

[0268] The orientation information between vectors is calculated in advance. Specifically, for example, the orientation information representing the orientation from the first temporary nearest neighbor 601 to vectors 611 to 615 is calculated in advance.

[0269] Based on pre-calculated distances and orientation information between vectors, the search unit 113 obtains a vector 611 within the search range 603 whose orientation information is more similar to orientation information α1 relative to the first temporary nearest neighbor 601. Specifically, the search unit 113, for example, obtains a vector 611 within the search range 603 whose orientation information corresponds to orientation information α1 whose Hamming distance is below a threshold.

[0270] The search unit 113 calculates the distance Ln2 between the obtained vector 611 and the query Q3. The search unit 113 determines whether the distance Ln2 is less than the temporary nearest neighbor distance Ln1. Here, since the distance Ln2 is less than the distance Ln1, the search unit 113 sets the vector 611 as the second temporary nearest neighbor 611.

[0271] In this case, ideally, we would like to search for vectors within a radius Ln2 of 605 centered on query Q3. In other words, we would like to exclude vectors outside the range 605 from the search. However, as mentioned above, the distances between query Q3 and each vector (e.g., vectors 612–615) are unknown before calculation.

[0272] Therefore, as Figure 13D As shown, the search unit 113 determines a search range 606 with a radius of 2Ln2 centered on the second temporary nearest neighbor 611. Vectors within the search range 606 may be closer to the query Q3 than the second temporary nearest neighbor 611. The search unit 113 can determine whether each vector of the search object is within the search range 606 based on the pre-calculated distance between vectors. Thus, the search unit 113 can exclude vectors outside the search range 606 from the search objects.

[0273] Furthermore, the search unit 113 determines a search range 609 obtained by removing range 608 from range 607. Range 607 is a range centered on the first temporary nearest neighbor 601 with a radius of (Ln1+Ln2), which is the distance obtained by adding distance Ln1 to distance Ln2. Range 608 is a range centered on the first temporary nearest neighbor 601 with a radius of (Ln1-Ln2), which is the distance obtained by subtracting distance Ln2 from distance Ln1. Vectors within search range 609 may be closer to query Q3 than the second temporary nearest neighbor 611. The search unit 113 can determine whether each vector of the search object is within search range 609 based on the pre-calculated distance between vectors. Thus, the search unit 113 can exclude vectors outside search range 609 from the search objects.

[0274] Furthermore, the search unit 113 determines the search range 610 where search range 606 and search range 609 overlap. Vectors within search range 610 may be closer to query Q3 than the second temporary nearest neighbor 611. The search unit 113 can determine whether each vector of the search object is within search range 610 based on pre-calculated distances between vectors, i.e., distances between the first temporary nearest neighbor 601 and each vector, and distances between the second temporary nearest neighbor 611 and each vector. Thus, the search unit 113 can exclude vectors outside search range 610 from the search objects.

[0275] If, within the search range 610, there exists a vector that has not been searched and whose distance to query Q3 is shorter than its distance to Ln2, the search unit 113 sets this vector as a new temporary nearest neighbor and further narrows the search range, performing a process to search for vectors within the search range. The search unit 113 repeats this process, for example, until there are no unsearched vectors within the search range. Furthermore, unsearched vectors are those whose distance to the query has not yet been calculated (evaluated).

[0276] If there are no unsearched vectors within the search range, the search unit 113 outputs the vector that was last set as the temporary nearest neighbor as the approximate nearest neighbor vector. For example, if there are no unsearched vectors within the search range 610, the search unit 113 outputs the vector that was set as the second temporary nearest neighbor 611 as the approximate nearest neighbor vector.

[0277] Based on the above principle of approximate nearest neighbor vector search, the search unit 113 can use the pre-calculated distance and orientation between vectors to narrow the search range of vectors and efficiently search for approximate nearest neighbor vectors.

[0278] Furthermore, the process of searching for the closest lower-level cluster among multiple lower-level clusters belonging to a certain upper-level cluster can also be performed in the same manner as the principle of using the approximate nearest neighbor vector search using distance (or both distance and orientation).

[0279] Next, refer to Figures 14-20 The processing performed in the approximate nearest neighbor search system 1 is described.

[0280] Figure 14 This is a flowchart illustrating an example of a construction process for building a hybrid index structure that includes hierarchical clusters and inter-cluster graphs. Specifically, the construction process involves using dataset 21 to generate hybrid index information 22 that includes hierarchical clusters HC and inter-cluster graphs CG. Processor 11 executes the construction process, for example, based on the fact that dataset 21 is stored in secondary storage device 14.

[0281] First, processor 11 constructs a hierarchical cluster HC using dataset 21 (step S101). Specifically, processor 11 determines the lowest-level cluster to which each of the multiple vectors included in dataset 21 belongs, such that more than one vector that is closer in position belongs to the same lowest-level cluster. In addition, processor 11 determines the hierarchical structure HC of the clusters in such a way that more than one cluster that is closer in reference position belongs to the same upper-level cluster.

[0282] The processor 11 generates cluster type index information 221 corresponding to multiple upper-level clusters based on the constructed hierarchical cluster HC, and stores it in the secondary storage device 14 (step S102). In addition, the processor 11 generates cluster type index information 222-1 corresponding to multiple lower-level clusters based on the constructed hierarchical cluster HC, and stores it in the secondary storage device 14 (step S103).

[0283] Next, the processor 11 constructs an inter-cluster graph CG that connects the nearest lowest-level clusters among the multiple lowest-level clusters (step S104). Based on the constructed inter-cluster graph CG, the processor 11 generates graph index information 222-2 corresponding to the multiple lowest-level clusters and stores it in the secondary storage device 14 (step S105). Then, the processor 11 ends the construction process.

[0284] Through the above construction process, the processor 11 can use the dataset 21 to construct a hybrid index structure 22 that includes hierarchical clusters HC and inter-cluster graphs CG. In addition, the processor 11 can generate index information 221 corresponding to the upper-level clusters and index information 222 corresponding to the lowest-level clusters based on the constructed hybrid index structure 22.

[0285] Figure 15This is a flowchart illustrating an example of the process for approximate nearest neighbor cluster (search start cluster) search processing. Approximate nearest neighbor cluster search processing is the process of determining the approximate nearest neighbor cluster (search start cluster) as the starting point for approximate nearest neighbor vector search processing. For example, processor 11 performs approximate nearest neighbor vector search processing based on the case where a query vector (query) is received from external device 2. Here, the case where the search object for determining the approximate nearest neighbor cluster is the hierarchical cluster HC is illustrated.

[0286] First, the processor 11 sets the topmost cluster of the hierarchical cluster HC as the object cluster (step S201). The processor 11 obtains the cluster type index information 221 of the object cluster (step S202). The cluster type index information 221 of the object cluster is read from the secondary storage device 14 into the main memory 12, for example.

[0287] Processor 11 uses the acquired cluster index information 221 to determine the closest lower-level cluster to the query among one or more lower-level clusters belonging to the object cluster (step S203). Specifically, processor 11 calculates, for example, the distance between the reference position of the object cluster and the query. Processor 11 uses the calculated distance and the relative position of each lower-level cluster relative to the object cluster to determine the closest lower-level cluster to the query.

[0288] Next, the processor 11 determines whether the determined lower-level cluster is the lowest-level cluster (step S204).

[0289] If the determined lower-level cluster is not the lowest-level cluster (No in step S204), the processor 11 sets the lower-level cluster as the new object cluster (step S205) and returns to step S202. That is, the process of determining the closest lower-level cluster to the query among one or more lower-level clusters that belong to the new object cluster is performed.

[0290] If the determined lower-level cluster is the lowest-level cluster ("Yes" in step S204), the processor 11 sets the lowest-level cluster as the approximate nearest neighbor cluster (step S206) and ends the approximate nearest neighbor cluster search process.

[0291] Through the above approximate nearest neighbor cluster search process, the processor 11 is able to determine the queried approximate nearest neighbor cluster in the hierarchical cluster HC. The approximate nearest neighbor cluster is used as the starting point for the approximate nearest neighbor vector search process. In addition, the search object used to determine the approximate nearest neighbor cluster may not be the hierarchical cluster HC, but multiple clusters with a graph structure (e.g., the lowest level cluster with a graph structure).

[0292] Figure 16This is a flowchart illustrating another example of the process for the search of the approximate nearest neighbor cluster (search start cluster). Here, the case where the search objects for determining the approximate nearest neighbor cluster are multiple clusters with a graph structure is illustrated. Multiple clusters with a graph structure are equivalent to multiple lowest-level clusters within a hierarchical cluster HC. That is, multiple clusters with a graph structure are the structure remaining after removing all higher-level clusters from the hierarchical cluster HC.

[0293] First, processor 11 determines the cluster (object cluster) to be set as the temporary starting point for the approximate nearest neighbor cluster search (step S251). The cluster set as the temporary starting point can be any cluster, for example, it can be a fixed specific cluster or a randomly selected cluster. Processor 11 obtains the neighbor list 222C of the object cluster (i.e., graph index information 222-2) (step S252). The neighbor list 222C of the object cluster is read from secondary storage device 14 to main memory 12, for example. The neighbor list 222C includes, for example, information indicating one or more clusters (neighboring clusters) adjacent to the object cluster and the relative positions of each of the one or more neighboring clusters relative to the object cluster.

[0294] Processor 11 uses the acquired neighbor list 222C to determine the cluster closest to the query among the object cluster and one or more neighboring clusters (step S253). Specifically, processor 11 calculates the distance between the reference position of the object cluster and the query. Processor 11 uses, for example, the calculated distance and the relative positions of each neighboring cluster with respect to the object cluster to determine the cluster closest to the query.

[0295] Processor 11 determines whether the cluster closest to the query is an object cluster (step S254). That is, processor 11 determines whether the cluster closest to the query is an object cluster or a neighboring cluster of the object cluster.

[0296] If the determined cluster is not the object cluster (No in step S254), the processor 11 sets the determined cluster as the new object cluster (step S255) and returns to step S252. That is, the process of determining the cluster closest to the query from the new object cluster and one or more adjacent clusters of the new object cluster is performed.

[0297] If the determined cluster is an object cluster ("Yes" in step S254), the processor 11 sets the object cluster as an approximate nearest neighbor cluster (step S256) and ends the approximate nearest neighbor cluster search process.

[0298] Through the above approximate nearest neighbor cluster search process, the processor 11 is able to determine the approximate nearest neighbor cluster to be queried among multiple clusters with a graph structure.

[0299] Figure 17This is a flowchart illustrating an example of the approximate nearest neighbor vector search process performed by processor 11. The approximate nearest neighbor vector search process is the process of determining the approximate nearest neighbor vector for a query. Processor 11 performs the approximate nearest neighbor vector search process, for example, based on the determination of an approximate nearest neighbor cluster. Here, for example, by referring to the above... Figure 15 or Figure 16 The approximate nearest neighbor cluster search process determines the approximate nearest neighbor cluster. In this case, the processor 11 starts with the approximate nearest neighbor cluster and performs an approximate nearest neighbor vector search process.

[0300] First, processor 11 determines the vector closest to the query (first candidate vector) among the vectors belonging to the approximate nearest neighbor cluster (step S301). Processor 11 sets the first candidate vector as the temporary nearest neighbor vector (step S302). Then, processor 11 sets the distance between the first candidate vector and the query as the temporary nearest neighbor distance (step S303).

[0301] Next, processor 11 obtains a neighbor list 222C of the approximately nearest neighbor cluster (step S304). Using the obtained neighbor list 222C, processor 11 determines unsearched neighbor clusters (search target clusters) that are closer to the approximately nearest neighbor cluster (step S305). Specifically, processor 11 determines the unsearched neighbor clusters that are closer to the approximately nearest neighbor cluster from among the one or more neighbor clusters shown in neighbor list 222C as search target clusters. A neighbor cluster that is closer to the approximately nearest neighbor cluster is a neighbor cluster whose distance and orientation information are closer to the approximately nearest neighbor cluster.

[0302] Processor 11 determines the vector closest to the query (second candidate vector) among the vectors belonging to the search object cluster (step S306). Then, processor 11 determines whether the distance between the second candidate vector and the query is shorter than the temporary nearest neighbor distance (step S307).

[0303] If the distance between the second candidate vector and the query is shorter than the temporary nearest neighbor distance ("Yes" in step S307), the processor 11 sets the second candidate vector as the new temporary nearest neighbor vector (step S308). The processor 11 sets the distance between the second candidate vector and the query as the new temporary nearest neighbor distance (step S309). Then, the processor 11 sets the search object cluster as the new approximate nearest neighbor cluster (step S310) and returns to step S304. That is, the process for searching for approximate nearest neighbor vectors is performed using the neighbor list 222C of the new approximate nearest neighbor cluster.

[0304] If the distance between the second candidate vector and the query is greater than or equal to the temporary nearest neighbor distance (No in step S307), the processor 11 determines whether the neighbor list 222C of the approximate nearest neighbor cluster contains an unsearched neighbor cluster (step S311).

[0305] If the neighbor list 222C of the approximate nearest neighbor cluster contains an unsearched neighbor cluster ("Yes" in step S311), the processor 11 returns to step S305. That is, the unsearched neighbor cluster that is closer to the approximate nearest neighbor cluster is set as the new search target cluster, and processing for searching for the approximate nearest neighbor vector is performed.

[0306] If the neighbor list 222C of the approximate nearest neighbor cluster does not contain any unsearched neighbor clusters (No in step S311), that is, if the processing for searching for approximate nearest neighbor vectors has been performed on all neighbor clusters shown in the neighbor list 222C, the processor 11 outputs the temporary nearest neighbor vector as the approximate nearest neighbor vector (step S312) and ends the approximate nearest neighbor vector search processing.

[0307] Through the above approximate nearest neighbor vector search process, the processor 11 can determine the queried approximate nearest neighbor vector, starting from the approximate nearest neighbor cluster determined in the approximate nearest neighbor cluster search process.

[0308] Figure 18 This is a flowchart illustrating an example of the first index update process performed based on vectors appended to dataset 21 in the vector database. The first index update process updates the hybrid index information 22 based on vectors appended to dataset 21. Processor 11 performs the first index update process, for example, based on the case that vectors have been appended to dataset 21. The appended vectors are referred to as appended object vectors.

[0309] First, the processor 11 determines the lowest-level cluster (the first cluster) that is closest to the appended object vector (step S401). The first cluster is, for example, the lowest-level cluster among all the lowest-level clusters whose reference position is closest to the appended object vector.

[0310] Processor 11 determines whether the value obtained by adding one to the number of vectors belonging to the first cluster exceeds the first upper limit (step S402). That is, processor 11 determines whether the number of vectors belonging to the first cluster exceeds the first upper limit if a new object vector is registered and added to the first cluster (i.e., whether an overflow occurs). The first upper limit is the upper limit of the number of vectors that can belong to one lowest-level cluster.

[0311] If the value obtained by adding one to the number of vectors belonging to the first cluster is below the first upper limit (No in step S402), the processor 11 registers the appended object vector to the first cluster (step S403) and ends the first index update process. Specifically, the processor 11 registers the information corresponding to the appended object vector (e.g., vector ID and relative position information relative to the reference position of the first cluster) in the vector list 222B of the first cluster.

[0312] If the value obtained by adding one to the number of vectors belonging to the first cluster exceeds the first upper limit ("Yes" in step S402), the processor 11 creates a new lowest-level cluster (second cluster) (step S404). Specifically, the processor 11 registers the entry of the second cluster in the index information 222 of the lowest-level cluster. The entry of the second cluster includes, for example, the cluster ID of the second cluster and the reference position (absolute position information).

[0313] Processor 11 registers the appended object vector in the second cluster (step S405). Processor 11 registers the second cluster as a lower-level cluster in the upper-level cluster closest to the second cluster (step S406). Specifically, processor 11 registers the information corresponding to the second cluster (e.g., cluster ID and relative position information relative to the reference position of the upper-level cluster) in the lower-level cluster list 221B of the upper-level cluster.

[0314] Processor 11 registers the first cluster in the adjacent list 222C of the second cluster (second adjacent list 222C) (step S407). Specifically, processor 11 registers the information corresponding to the first cluster (e.g., cluster ID and relative position information relative to the reference position of the second cluster) in the second adjacent list 222C.

[0315] Processor 11 calculates the distances between each of the M clusters shown in the first cluster's neighbor list 222C (first neighbor list 222C) and the second cluster (step S408). That is, processor 11 calculates the distances between the reference positions of the M clusters and the reference positions of the second cluster. M is an integer greater than or equal to 1. Processor 11 determines the unprocessed cluster (third cluster) among the M clusters that is closer to the second cluster (step S409). Then, processor 11 determines whether the number of clusters registered in the third cluster's neighbor list 222C (third neighbor list 222C) is less than a second upper limit value (step S410). The second upper limit value is the upper limit on the number of neighboring clusters that can be registered for one lowest-level cluster.

[0316] If the number of clusters registered in the third adjacent list 222C is less than the second upper limit ("Yes" in step S410), the processor 11 proceeds to step S412.

[0317] If the number of clusters registered in the third adjacent list 222C is greater than or equal to the second upper limit (No in step S410), the processor 11 will remove the cluster with the longest distance that will not be isolated even if the edge connected to the third cluster (i.e., inter-cluster edge) is deleted from the third adjacent list 222C (step S411) and proceed to step S412.

[0318] Next, processor 11 registers the third cluster in the second adjacent list 222C (step S412). Processor 11 registers the second cluster in the third adjacent list 222C (step S413). Then, processor 11 determines whether the number of clusters registered in the second adjacent list 222C is less than the second upper limit value (step S414).

[0319] If the number of clusters registered in the second adjacent list 222C is less than the second upper limit (Yes in step S414), the processor 11 determines whether the M clusters include unprocessed clusters (step S415). If the M clusters include unprocessed clusters (Yes in step S415), the processor 11 returns to step S409. That is, processing is performed to register the unprocessed clusters and the second cluster in each other's adjacent lists 222C.

[0320] If the number of clusters registered in the second neighbor list 222C is greater than or equal to the second upper limit (No in step S414), and if the M clusters do not include unprocessed clusters (No in step S415), the processor 11 registers the second cluster in the first neighbor list 222C (step S416). Then, the processor 11 moves the vectors belonging to one or more neighboring clusters shown in the second neighbor list 222C that are closer to the second cluster than their respective neighboring clusters to the second cluster (step S417), and ends the first index update process. Specifically, the processor 11 deletes the information corresponding to the moved vector from the vector list 222B of the neighboring cluster to which the vector belongs, and registers it in the vector list 222B of the second cluster.

[0321] Through the first index update process described above, processor 11 can update the hybrid index information 22 based on the appended object vector.

[0322] Figure 19 This is a flowchart illustrating an example of the intra-cluster nearest neighbor vector search process performed when the cluster being processed during append processing is determined to be the starting cluster for the search. The intra-cluster nearest neighbor vector search process involves searching for the vector closest to the query from the cluster that has been determined as the starting cluster.

[0323] Processor 11 determines whether the search start cluster determined based on the query is a cluster (second cluster) in the process of performing append processing (cluster append processing) (step S421). The second cluster is equivalent to the lowest level cluster newly set at a position close to a certain lowest level cluster (first cluster).

[0324] If the search start cluster is a cluster (the second cluster) in the process of performing cluster appending processing ("Yes" in step S421), the processor 11 adds all vectors belonging to the first cluster as search objects (step S422).

[0325] Processor 11 finds the vector closest to the query from all vectors belonging to the second cluster and all vectors belonging to the first cluster (step S423).

[0326] On the other hand, if the search start cluster is not a cluster in the process of performing cluster append processing (No in step S421), the processor 11 searches for the vector closest to the query from all vectors belonging to the search start cluster (step S424).

[0327] Through the above processing, even if the second cluster is determined as the search starting cluster before the vector has moved from the first cluster to the second cluster, the vector closest to the query can still be found correctly.

[0328] Figure 20 This is a flowchart illustrating an example of the second index update process performed when a vector is deleted from dataset 21 of the vector database. The second index update process updates the hybrid index information 22 based on the vector deleted from dataset 21. For example, processor 11 performs the second index update process based on a request to delete a vector from dataset 21. The vector to be deleted is referred to as the deleted object vector.

[0329] First, processor 11 determines the lowest-level cluster (cluster 4) to which the object vector to be deleted belongs (step S501). Then, processor 11 deletes the object vector from cluster 4 (step S502). Specifically, processor 11 deletes the information corresponding to the object vector to be deleted (e.g., vector ID and relative position information relative to the reference position of cluster 4) from the vector list 222B of cluster 4. Then, processor 11 determines whether the number of vectors belonging to cluster 4 is 0 (step S503).

[0330] If the number of vectors belonging to the 4th cluster is 1 or more (No in step S503), the processor 11 ends the second index update process.

[0331] If the number of vectors belonging to cluster 4 becomes 0 ("Yes" in step S503), processor 11 determines the unprocessed cluster (cluster 5) among the N clusters shown in the neighbor list 222C (4th neighbor list 222C) of cluster 4 (step S504). N is an integer greater than or equal to 1. Processor 11 removes cluster 4 from the neighbor list 222C (5th neighbor list 222C) of cluster 5 (step S505). Processor 11 determines the clusters from the N clusters excluding cluster 5 and the clusters already registered in the 5th neighbor list 222C (step S506). Processor 11 registers the determined clusters in the 5th neighbor list 222C (step S507). Then, processor 11 determines whether the number of clusters registered in the 5th neighbor list 222C exceeds the second upper limit value (step S508).

[0332] If the number of clusters registered in the 5th adjacent list 222C exceeds the second upper limit ("Yes" in step S508), the processor 11 removes the cluster with the longest distance that will not be isolated even if the edge connected to the 5th cluster is deleted from the 5th adjacent list 222C (step S509) and returns to step S508. That is, it determines whether the number of clusters registered in the 5th adjacent list 222C has fallen below the second upper limit due to the deletion of the cluster in step S509.

[0333] If the number of clusters registered in the 5th adjacent list 222C is below the 2nd upper limit (No in step S508), the processor 11 determines whether the N clusters include unprocessed clusters (step S510).

[0334] If N clusters include unprocessed clusters ("Yes" in step S510), processor 11 returns to step S504. That is, it performs the process of deleting the 4th cluster from the neighbor list 222C of the unprocessed clusters and registering the clusters in the neighbor list 222C of the 4th cluster that were not registered in the neighbor list 222C of the unprocessed clusters into the neighbor list 222C of the unprocessed clusters.

[0335] If the N clusters do not include any unprocessed clusters (No in step S510), the processor 11 removes the fourth cluster, which is registered as a lower-level cluster, from the parent cluster of the fourth cluster (step S511) and ends the second index update process. Specifically, the processor 11 removes the information corresponding to the fourth cluster (e.g., cluster ID and relative position information relative to the reference position of the parent cluster) from the lower-level cluster list 221B of the parent cluster.

[0336] Through the second index update process described above, processor 11 can update the hybrid index information 22 based on the deleted object vector.

[0337] Next, refer to Figure 21 , Figure 22 as well as Figure 23 Several variations of the search process for the approximate nearest neighbor cluster (search start cluster) are explained.

[0338] Figure 21 This is a diagram representing the first variation of the search process for the approximate nearest neighbor cluster (search start cluster). Figure 21 This corresponds to the example of determining the starting cluster for the search without placing the structure used to determine the starting cluster at the top of the inter-cluster graph.

[0339] Figure 21 The lower part represents the positional relationship between multiple clusters (reference positions) and multiple vectors in the vector space. This positional relationship between clusters (reference positions) and vectors is based on the data structure that, for each cluster, vectors close to the reference position of that cluster belong to that cluster (Spatial Separated Clustering).

[0340] in addition, Figure 21 The upper part represents the inter-cluster diagram. Furthermore, in Figure 21 In the diagram, the relationship between the reference positions of each cluster on the inter-cluster graph and the reference positions of each cluster in the vector space is represented by dashed arrows.

[0341] The search begins with any cluster in the inter-cluster graph. This starting cluster is called the entry point. Figure 21 In this approach, the structure used to determine the starting cluster is not placed at the top of the inter-cluster graph; instead, long-distance edges are added to the inter-cluster graph. In the starting cluster search process, the distance from the cluster's reference position to the query is calculated while traversing the inter-cluster graph for each cluster, thereby determining the starting cluster. Furthermore, in... Figure 21 In Chinese, queries are represented by an asterisk.

[0342] Figure 22 This is a diagram representing the second variation of the search process used for the approximate nearest neighbor cluster (search start cluster).

[0343] exist Figure 22 In the diagram, a tree structure is placed at the top of the inter-cluster graph to determine the starting cluster for the search. The hierarchical clusters described earlier can also be represented using such a tree structure. Furthermore, in... Figure 22 In the diagram, the relationship between higher-level clusters and their subordinate clusters is indicated by bold arrows with dotted lines. The cluster search process at the start of the search can be performed following the same steps as when using hierarchical clusters.

[0344] Figure 23This is a diagram representing the third variation of the search process used for the approximate nearest neighbor cluster (search start cluster).

[0345] exist Figure 23 In this model, a hierarchical graph is set above the inter-cluster graph. Within the hierarchical graph, the higher the level, the coarser the granularity. During the initial cluster search process, the search begins from the highest-level graph, repeatedly performing the process of searching for clusters close to the query and moving down to lower-level graphs, thereby determining the starting cluster for the search.

[0346] As explained above, according to this embodiment, instead of using a graph that connects vectors, an inter-cluster graph that connects clusters to which each has multiple vectors belongs is used.

[0347] Since the number of clusters is far less than the number of vectors, it is possible to reduce the number of adjacency lists used to represent edges within a graph, thus minimizing the increase in the size of the index information, even when building a large-scale vector database with more than a billion nodes.

[0348] Furthermore, when adding a new vector to the vector database, the new vector only needs to be registered in one cluster, without rewriting the adjacent list (edge ​​information). This reduces the frequency of graph updates, thereby reducing the amount of writing to the secondary storage device 14. As a result, the lifespan of the secondary storage device 14, similar to an SSD, can be extended, and the time required to register a vector can be shortened.

[0349] Furthermore, in the approximate nearest neighbor vector search process, not only is the process of searching for the nearest neighbor vector within the search start cluster from the vectors belonging to the search start cluster with the reference position closest to the query performed, but also the process of selecting one or more search object clusters close to the search start cluster by traversing along the inter-cluster graph CG, and then searching for the nearest neighbor vector within each search start cluster from the vectors belonging to each of the one or more search object clusters. Therefore, not only are the vectors belonging to the search start cluster searchable, but the vectors of each adjacent cluster belonging to the search start cluster are also searchable. As a result, while significantly reducing the number of edge information rewrites, it is possible to achieve search accuracy comparable to that of approximate nearest neighbor search algorithms using inter-vector graphs that connect vectors.

[0350] Therefore, it can reduce the amount of data that needs to be rewritten as the graph index is updated, and can perform approximate nearest neighbor search with sufficient search accuracy.

[0351] Furthermore, the process of determining the search starting cluster can be performed using hierarchical clusters with a structure in which each higher-level cluster has a lower-level cluster belonging to a reference position close to that higher-level cluster. In this case, by using pre-calculated relative position information, the lower-level cluster belonging to the object cluster with the closest reference position to the query is searched, thereby quickly determining the search starting cluster.

[0352] Furthermore, each of the various functions described in this embodiment can be implemented by a circuit (processing circuit). Examples of processing circuits include a programmed processor, such as a central processing unit (CPU). This processor executes the described functions by executing a computer program (command group) stored in memory. The processor can be a microprocessor that includes electrical circuitry. Examples of processing circuits also include digital signal processors (DSPs), application-specific integrated circuits (ASICs), microcontrollers, controllers, and other electrical circuit components. Other components besides the CPU described in this embodiment can also be implemented by processing circuits.

[0353] Several embodiments of the present invention have been described, but these embodiments are provided as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a wide variety of other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included within the scope and spirit of the invention, as well as within the scope of the invention as described in the claims and its equivalents.

Claims

1. An approximate nearest neighbor search method, which is used for a vector database, wherein the vector database is configured to store multiple vectors, each of the multiple vectors containing multiple feature values ​​corresponding to multiple dimensions, and the approximate nearest neighbor search method includes: The system manages cluster index information for defining multiple clusters, each of which has a reference position. For each of the multiple clusters, a group of vectors whose reference positions are close to that cluster belongs to that cluster. The management system defines graph index information for inter-cluster graphs, which include multiple nodes and multiple edges. The multiple nodes correspond to multiple clusters, and the multiple edges are used to connect nodes corresponding to clusters that have mutually close reference positions. Receive a query vector, which is for each of the multiple dimensions and includes feature values; Perform the first process of determining the cluster with the reference position closest to the query vector among the plurality of clusters as the search start cluster; From the vectors belonging to the search start cluster, search for the vector that is closest to the query vector and use it as the nearest neighbor vector within the search start cluster; While moving along the inter-cluster graph, select one or more search object clusters that are close to the search start cluster, and search from the vectors of each search object cluster belonging to the one or more search object clusters the vector that is closest to the query vector as the nearest neighbor vector within each search object cluster; and The nearest neighbor vector found from the search start cluster and the vector that is closest to the query vector from the nearest neighbor vectors found from the one or more search object clusters are output as the approximate nearest neighbor vector of the query vector.

2. The approximate nearest neighbor search method according to claim 1, The cluster index information, for each of the plurality of clusters, includes a list of vectors to which the identifiers of the vectors belonging to that cluster belong. The graph index information, for each of the plurality of clusters, includes an adjacency list representing the identifiers of adjacent clusters connected to the cluster by edges. The approximate nearest neighbor search method further includes: When appending a new vector to the vector database Determine the first cluster among the plurality of clusters that has the reference position closest to the new vector; The new vector is registered in the first vector list of the first cluster, without rewriting the adjacent lists corresponding to the adjacent clusters of the first cluster.

3. The approximate nearest neighbor search method according to claim 2 further includes: When appending the new vector to the vector database The system detects cases where the total number of vectors already belonging to the first cluster plus one exceeds the upper limit. Based on the detection that the total value exceeds the upper limit, a new cluster is created with a reference position close to the reference position of the first cluster; Perform the second process of appending the new cluster as the second cluster to the inter-cluster graph. The second process includes: The process of registering the identifier of the first cluster and the identifier of the third cluster, which is a neighboring cluster of the first cluster, in the second neighboring list corresponding to the second cluster; The process of registering the identifier of the second cluster in the first adjacent list corresponding to the first cluster; The process of determining, from all the vectors already belonging to the first cluster and the new vectors, one or more vectors whose distance from the reference position of the second cluster is shorter than the distance from the reference position of the first cluster; The process of registering the identifiers of each of the determined one or more vectors in the second list of vectors corresponding to the second cluster; and The process of deleting the identifiers of each vector in the first vector list corresponding to the first cluster from the first vector list.

4. The approximate nearest neighbor search method according to claim 3 further includes: The process of performing the second process is to detect the case where the second cluster is determined to be the search start cluster. and Based on the detection that the second cluster was determined as the search start cluster during the execution of the second process, Add all vectors registered in the first cluster to the first vector list as search objects; From all vectors registered in the second vector list of the second cluster and all vectors registered in the first vector list of the first cluster, the vector closest to the query vector is searched as the nearest neighbor vector in the search starting cluster.

5. The approximate nearest neighbor search method according to claim 3 further includes: When deleting a vector from the vector database Determine the 4th cluster to which the vector belongs; Delete the vector from the fourth vector list of the fourth cluster; Based on the fact that the number of vectors belonging to the fourth cluster is detected to be zero due to the deletion of one of the vectors, Identify one or more fifth clusters that are registered as adjacent clusters of the fourth cluster in the fourth neighbor list of the fourth cluster; For each of the more than one fifth cluster, the following processes are performed: deleting the identifier of the fourth cluster from the fifth neighbor list of the fifth cluster, determining the cluster that is registered in the fourth neighbor list of the fourth cluster but not registered in the fifth neighbor list of the fifth cluster, and registering the identifier of the determined cluster in the fifth neighbor list of the fifth cluster.

6. The approximate nearest neighbor search method according to claim 1, The multiple clusters are managed using a hierarchical cluster structure that includes a lowest level and multiple higher levels. The lowest level comprises multiple lowest level clusters, each with a reference position. For each of the multiple lowest level clusters, the group of vectors closest to the reference position of that lowest level cluster belongs to that lowest level cluster. The plurality of clusters correspond to the plurality of lowest-level clusters, respectively. The highest-level layer among the plurality of upper-level layers includes an upper-level layer cluster as the highest-level layer cluster. The upper-level layer cluster has a reference position, and a plurality of lower-level layer clusters, each having a reference position close to the reference position of the upper-level layer cluster, belong to the upper-level layer cluster. Each of the multiple upper-level layers, excluding the topmost layer, comprises multiple upper-level layer clusters. Each of these upper-level layer clusters has a reference position. For each of these upper-level layer clusters, multiple lower-level layer clusters with reference positions close to the reference position of that upper-level layer cluster belong to that upper-level layer cluster. The cluster index information for each of the multiple upper-level clusters includes (1) a list of lower-level clusters representing the identifiers of the respective lower-level clusters belonging to the upper-level cluster; and (2) first relative position information between the reference position of the upper-level cluster and the reference position of the respective lower-level clusters belonging to the upper-level cluster. (3) The second relative position information between the reference position of the upper-level cluster and the reference position of each of the same-level clusters, wherein each of the same-level clusters is another upper-level cluster other than the upper-level cluster contained in the same layer as the layer that includes the upper-level cluster. Performing the first process includes: The topmost cluster is set as the object cluster. Using the first relative position information of the object cluster and the second relative position information corresponding to the lower clusters belonging to the object cluster, the lower clusters belonging to the object cluster are searched for the lower clusters with the reference position closest to the query vector. The search process includes the following steps: setting the found lower-level clusters as new object clusters; using the first relative position information corresponding to the new object cluster and the second relative position information corresponding to each of the lower-level clusters belonging to the new object cluster, searching from the lower-level clusters belonging to the new object cluster for a lower-level cluster with a reference position closest to the query vector; and The search process is repeated until one of the plurality of lowest-level clusters is found as the lowest-level cluster with the reference position closest to the query vector.

7. The approximate nearest neighbor search method according to claim 6, The first relative position information, for each of the lower-level clusters belonging to the upper-level cluster, includes distance information representing the distance between the reference position of the upper-level cluster and the reference position of the lower-level cluster. The second relative position information, for each of the same-layer clusters, includes distance information representing the distance between the reference position of the upper-layer cluster and the reference position of the same-layer cluster.

8. The approximate nearest neighbor search method according to claim 7, The first relative position information, for each of the lower-level clusters belonging to the upper-level cluster, further includes azimuth information representing the orientation from the reference position of the upper-level cluster to the reference position of the lower-level cluster. The second relative position information, for each of the same-layer clusters, further includes orientation information representing the orientation from the reference position of the upper-layer cluster to the reference position of the same-layer cluster.

9. The approximate nearest neighbor search method according to claim 1, The graphic index information includes: For each of the plurality of clusters, there is an adjacent list representing the identifiers of the adjacent clusters connected to the cluster by edges, and For each of the plurality of clusters, a first azimuth information is provided, representing the azimuth from the reference position of the cluster to the reference position of the neighboring cluster. The approximate nearest neighbor search method further includes: When performing a search on one or more search object clusters that are close to the search start cluster, Calculate the first position of the query vector from the reference position of the search starting cluster; and Using the azimuth information corresponding to each of the neighboring clusters of the search starting cluster, the neighboring cluster with the azimuth most similar to the first azimuth is selected as one of the more than one search target clusters, prioritizing it over other neighboring clusters of the search starting cluster.

10. The approximate nearest neighbor search method according to claim 1, The graph index information, for each of the plurality of clusters, includes an adjacency list representing the identifiers of adjacent clusters connected to the cluster by edges. The multiple vectors, the cluster index information, and the graph index information are stored in a secondary storage device. The adjacent list is stored in a second storage area of ​​a secondary storage device, which is different from the first storage area in the secondary storage device that stores the plurality of vectors.

11. An approximate nearest neighbor search system, comprising: Main memory; A secondary storage device is configured as a vector database, wherein the vector database stores multiple vectors, each of which contains multiple feature values ​​corresponding to multiple dimensions. as well as The processor, which has access to the main memory and the secondary storage device, The processor is configured such that, The system manages cluster index information for defining multiple clusters, each of which has a reference position. For each of the multiple clusters, a group of vectors whose reference positions are close to that cluster belongs to that cluster. The management system defines graph index information for inter-cluster graphs, which include multiple nodes and multiple edges. The multiple nodes correspond to multiple clusters, and the multiple edges are used to connect nodes corresponding to clusters that have mutually close reference positions. Receive a query vector, which is for each of the multiple dimensions and includes feature values; Perform the first process of determining the cluster with the reference position closest to the query vector among the plurality of clusters as the search start cluster; From the vectors belonging to the search start cluster, search for the vector that is closest to the query vector and use it as the nearest neighbor vector within the search start cluster; While moving along the inter-cluster graph, select one or more search object clusters that are close to the search start cluster, and search from the vectors of each search object cluster belonging to the one or more search object clusters the vector that is closest to the query vector as the nearest neighbor vector within each search object cluster; and The nearest neighbor vector found from the search start cluster and the vector that is closest to the query vector from the nearest neighbor vectors found from the one or more search object clusters are output as the approximate nearest neighbor vector of the query vector.

12. The approximate nearest neighbor search system according to claim 11, The cluster index information, for each of the plurality of clusters, includes a list of vectors to which the identifiers of the vectors belonging to that cluster belong. The graph index information, for each of the plurality of clusters, includes an adjacency list representing the identifiers of adjacent clusters connected to the cluster by edges. The processor is further configured to, When appending a new vector to the vector database Determine the first cluster among the plurality of clusters that has the reference position closest to the new vector; The new vector is registered in the first vector list of the first cluster, without rewriting the adjacent lists corresponding to the adjacent clusters of the first cluster.

13. The approximate nearest neighbor search system according to claim 12, The processor is further configured to, When appending the new vector to the vector database If the total value obtained by adding one to the number of vectors that already belong to the first cluster exceeds the upper limit, a new cluster is created with a reference position that is close to the reference position of the first cluster. Perform the second process of appending the new cluster as the second cluster to the inter-cluster graph. The second process includes: The process of registering the identifier of the first cluster and the identifier of the third cluster, which is a neighboring cluster of the first cluster, in the second neighboring list corresponding to the second cluster; The process of registering the identifier of the second cluster in the first adjacent list corresponding to the first cluster; The process of determining, from all the vectors already belonging to the first cluster and the new vectors, one or more vectors whose distance from the reference position of the second cluster is shorter than the distance from the reference position of the first cluster; The process of registering the identifiers of each of the determined one or more vectors in the second list of vectors corresponding to the second cluster; and The process of deleting the identifiers of each vector in the first vector list corresponding to the first cluster from the first vector list.

14. The approximate nearest neighbor search system according to claim 13, The processor is further configured to, Based on the detection that the second cluster was determined as the search start cluster during the execution of the second process, Add all vectors registered in the first cluster to the first vector list as search objects; From all vectors registered in the second vector list of the second cluster and all vectors registered in the first vector list of the first cluster, the vector closest to the query vector is searched as the nearest neighbor vector in the search starting cluster.

15. The approximate nearest neighbor search system according to claim 13, The processor is further configured to, When deleting a vector from the vector database Determine the 4th cluster to which the vector belongs; Delete the vector from the fourth vector list of the fourth cluster; Based on the fact that the number of vectors belonging to the fourth cluster is detected to be zero due to the deletion of one of the vectors, Identify one or more fifth clusters that are registered as adjacent clusters of the fourth cluster in the fourth neighbor list of the fourth cluster; For each of the more than one fifth cluster, the following processes are performed: deleting the identifier of the fourth cluster from the fifth neighbor list of the fifth cluster, determining the cluster that is registered in the fourth neighbor list of the fourth cluster but not registered in the fifth neighbor list of the fifth cluster, and registering the identifier of the determined cluster in the fifth neighbor list of the fifth cluster.

Images