A neighbor graph index updating method for dynamic vector database

Abstract

The present invention belongs to the technical field of database-related fields, and discloses a method for updating a neighbor graph index of a dynamic vector database, the method comprising: obtaining a point to be deleted that needs to be deleted from the current graph index of the dynamic vector database; deleting all incoming edges of the point to be deleted from the graph index; for each point to be deleted p that has an outgoing neighbor, executing: obtaining points within two hops from the current graph index with the point to be deleted p as the starting point to construct p out The candidate set of new inner neighbors, p out For the outer neighbors of the point p to be deleted, connect the points in the candidate set to p out Delete all pending points and all outgoing edges to update the graph index in the dynamic vector database. This method can quickly and accurately find edges that help repair the monotonic search path, has high update efficiency, and maintains high accuracy of the graph index after updating it.

Description

Technical Field

[0001] The present invention belongs to the technical field of database-related technologies, and more specifically, relates to a method for updating a neighbor graph index of a dynamic vector database. Background Art

[0002] Major breakthroughs in large language models (LLMs) have demonstrated unprecedented capabilities in understanding, generating, and reasoning about human language. This capability stems from knowledge vector representations extracted from multimodal data. Once embedded in a vector space, the correlation between data from different modalities can be measured by the distance between their vector representations. Approximate nearest neighbor (ANN) search, which can be used to find knowledge related to a user-provided prompt in a vector library, is a fundamental function of LLMs and a core feature in applications such as data mining, recommendation, and information retrieval.

[0003] Graph-based ANN search methods are the most promising due to their high accuracy and efficiency. Graph-based ANN search methods often build indexes based on a nearest neighbor graph. These methods construct a graph structure by treating each high-dimensional vector as a point in the graph and connecting closely spaced points according to a specific strategy. For each query, the graph index executes a greedy search algorithm from a fixed search starting point to obtain the ANN search results. The greedy search algorithm begins from the starting point and follows a monotonic search path in the graph index toward the query target, sequentially checking the distance between each point on the path and the query target, and selecting the point closest to the query as the ANN search result.

[0004] Most current research focuses on building high-quality indexes for static datasets. However, static ANN search methods are ill-suited for dynamic scenarios with frequent data updates, such as e-commerce and news recommendation. In news recommendation, trending topics and breaking news frequently change, requiring real-time updates to provide the most relevant content. In e-commerce, order data updates rapidly, especially during shopping festivals when a large number of consumers frequently place, modify, or cancel orders. Therefore, dynamic ANN search methods that can adapt to these rapid changes are becoming increasingly important and deserve greater attention. FreshDiskAnn, a state-of-the-art graph-based dynamic ANN search method, improves update performance by modifying the graph structure. However, this method still faces two major challenges: First, the deletion operations performed when updating the graph index by this method destroy the monotonic search paths in the graph index, which are crucial for ANN search. Furthermore, the path repair process is ineffective in repairing these monotonic search paths, resulting in a decrease in the accuracy of the updated graph index. This makes it difficult for greedy search algorithms to approach the query target, making it difficult to meet the requirements of high-precision applications. Second, this method suffers from low update efficiency, as the path repair process is time-consuming, resulting in low update efficiency and difficulty meeting the requirements of real-time update scenarios. Summary of the Invention

[0005] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a method for updating the neighbor graph index of a dynamic vector database. Its purpose is to face a dynamically changing database and quickly update the graph index while ensuring the correctness of the graph index so as to quickly adapt to the dynamic changes of the database.

[0006] To achieve the above object, the present invention provides a method for updating a neighbor graph index of a dynamic vector database, which comprises:

[0007] Obtain the points to be deleted in the current graph index of the dynamic vector database;

[0008] Deleting all incoming edges of the points to be deleted from the graph index;

[0009] For each point p to be deleted that has an external neighbor, execute: Get the points within two hops of the point p to be deleted in the current graph index to construct p out The candidate set of new inner neighbors, p out For the outer neighbors of the point p to be deleted, connect the points in the candidate set to p out ;

[0010] Delete all the points to be deleted and all their outgoing edges to update the graph index of the dynamic vector database;

[0011] Among them, the outer neighbors of any point are the points connected to the outgoing edges of the point, and the inner neighbors of any point are the points connected to the incoming edges of the point.

[0012] Optionally, deleting all incoming edges of the points to be deleted from the graph index includes:

[0013] Scan each point in the current graph index in turn and delete the outgoing edge from the currently scanned point to the point to be deleted.

[0014] Optionally, obtain the points within two hops from the point to be deleted p in the current graph index to construct p out The candidate set of new inner neighbors includes:

[0015] Initialize the candidate set to be empty;

[0016] Scan the outer neighbors p of the point to be deleted according to the given point to be deleted out , and after scanning the current p out After that, scan the current p out The outer neighbor and the current p out After all the outer neighbors of point p are scanned, scan and delete the next outer neighbor p of point p. out; Add the scanned points to the candidate set in sequence according to the scanning order until the storage limit of the candidate set is reached.

[0017] Optionally, obtain the points within two hops from the point to be deleted p in the current graph index to construct p out The candidate set of new inner neighbors includes:

[0018] The outer neighbors of the point to be deleted and the points with the same out The distance is less than d(p,p out ) are added to the candidate set, and points that are outer neighbors of the point to be deleted p and that meet three set conditions are added to the candidate set, wherein the three set conditions include:

[0019] d(c,p out ) <d(p,p out )

[0020] d(c,p)>d(p,p out )

[0021] d 2 (c,p out )+d 2 (p,p out )>d 2 (c,p)

[0022] In the formula, p is the point to be deleted, p out is the outer neighbor of p, c is the point of p in the candidate set, d(c,p out ) represents c and p out The distance in the graph index, d(p,p out ) for p and p out The distance in the graph index, d(c,p) is the distance between c and p in the graph index.

[0023] Optionally, obtain the points within two hops from the point to be deleted p in the current graph index to construct p out The candidate set of new inner neighbors includes:

[0024] Initialize the candidate set to be empty;

[0025] According to the given point p to be deleted and one of its outer neighbors p out , scan the other outer neighbors p of the point to be deleted in turn out ', when scanning to the same out The distance is less than d(p,p out ) out 'After that, scan the current p out 'Outer neighbors until the current p out'After all the outer neighbors are scanned, scan the next outer neighbor p of the point to be deleted out '; Add the points that meet the conditions for joining the candidate set to the candidate set in sequence according to the scanning order until the storage limit of the candidate set is reached.

[0026] Optionally, the points in the candidate set are connected to p out ,include:

[0027] First, the distance p in the current candidate set out The nearest point Move into the result set and delete the candidates in the current candidate set that meet the current point The distance is less than p out Repeat this step until the current candidate set is empty or the number of nodes in the result set reaches its upper limit;

[0028] Add a point from the result set to p out The outgoing edge.

[0029] Optionally, the method further includes:

[0030] For the point v in the result set whose outgoing edge is greater than the set upper limit, the edge reduction operation is performed. The edge reduction operation includes: removing the point closest to v in the current outer neighbor set of v. Move into the candidate set of v's outer neighbors and delete the ones in the current set of outer neighbors that meet the requirements of the current point. The distance from is less than the distance from v and Repeat this step until the current outer neighbor set is empty or the number of nodes in its outer neighbor candidate set reaches the upper limit of the number of outgoing edges, and finally use its outer neighbor candidate set as the outer neighbor set of point v.

[0031] The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0032] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0033] The present invention also provides a computer program product, comprising a computer program or instructions, which implement the steps of any of the above methods when executed by a processor.

[0034] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0035] 1. The present invention selects a point within two hops of the point to be deleted p as the starting point to construct p out The candidate set of new inner neighbors, through analysis, point p is p out One of the inner neighbors of , so point p is located at p out Nearby points, and the neighbors within two hops of point p are also located at p out points nearby, and with p out The greedy search result for the query is located at p out Nearby, so the neighbors within two hops of point p are highly overlapped with the greedy search results. Therefore, the two-hop neighbors of point p can effectively replace the greedy search results. Therefore, constructing a candidate set of repair paths based on the two-hop neighbors of point p can effectively repair the monotone search path destroyed by deleting point p, and ensure the correctness of the graph index update. Moreover, compared with constructing a candidate set through greedy search, the time consumed by obtaining the two-hop neighbors of point p to construct a candidate point set is also greatly reduced. In summary, the present invention effectively repairs the monotone search path destroyed by deletion by repairing the monotone search path to the outer neighbors of the deleted point, thereby achieving a high accuracy of the graph index after updating the graph index. Moreover, the candidate set acquisition method adopted in the present invention can quickly and accurately find edges that help repair the monotone search path. Compared with other methods, the present invention has high update efficiency.

[0036] 2. The present invention can extend existing graph indexes to support dynamic graph indexing. Compared to other existing solutions that require redeploying indexes, the present invention can extend the functionality of already deployed indexes, thus having the advantage of lower deployment difficulty.

[0037] 3. Furthermore, the present invention has a flexible upper limit parameter, which allows for a trade-off between index accuracy and update efficiency. Different parameter settings can be used for different usage scenarios and requirements, thus enabling the present invention to adapt to more complex real-world scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flowchart of the steps of a method for updating a neighbor graph index of a dynamic vector database in one embodiment of the present invention;

[0039] Figure 2 This is a specific flow chart for constructing a candidate set in one embodiment of the present invention;

[0040] Figure 3 is a specific flow chart of constructing a candidate set in another embodiment of the present invention;

[0041] Figure 4 is a flowchart of the detailed steps of a method for updating a neighbor graph index of a dynamic vector database in one embodiment of the present invention;

[0042] Figure 5 is an example diagram of a selected candidate set in one embodiment of the present invention;

[0043] Figure 6 is an example diagram of screening candidate sets and repairing monotonic search paths in one embodiment of the present invention;

[0044] Figure 7 is an example diagram of an effective repair monotonic search path in one embodiment of the present invention;

[0045] Figure 8 This is an experimental comparison chart of the indexing accuracy between the present invention and FreshDiskAnn;

[0046] Figure 9 This is an experimental comparison chart of the deletion throughput between the present invention and FreshDiskAnn. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0048] Example 1

[0049] like Figure 1 FIG2 is a flowchart of a method for updating a neighbor graph index of a dynamic vector database according to an embodiment of the present invention. The steps are described in detail below.

[0050] Step S1: Obtain the points to be deleted in the current graph index of the dynamic vector database.

[0051] Among them, the vector database is a database specially designed for storing, managing, and querying vector data. Its core functions include storing vector data and providing users with similarity query interfaces and data update interfaces. Its data changes dynamically, which is called a dynamic vector database. The graph index is the core component of the vector database. The graph index is a network structure composed of a series of nodes and edges. Different nodes represent different vector representations in the vector library, and the edge represents the relationship between two nodes. The length of the edge reflects the strength of the relationship between the two vector representations. Through its graph index, you can search the vector library for knowledge related to the prompt word provided by the user, realizing applications such as data mining, recommendation, and information retrieval. When the database changes dynamically, its graph index also needs to be updated. The speed and accuracy of its update directly affect the user experience.

[0052] When dynamic changes in the database involve data deletion, the node representing the data in the current graph index needs to be deleted, and the edges connected to the node will also be deleted, which will destroy the original monotonic search path. Therefore, when updating the graph index, it is necessary to both delete the nodes and repair the paths. Only in this way can the graph index adapt to the dynamic changes of the current database and ensure its accuracy.

[0053] In the present invention, when the database changes dynamically, the deleted data therein can be directly obtained, and the points that need to be deleted in the current graph index can be clearly identified, which are called points to be deleted.

[0054] Step S2: Delete all incoming edges of the points to be deleted from the graph index.

[0055] Specifically, there may be one or more points to be deleted. After the point to be deleted is clearly identified, the incoming edge and outgoing edge of the point to be deleted in the graph index can be identified. The point to be deleted, all its incoming edges and all its outgoing edges need to be deleted. The present invention implements the deletion of incoming edges and outgoing edges through different strategies. It can be understood that the graph index is a directed graph, the incoming edge of a node is the edge from other nodes pointing to the node, and the outgoing edge is the edge from the node pointing to other nodes. For a certain node, the node at the other end of the incoming edge of the node is the inner neighbor of the node, and the node at the other end of the outgoing edge of the node is the outer neighbor of the node.

[0056] This embodiment first deletes all incoming edges of all points to be deleted.

[0057] Specifically, the graph index is usually stored in the form of an edge table. Each node corresponds to an edge table, which records all outgoing edges of the node. The graph index can be updated by modifying the outgoing edges in the storage space.

[0058] In one embodiment, the following method can be used to delete the incoming edge:

[0059] Scan the points to be deleted in turn, obtain all the inner neighbors of the currently scanned point to be deleted, and delete the outgoing edges pointing to the current point to be deleted in its inner neighbor edge table.

[0060] Through the above method, all incoming edges with points to be deleted can be deleted.

[0061] In another embodiment, the following method can be used to delete the incoming edge:

[0062] Scan each point in the current graph index in turn and delete the outgoing edge from the currently scanned point to the point to be deleted.

[0063] The previous method of deleting incoming edges requires multiple modifications to the edge tables of these neighbors when some points are in-neighbors of multiple points to be deleted, which is inefficient. Furthermore, when using multiple threads to perform deletion, this method is likely to cause contention among multiple threads for access to the edge tables of these points, further exacerbating efficiency issues. Furthermore, the previous method requires additional maintenance of the in-edge table for each point in the graph index, increasing index maintenance overhead and implementation complexity.

[0064] In the latter approach to deleting incoming edges, each point in the graph only needs to modify the edge table once at most, and assigning disjoint sets of points to each thread avoids thread conflicts, greatly improving the efficiency of multi-threaded incoming edge deletion. Although this approach may require scanning points that do not require edge table modifications, considering that deleting a point requires modifying the edge tables of multiple points, when the number of points marked for deletion is large, almost all points in the graph will need to modify the edge table. Therefore, this method minimizes the efficiency loss caused by scanning points that do not require edge table modifications.

[0065] Step S3: For each point p to be deleted that has an external neighbor, execute: obtain the points within two hops of the point p to be deleted in the current graph index to construct p out The candidate set of new inner neighbors, p out For the outer neighbors of the point p to be deleted, connect the points in the candidate set to p out .

[0066] Specifically, for the convenience of description, p is used to represent the point to be deleted, N out (p) represents the set of outer neighbors of point p, p out represents any outer neighbor of p, p out ′ represents any other outer neighbor of p.

[0067] For each point p to be deleted that has an external neighbor, its external neighbor set N can be obtained out (p), for its external neighbor set N out Every outer neighbor p in (p) out , it is necessary to establish its candidate set and repair its path.

[0068] The traditional way to establish a candidate set and perform path repair is as follows:

[0069] With p out To query the target, we use the graph index greedy search algorithm to start from the graph index search starting point and obtain the out It can be understood that the monotone search path means that along the extension direction of the path, the distance between each point on the path and the query target is getting shorter and shorter, that is, each point on the path is getting closer and closer to the query target, so as to be monotonically pout The monotone search path closest to p out A preset number of points are added to the candidate set, that is, each point in the candidate set has a monotonicity of p out Monotonic search path of ;

[0070] This method can ensure that all points in the candidate set are within reach of p out On the monotone search path, establish these points pointing to p out The edge of the path repair can repair the damaged monotone search path in the graph index with a high probability. However, although this method can ensure that the points in the candidate set are monotone to p out The monotonic search path has a good repair effect on the graph index. However, the greedy search algorithm has a high time overhead, so the solution of obtaining candidate sets through search limits the deletion efficiency.

[0071] Therefore, in order to improve efficiency and ensure the repair effect at the same time, the present invention proposes another method for quickly constructing a candidate set.

[0072] Specifically, obtain the points within two hops from the point to be deleted p in the current graph index to construct p out Candidate set of new inner neighbors.

[0073] The reasons for the above selection are:

[0074] Point p is p out One of the inner neighbors of , so point p is located at p out Nearby points, and the neighbors within two hops of point p are also located at p out points nearby, and with p out The greedy search result for the query is located at p out Since the neighbors of point p within two hops are highly overlapped with the greedy search results, the two-hop neighbors of point p can effectively replace the greedy search results. Moreover, since obtaining the two-hop neighbors of point p only requires O(1) time complexity, this approximate replacement can greatly improve the efficiency of candidate set generation.

[0075] Specifically, we can directly construct p with points within two hops. out The candidate set of new inner neighbors can also be constructed by further filtering some points from points within two hops. The former is more efficient, and the latter has a better repair effect. The specific selection can be flexibly made according to actual needs.

[0076] The selection of data usually has an upper limit. For the graph index of the database, its structure is very complex and each node usually has a large number of neighbors. Considering the data upper limit, points that do not exceed the data upper limit can be selected to construct a candidate set.

[0077] In one embodiment, points within two hops can be directly obtained to construct p out The candidate set of the new inner neighbor and ensure that the number of candidate sets does not exceed its upper limit:

[0078] Initialize the candidate set to be empty;

[0079] Scan the outer neighbors p of the point to be deleted according to the given point to be deleted out , and after scanning the current p out After that, scan the current p out The outer neighbor and the current p out After all the outer neighbors of point p are scanned, scan and delete the next outer neighbor p of point p. out ; Add the scanned points to the candidate set in sequence according to the scanning order until the storage limit of the candidate set is reached.

[0080] like Figure 2 The figure shows a specific flow chart of constructing a candidate set in one embodiment of the present invention, where the upper limit of the candidate set size is set to C s First, input the point p to be deleted and initialize the candidate set C to be empty. Then, take the first outer neighbor in the edge table of point p as p. out Add to candidate set C, if the size of candidate set C reaches C s , then stop building the candidate set and return to the current candidate set C. Otherwise, continue to build the candidate set out The outer neighbors of are also added to the candidate set C one by one. In this process, if the size of C reaches C s Then stop building the candidate set and return the current candidate set C, otherwise take the next point in the edge table of point p as p out Repeat this process until the size of the candidate set C reaches C s Or all outer neighbors of point p are scanned.

[0081] In another embodiment, in order to further improve the accuracy of path repair, further conditional constraints are added to filter out points that meet the constraint conditions from the points within two hops from the deletion point p to form a candidate set. Specifically, the points that are the outer neighbors of the point to be deleted and have a certain distance from p are selected. out The distance is less than d(p,p out ) are added to the candidate set, and points that are outer neighbors of the outer neighbors of the point to be deleted p and meet the three set conditions are added to the candidate set. The three set conditions include:

[0082] d(c,p out ) <d(p,p out )

[0083] d(c,p)>d(p,p out )

[0084] d 2 (c,p out )+d 2 (p,p out )>d 2 (c,p)

[0085] In the formula, p is the point to be deleted, p out is the outer neighbor of p, c is the point of p in the candidate set, d(c,p out ) represents c and p out The distance in the graph index, d(p,p out ) for p and p out The distance in the graph index, d(c,p) is the distance between c and p in the graph index.

[0086] For example, define a candidate region CR(p,p out ):

[0087] CR(p,p out )={x|d(x,p out ) <d(p,p out ),d(x,p)>d(p,p out ),

[0088] d 2 (x,p out )+d 2 (p,p out )>d 2 (x,p)}

[0089] The points in the above candidate areas are the points that meet the three conditions.

[0090] Similarly, considering the data upper limit, the following specific method is used to filter out points that meet the constraints from the points within two hops from the deletion point p and ensure that the number of candidate sets does not exceed its upper limit:

[0091] Initialize the candidate set to be empty;

[0092] According to the given point p to be deleted and one of its outer neighbors p out , scan the other outer neighbors p of the point to be deleted in turn out ', when scanning to the same out The distance is less than d(p,p out ) out 'After that, scan the current p out 'Outer neighbors until the current p out 'After all the outer neighbors are scanned, scan the next outer neighbor p of the point to be deleted out '; Add the points that meet the conditions for joining the candidate set to the candidate set in sequence according to the scanning order until the storage limit of the candidate set is reached.

[0093] like Figure 3 The figure shows a specific flow chart of constructing a candidate set in another embodiment of the present invention, setting the parameter C s , first input the point to be deleted p and the current outer neighbor p out , and then calculate the points p in the edge table one by one out distance, and the first distance p out Less than d(p,p out ) out ' is added to the candidate set C. If the size of the candidate set C reaches C s , then stop building the candidate set and return the current candidate set C, otherwise p out ′’s outer neighbors are located in the candidate region CR(p,p out ) points are added to the candidate set. In this process, if the size of C reaches C s Then stop building the candidate set and return the current candidate set C, otherwise the next distance p in the edge table of point p out Less than d(p,p out ) as the point p out ’ and repeat the process until the size of the candidate set C reaches C s Or all outer neighbors of point p are scanned.

[0094] The following is the candidate region CR(p,p out ) are explained below.

[0095] d(c,p out ) <d(p,p out ) can ensure that the points in the candidate set are as close to p as possible out Since traditional graph index algorithms give higher priority to short edges when pruning edges, that is, they tend to retain shorter edges, in order to successfully repair the monotone search path, it is necessary to make the new connection to p out The edges of the graph index are not deleted as much as possible by the edge pruning strategy. In order to meet this condition as much as possible, the scheme requires that the points in the candidate set need to be at a distance of p out Close enough so that the newly connected edge is as short as possible, and the edge (p, p out ) as an edge that existed in the graph before deletion, can be used as a criterion for judging whether it is close enough, that is, it can be considered that it is closer than the edge (p, p out ) Longer edges are likely to be deleted by the graph index edge pruning algorithm, while shorter edges are more likely to remain in the graph index.

[0096] d(c,p)>d(p,p out) can ensure that the points in the candidate set are as far away from point p as possible. Since the points farther away from point p are deleted, it is likely that the monotonic search path to point c still exists. The points in the candidate set can repair the path to point p. out The premise of a monotone path is that the point has a monotone out If the point in the candidate set does not even have a monotone search path to reach the point after point p is deleted, then it is impossible to repair the point to reach p. out Therefore, the points in the candidate set should still have a monotonic search path after point p is deleted. To achieve this goal as much as possible, the scheme requires that the points in the candidate set are as far away from point p as possible.

[0097] d 2 (c,p out )+d 2 (p,p out )>d 2 (c,p) can ensure that the points in the candidate set are located on the side close to point p. Since the points near point p are connected to p out After that, the edges formed are monotone with respect to the endpoints of most of the broken monotone search paths. In order to repair the broken monotone search paths, it is necessary to establish monotone search paths that reach the outer neighbors of the point to be deleted and are also monotone with respect to the endpoints of the broken paths. A necessary condition for achieving this goal is that the edges newly connected to repair the monotone search paths are monotone with respect to the endpoints of these broken paths. For the points located on the side close to point p, since the edges pointing from these points to p are monotone, the edges newly connected to repair the monotone search paths are monotone with respect to the endpoints of these broken paths. out The edge has the edge (p,p out ) components, so these edges can be monotonic to the endpoints of most broken paths.

[0098] Through the above method, although further screening is time-consuming, it can significantly improve the repair effect and obtain the most effective candidate set for repairing monotonic paths.

[0099] After determining the candidate set, connect the points in the candidate set to p out , so that p can be repaired out , and then the path is repaired.

[0100] Considering connecting a large number of points to p out This will seriously affect the update efficiency, because it is time-consuming to modify the edge table of a large number of points. In one embodiment, after obtaining the candidate set, in order to ensure the repair effect and improve the efficiency, valid points are further selected from the candidate set to connect to p out , that is, increase p by the following means out The incoming edge:

[0101] First, the distance p in the current candidate set outThe nearest point Move into the result set and delete the candidates in the current candidate set that meet the current point The distance is less than p out Repeat this step until the current candidate set is empty or the number of nodes in the result set reaches its upper limit;

[0102] Add a point from the result set to p out The outgoing edge.

[0103] For example, setting the upper limit of the result set δ in When screening the candidate set C, first sort the points in the candidate set according to the distance p out Sort by increasing distance, and then select the distance p in C out The nearest point Add to the result set, then delete from C all Point c on the same side, that is, Then, the process of selecting the nearest point and excluding the points on the same side is repeated in the remaining candidate sets until the candidate set is empty or the result set size reaches the upper limit δ in Finally, connect the filtered results to p out , that is, add the edge table of the points in the result set to point to p out edge.

[0104] The reason for screening by the above method is that: since it is necessary to make the newly connected edge monotone to the end point of the destroyed monotone search path as monotone as possible, the edge finally connected to p out The edges of p should point in all directions as much as possible, so these new connections to p out The starting point of the edge should be located at p as much as possible out In this embodiment, the points in the candidate set are regarded as p out The above strategy can eliminate the edges with similar directions, so that the results after screening are evenly distributed in p out Therefore, the results after the above strategy screening can satisfy the requirement of making the newly connected edges monotone to the end point of the destroyed monotone search path as much as possible.

[0105] Since existing graph index algorithms usually limit the out-degree upper limit of each point, not every point in the filtered result set can be directly added to the edge table pointing to p out Therefore, when connecting the points in the result set to p out Afterwards, if the out-degree (number of outgoing edges) of the points in the result set exceeds the graph index out-degree threshold, the edge pruning algorithm is further executed to limit the out-degree, that is, the method further includes:

[0106] For the points in the result set whose outgoing edges are greater than the set upper limit, the edge reduction operation is performed according to the SNG edge trimming algorithm. The specific process is: for the point v in the result set whose outgoing edges are greater than the set upper limit, the edge reduction operation is performed. The edge reduction operation includes: removing the point closest to v in the current outer neighbor set of v. Move into the candidate set of v's outer neighbors and delete the ones in the current set of outer neighbors that meet the requirements of the current point. The distance from is less than the distance from v and Repeat this step until the current outer neighbor set is empty or the number of nodes in its outer neighbor candidate set reaches the upper limit of the number of outgoing edges, and finally use its outer neighbor candidate set as the outer neighbor set of point v.

[0107] For example, the SNG edge trimming algorithm removes the original outgoing edges and points to p out The edges of the node are the candidate set of the outgoing edges of the node, and the outgoing degree upper limit θ of the graph index is set. d , according to the above SNG edge trimming algorithm, delete the edges that do not meet the SNG conditions, and use the screening results as the new outgoing edges of the point.

[0108] Step S4: Delete all the points to be deleted and all their outgoing edges to update the graph index of the dynamic vector database.

[0109] Specifically, after executing the above process, the monotone search path with damaged graph index has been repaired, so all outgoing edges of the point to be deleted can be directly deleted, that is, the outgoing edge table of the point to be deleted is set to an empty table.

[0110] Finally, all resources occupied by the point to be deleted can be released. At this point, the deletion process is completed.

[0111] like Figure 4 The figure shows a detailed flowchart of a method for updating a neighbor graph index of a dynamic vector database in an embodiment of the present invention. In a specific embodiment, the method is executed as follows: first, a list of points to be deleted is input, then all the incoming edges of the points to be deleted are deleted from the graph, and then the points to be deleted are processed one by one. The processing process is as follows: the currently processed point to be deleted is set as point p, and one of the outer neighbors of point p is selected. out Repair the outer neighbors of the monotone search path for the current one and get p out The candidate set of the new inner neighbor is then further filtered through the following process: first, the current candidate set is the distance p out The nearest point Move into the result set and delete the candidates in the current candidate set that meet the current point The distance is less than p out Repeat this step until the current candidate set is empty or the number of nodes in the result set reaches its upper limit; after obtaining the screening results, connect the points in the result set to pout , and the out-degree exceeds the upper limit of the out-degree of the graph index θ during the connection process d The point is subjected to the edge reduction operation according to the SNG edge trimming algorithm, and the existing outer neighbor set of the point is used as the candidate set and the number of neighbors less than θ is selected from them. d The outer neighbors of point p are used as its final outer neighbor set. After the monotonic search paths of all outer neighbors of point p are repaired according to the above process, the processing of the point to be deleted is completed. After all the points to be deleted are processed, the outgoing edges of the point to be deleted in the graph and the resources they occupy are released, and the deletion is completed.

[0112] like Figure 5 The figure shows the repaired point p after deleting the point p in one embodiment of the present invention. out Example of a candidate set obtained by the monotone search path. The yellow points in the figure are the selected candidate points, where c1 and c2 are the distances p between the outer neighbors of point p. out The distance is less than d(p,p out ), c3, c4 are outer neighbors of c1, c2 and are located in the region CR(p, p out ). Figure 6 The figure shows an embodiment of the present invention in which a candidate set is screened and the screening result is connected to p out In the example diagram, the gray points are the points deleted from the candidate set, and the orange points are the final screening results, such as Figure 7 The figure shows an example of the monotonic search path repair effect in one embodiment of the present invention, where s is the starting point and the orange path in the figure is the repaired monotonic search path. It can be seen that after deleting point p in this example, the damaged monotonic search path in the figure is effectively repaired.

[0113] like Figure 8 The figure shows the experimental comparison of the present invention and FreshDiskAnn in indexing accuracy. Figure 9 The figure shows the experimental comparison of the deletion throughput of the present invention and FreshDiskAnn. The experiments are based on the HNSW index (the existing approximate nearest neighbor search static index) and are conducted on the SIFT1M dataset. In each round, 1% of the points in the dataset are randomly selected, deleted from the index, and then reinserted to ensure that the exact solution of the query remains unchanged. The same query set is used to test the index accuracy. Figure 8 It can be seen from the figure that the FreshDiskAnn index has a problem of decreased accuracy in the first few rounds of updates, while the present invention significantly improves the index accuracy after the update and maintains a high accuracy. Figure 9 It can be seen from the figure that the deletion throughput of the present invention is much higher than that of FreshDiskAnn, which shows that the present invention has a very high update efficiency.

[0114] Example 2

[0115] The present invention also relates to an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0116] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The processor may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The memory may be used to store computer programs and / or modules, and the processor may perform various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory.

[0117] Example 3

[0118] The present invention also relates to a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0119] Specifically, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0120] Example 4

[0121] An embodiment of the present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the method of the above embodiment of the present invention.

[0122] The technical features of the above-described embodiments may be combined in any manner. To simplify the description, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. It should be noted that the phrases "in one embodiment," "for example," "and another example," etc., of the present invention are intended to illustrate the present invention and are not intended to limit the present invention.

[0123] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, and all such variations and improvements fall within the scope of protection of the present invention.

Claims

1. A method for updating a neighbor graph index of a dynamic vector database, characterized in that: include: Obtain the points to be deleted in the current graph index of the dynamic vector database; Deleting all incoming edges of the points to be deleted from the graph index; For each point p to be deleted that has an external neighbor, execute: Get the points within two hops of the point p to be deleted in the current graph index to construct p out The candidate set of new inner neighbors, p out For the outer neighbors of the point p to be deleted, connect the points in the candidate set to p out ; Delete all the points to be deleted and all their outgoing edges to update the graph index of the dynamic vector database; Among them, the outer neighbors of any point are the points connected to the outgoing edges of the point, and the inner neighbors of any point are the points connected to the incoming edges of the point; Among them, obtain the points within two hops from the point to be deleted p in the current graph index to construct p out The candidate set of new inner neighbors includes: The outer neighbors of the point to be deleted and the points with the same out The distance is less than d(p,p out ) are added to the candidate set, and points that are outer neighbors of the point to be deleted p and that meet three set conditions are added to the candidate set, wherein the three set conditions include: d(c,p out )<d(p,p out ) d(c,p)>d(p,p out ) d 2 (c,p out )+d 2 (p,p out )>d 2 (c,p) In the formula, p is the point to be deleted, p out is the outer neighbor of p, c is the point of p in the candidate set, d(c,p out ) represents c and p out The distance in the graph index, d(p,p out ) for p and p out The distance in the graph index, d(c,p) is the distance between c and p in the graph index.

2. The method for updating a neighbor graph index of a dynamic vector database according to claim 1, wherein: Deleting all incoming edges of the points to be deleted from the graph index includes: Scan each point in the current graph index in turn and delete the outgoing edge from the currently scanned point to the point to be deleted.

3. The method for updating a neighbor graph index of a dynamic vector database according to claim 1, wherein: Get the points within two hops of the point to be deleted p in the current graph index to construct p out The candidate set of new inner neighbors includes: Initialize the candidate set to be empty; According to the given point p to be deleted and one of its outer neighbors p out , scan the other outer neighbors p of the point to be deleted in turn out ', when scanning to the same out The distance is less than d(p,p out ) out 'After that, scan the current p out 'Outer neighbors until the current P out 'After all the outer neighbors are scanned, scan the next outer neighbor p of the point to be deleted out '; Add the points that meet the conditions for joining the candidate set to the candidate set in sequence according to the scanning order until the storage limit of the candidate set is reached.

4. The method for updating a neighbor graph index of a dynamic vector database according to claim 1, wherein: The point in the candidate set is connected to p out ,include: First, the distance p in the current candidate set out The nearest point Move into the result set and delete the candidates in the current candidate set that meet the current point The distance is less than p out Repeat this step until the current candidate set is empty or the number of nodes in the result set reaches its upper limit; Add a point from the result set to p out The outgoing edge.

5. The method for updating a neighbor graph index of a dynamic vector database according to claim 4, wherein: The method further comprises: For the point v in the result set whose outgoing edge is greater than the set upper limit, the edge reduction operation is performed. The edge reduction operation includes: removing the point closest to v in the current outer neighbor set of v. Move into the candidate set of v's outer neighbors and delete the ones in the current set of outer neighbors that meet the requirements of the current point. The distance from is less than the distance from v and Repeat this step until the current outer neighbor set is empty or the number of nodes in its outer neighbor candidate set reaches the upper limit of the number of outgoing edges, and finally use its outer neighbor candidate set as the outer neighbor set of point v.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images