Vector index updating method and device, computer equipment and storage medium

Abstract

The invention provides a vector index updating method and device, computer equipment and a storage medium. The method comprises the following steps: according to a target vector to be merged into a vector index, determining a neighbor node corresponding to the target vector from the vector index; wherein the vector similarity between the target vector and the neighbor node is greater than or equal to a preset similarity threshold; determining a target disk page; wherein the number of available storage positions in the target disk page is greater than a preset number threshold; based on the target vector and the corresponding neighbor node, generating a record corresponding to the target vector, and writing the record into the target disk page; and based on the target vector, updating the record of the neighbor node to obtain a new record corresponding to the neighbor node, and writing the new record corresponding to the neighbor node into the target disk page.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, computer device, and storage medium for updating a vector index. Background Technology

[0002] With the development of artificial intelligence technology, the semantic information of multimodal data such as text, audio, and video is usually encoded as high-dimensional vectors. Content-based retrieval can be achieved by searching for the nearest vectors to the target vector within the dataset. However, for high-dimensional vectors, exact retrieval algorithms are highly complex and difficult to adapt to large-scale multimodal datasets; therefore, approximate retrieval has become the mainstream approach.

[0003] The core of approximate retrieval is vector indexing. It first builds a vector index on high-dimensional vectors in a multimodal dataset, and then performs retrieval on the target vector within this index. Approximate Nearest Neighbor Search (ANNS), as an approximate retrieval technique, is a key technology for multimodal data retrieval in web search and retrieval enhancement generation. Multimodal data (such as text, images, and audio) is encoded into high-dimensional vectors by neural networks, and a vector index is built based on these high-dimensional vectors. During the search, the k nearest neighbors of the query vector are obtained by traversing the vector index, and data retrieval is performed based on these k nearest neighbors. In ANNS vector indexing, graph-based vector indexes are favored due to their performance and cost-effectiveness. In graph-based vector indexes, high-dimensional vectors are organized into a graph and stored on disk. Compared to other types of indexes, graph-based vector indexes can maintain higher accuracy and higher throughput in large-scale approximate vector retrieval.

[0004] Currently, there is a problem of poor update efficiency when updating graph-based vector indices. Summary of the Invention

[0005] In view of this, this application provides a method, apparatus, computer device, and storage medium for updating a vector index.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] In a first aspect, embodiments of this disclosure provide a method for updating a vector index, the method comprising:

[0008] Based on the target vector to be merged into the vector index, the neighboring nodes corresponding to the target vector are determined from the vector index; wherein the vector similarity between the target vector and the neighboring nodes is greater than or equal to a preset similarity threshold;

[0009] Identify the target disk page; wherein the number of available storage locations in the target disk page is greater than a preset threshold.

[0010] Based on the target vector and its corresponding neighbor nodes, a record corresponding to the target vector is generated, and the record is written to the target disk page; and...

[0011] Based on the target vector, the records of the neighboring nodes are updated to obtain the new records corresponding to the neighboring nodes, and the new records corresponding to the neighboring nodes are written to the target disk page.

[0012] Optionally, determining the target disk page to which the target vector is inserted includes at least one of the following:

[0013] The target disk page is determined from the list of blank disk pages;

[0014] The disk pages corresponding to each node in the search path when determining neighboring nodes are used as candidate disk pages, and the target disk page is determined from the candidate disk pages.

[0015] Allocate a new blank page on the disk as the target disk page.

[0016] Optionally, the step of updating the records of the neighboring nodes based on the target vector to obtain the new records corresponding to the neighboring nodes includes:

[0017] If the number of outgoing neighbors of a neighboring node has not reached the maximum outgoing degree, the target vector is used as a new outgoing neighbor of the neighboring node, the identifier of the target vector is added to the record corresponding to the neighboring node, and a new record corresponding to the neighboring node is generated.

[0018] When the number of outgoing neighbors of the neighbor node reaches the maximum outgoing degree, the target vector is used as an increment to perform an incremental triangle inequality judgment on the target vector and the original outgoing neighbors of the neighbor node, so as to determine the outgoing neighbors to be deleted from the original outgoing neighbors of the neighbor node.

[0019] The outgoing neighbor to be deleted is removed from the record corresponding to the neighbor node, and the target vector is used as the new outgoing neighbor of the neighbor node. The identifier of the target vector is added to the record corresponding to the neighbor node, and a new record corresponding to the neighbor node is generated.

[0020] Optionally, the method further includes:

[0021] Change the storage location corresponding to the original record of the neighboring node to an available state; the storage location in an available state can be written with other records.

[0022] Optionally, after changing the storage location corresponding to the original record of the neighboring node to an available state, the method further includes:

[0023] Determine whether all storage locations in the disk page where the original record of the neighboring node is located are in an available state;

[0024] If all storage locations in the disk page containing the original record of the neighboring node are available, add the disk page containing the original record of the neighboring node to the list of blank disk pages.

[0025] Optionally, the method further includes:

[0026] In the storage location lookup table, the storage location of the record corresponding to the neighbor node is updated to the storage location of the new record corresponding to the neighbor node in the target disk page;

[0027] The storage location lookup table is used to store the correspondence between each node in the vector index and its storage location on the disk.

[0028] Optionally, before updating the records of the neighboring nodes based on the target vector, the process further includes:

[0029] The original record of the neighboring node is read, and a locking operation is performed on the original record corresponding to the neighboring node so that the original record corresponding to the neighboring node enters a locked state; in the locked state, the original record corresponding to the neighboring node cannot be accessed by the retrieval operation;

[0030] After writing the new record corresponding to the neighbor node to the target disk page, the method further includes:

[0031] Perform an unlock operation on the original record corresponding to the neighbor node.

[0032] Optionally, the method further includes:

[0033] In the case of concurrent retrieval operations, and the retrieval operation needs to read the record corresponding to the neighbor node, before the retrieval operation reads the record corresponding to the neighbor node, it is determined whether the record corresponding to the neighbor node is in a locked state;

[0034] When the record corresponding to the neighbor node is in a locked state, after waiting for the record corresponding to the neighbor node to be unlocked, the record corresponding to the neighbor node is locked, and after reading the record corresponding to the neighbor node, the record corresponding to the neighbor node is unlocked.

[0035] Secondly, embodiments of this disclosure also provide a vector index updating apparatus, the apparatus comprising:

[0036] The first determining module is used to determine the neighboring nodes corresponding to the target vector from the vector index based on the target vector to be merged into the vector index; wherein the vector similarity between the target vector and the neighboring nodes is greater than or equal to a preset similarity threshold.

[0037] The second determining module is used to determine the target disk page; wherein the number of available storage locations in the target disk page is greater than a preset number threshold.

[0038] The first update module is configured to generate a record corresponding to the target vector based on the target vector and the corresponding neighbor nodes, and write the record to the target disk page; and,

[0039] The second update module is used to update the records of the neighboring nodes based on the target vector, obtain the new records corresponding to the neighboring nodes, and write the new records corresponding to the neighboring nodes to the target disk page.

[0040] Optionally, the second determining module, when determining the target disk page to which the target vector is inserted, performs at least one of the following:

[0041] The target disk page is determined from the list of blank disk pages;

[0042] The disk pages corresponding to each node in the search path when determining neighboring nodes are used as candidate disk pages, and the target disk page is determined from the candidate disk pages.

[0043] Allocate a new blank page on the disk as the target disk page.

[0044] The second update module, when updating the records of the neighboring nodes based on the target vector to obtain the new records corresponding to the neighboring nodes, is used for:

[0045] If the number of outgoing neighbors of a neighboring node has not reached the maximum outgoing degree, the target vector is used as a new outgoing neighbor of the neighboring node, the identifier of the target vector is added to the record corresponding to the neighboring node, and a new record corresponding to the neighboring node is generated.

[0046] When the number of outgoing neighbors of the neighbor node reaches the maximum outgoing degree, the target vector is used as an increment to perform an incremental triangle inequality judgment on the target vector and the original outgoing neighbors of the neighbor node, so as to determine the outgoing neighbors to be deleted from the original outgoing neighbors of the neighbor node.

[0047] The outgoing neighbor to be deleted is removed from the record corresponding to the neighbor node, and the target vector is used as the new outgoing neighbor of the neighbor node. The identifier of the target vector is added to the record corresponding to the neighbor node, and a new record corresponding to the neighbor node is generated.

[0048] Optionally, the second update module is also used for:

[0049] Change the storage location corresponding to the original record of the neighboring node to an available state; the storage location in an available state can be written with other records.

[0050] Optionally, after changing the storage location corresponding to the original record of the neighboring node to an available state, the second update module is further configured to:

[0051] Determine whether all storage locations in the disk page where the original record of the neighboring node is located are in an available state;

[0052] If all storage locations in the disk page containing the original record of the neighboring node are available, add the disk page containing the original record of the neighboring node to the list of blank disk pages.

[0053] Optionally, the second update module is also used for:

[0054] In the storage location lookup table, the storage location of the record corresponding to the neighbor node is updated to the storage location of the new record corresponding to the neighbor node in the target disk page;

[0055] The storage location lookup table is used to store the correspondence between each node in the vector index and its storage location on the disk.

[0056] Optionally, before updating the records of the neighboring nodes based on the target vector, the second update module is further configured to:

[0057] The original record of the neighboring node is read, and a locking operation is performed on the original record corresponding to the neighboring node so that the original record corresponding to the neighboring node enters a locked state; in the locked state, the original record corresponding to the neighboring node cannot be accessed by the retrieval operation;

[0058] The second update module, after writing the new record corresponding to the neighbor node to the target disk page, is further configured to:

[0059] Perform an unlock operation on the original record corresponding to the neighbor node.

[0060] Optionally, it also includes: a processing module for:

[0061] In the case of concurrent retrieval operations, and the retrieval operation needs to read the record corresponding to the neighbor node, before the retrieval operation reads the record corresponding to the neighbor node, it is determined whether the record corresponding to the neighbor node is in a locked state;

[0062] When the record corresponding to the neighbor node is in a locked state, after waiting for the record corresponding to the neighbor node to be unlocked, the record corresponding to the neighbor node is locked, and after reading the record corresponding to the neighbor node, the record corresponding to the neighbor node is unlocked.

[0063] Thirdly, an optional implementation of this disclosure also provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, wherein when the machine-readable instructions are executed by the processor, they perform the steps of the first aspect above, or any possible implementation of the first aspect.

[0064] Fourthly, an optional implementation of this disclosure also provides a computer-readable storage medium storing a computer program that, when run, performs the steps of the first aspect or any possible implementation of the first aspect.

[0065] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0066] The graph-based vector index update method provided in the disclosed embodiments, during the process of inserting a target vector and updating the records of the corresponding neighbor nodes on the disk, directly inserts the updated records corresponding to the neighbor nodes into the target disk page; the number of available storage controls in the target disk page is greater than a preset number threshold, thereby controlling the specific value of the number threshold to control the number of records written to a target disk page, so that multiple records can be written to the target disk page, thereby merging disk writes to reduce the number of disk accesses during the record update process, thereby improving the efficiency of vector index update.

[0067] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0069] Figure 1 Specific examples of storing graph-based vector indexes on disk, as provided in some embodiments of this disclosure, are illustrated.

[0070] Figure 2 A flowchart illustrating a method for updating a vector index provided in some embodiments of this disclosure is shown;

[0071] Figure 3 Specific examples of updating vector indexes provided by some embodiments of this disclosure are shown;

[0072] Figure 4 A schematic diagram of a vector index updating apparatus provided in some embodiments of this disclosure is shown;

[0073] Figure 5 A schematic diagram of a computer device provided in some embodiments of the present disclosure is shown. Detailed Implementation

[0074] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0075] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0076] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0077] In graph-based vector indexing, vectors are organized into a directed graph and stored on disk as an adjacency list; each disk page stores multiple records. Each record consists of a vector (node) and the identifiers (edges) of its outgoing neighbors. To ensure a fixed record size, the maximum out-degree of a node (i.e., the maximum number of outgoing neighbors) is globally limited to R, where R is a configurable parameter.

[0078] like Figure 1 The diagram illustrates a specific example of storing a vector index on disk; in this example, the vector index comprises 8 nodes, numbered 0 to 7; the adjacency relationships between nodes 0 to 7 are as follows: Figure 1 As shown in Figure a. For any given node, based on its identifier and the identifiers of its corresponding outgoing neighbor nodes, a record corresponding to that node is created and stored in a storage location within a disk page, as shown. Figure 1 As shown in Figure b, the records corresponding to nodes 0 to 4 are stored in disk page Page 0, and the records corresponding to nodes 5 to 7 are stored in disk page Page 1. Taking node 1 as an example, its outgoing neighbor nodes include: node 0, node 2, node 3, and node 5; therefore, the record corresponding to node 1 includes: the identifier "1" of node 1, and the identifiers of the outgoing neighbor nodes corresponding to node 1: "0", "2", "3", and "5".

[0079] Vector indexes need to support updates to adapt to changing datasets. There are generally two ways to update vector indexes: index rebuilding and index updating. Index rebuilding involves reconstructing the vector index using the latest incremental vectors and existing vectors from the original index. This is time-consuming, and during the rebuilding process, the existing vector index must support the front-end search task, making it impossible to keep the search results up-to-date. Index updating, on the other hand, merges the high-dimensional vectors corresponding to the incremental multimodal data into the existing vector index. Specifically, the vectors to be updated are stored in memory as incremental indexes, and these incremental indexes are periodically merged into the full index on disk. Compared to index rebuilding, this incremental updating method ensures the real-time nature of search results. However, in practice, due to disk bandwidth limitations, the merging and updating process can interfere with the front-end search task, reducing the efficiency of the merging and updating process and significantly increasing the latency of the front-end search task.

[0080] Furthermore, in the index update method described above, the vectors to be updated are first organized into a memory index (vector index stored in memory) and stored in memory. During the search process, the search is performed from both the memory index and the disk index (vector index stored on disk). When the size of the memory index reaches a certain threshold, a merge update process is triggered. During the merge update, all vectors in the memory index are used as incremental vectors and merged into the disk index. During the merge update, similar neighbor vectors corresponding to the incremental vector need to be determined in the vector index. Based on the position of the neighbor vectors in the vector index, the incremental vector is inserted into the vector index, and the records of similar vectors in the vector index are updated. In graph-based vector indexes, each vector may connect dozens or even hundreds of vectors as its neighbor vectors. The process of inserting the vector to be updated into the vector index, in addition to writing the vector to be updated to disk, also requires adding bidirectional edges between the vector to be updated and its neighbors. This means that the records of dozens or even hundreds of neighbor vectors of the vector to be updated need to be updated on disk, resulting in a large number of random SSD writes. To reduce writes, the current solution is to use a log-structured data layout. However, assuming the maximum out-degree of the component index is 128 (meaning the maximum number of out-neighbors for each node in the vector index is 128), inserting 10% new vectors into the vector index will increase the vector index by 12.8 times. Of these, only 7.9% of the records are valid, and the rest are invalid data that needs to be garbage collected (GC) to handle. This process will further interfere with the processing of front-end search requests.

[0081] Meanwhile, during the merging process, all graph updates on disk are buffered in memory, resulting in the current index update method consuming a significant amount of memory. Updating similar vectors also consumes substantial memory and disk bandwidth, reducing the efficiency of vector index updates.

[0082] Based on this, this disclosure provides a graph-based vector index update method. In the process of inserting a target vector and updating the records of the corresponding neighbor nodes on the disk, the updated records of the neighbor nodes are directly inserted into the target disk page. The number of available storage controls in the target disk page is greater than a preset threshold. By controlling the specific value of the threshold, the number of records written to a target disk page can be controlled, so that multiple records can be written to the target disk page. This achieves disk merging to reduce the number of disk accesses during record updates, thereby improving the efficiency of vector index updates.

[0083] Furthermore, the embodiments of this disclosure employ an in-situ insertion mechanism. After obtaining the target vector to be inserted, the target vector to be inserted is directly written into the full index on the disk, thereby amortizing the interference of vector insertion operation on the front-end query in the time dimension.

[0084] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0085] To facilitate understanding of this embodiment, a detailed description of the vector index update method disclosed in this disclosure is provided first. The execution entity of the vector index update method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the vector index update method can be implemented by the processor calling computer-readable instructions stored in memory.

[0086] The method for updating vector indexes provided in the embodiments of this disclosure will now be described.

[0087] See Figure 2 The diagram shows a flowchart of a vector index update method provided in this embodiment of the present disclosure. The method includes steps S201 to S204, wherein:

[0088] S201: Based on the target vector to be merged into the vector index, determine the neighboring nodes corresponding to the target vector from the vector index; wherein the vector similarity between the target vector and the neighboring nodes is greater than or equal to a preset similarity threshold;

[0089] S202: Determine the target disk page; wherein the number of available storage locations in the target disk page is greater than a preset threshold.

[0090] S203: Based on the target vector and the corresponding neighbor nodes, generate a record corresponding to the target vector and write the record to the target disk page;

[0091] S204: Based on the target vector, update the records of the neighboring nodes to obtain the new records corresponding to the neighboring nodes, and write the new records corresponding to the neighboring nodes to the target disk page.

[0092] In this embodiment of the disclosure, when updating the vector index, the neighboring nodes corresponding to the target vector to be merged into the vector index are determined from the vector index, and a target disk page for inserting the target vector is determined; the number of available storage controls in the target disk page is greater than a preset number threshold; then, based on the target vector and the corresponding neighboring nodes, a record corresponding to the target vector is generated and written to the target disk page; and, based on the target vector, the records of the neighboring nodes are updated to obtain new records corresponding to the neighboring nodes, and the new records corresponding to the neighboring nodes are written to the target disk page. In this way, by controlling the specific value of the number threshold, the number of records written to a single target disk page can be controlled, enabling multiple records to be written to the same target disk page, thereby reducing the number of disk accesses during record updates and improving the efficiency of vector index updates.

[0093] The following provides a detailed explanation of S201 to S204.

[0094] Regarding the above S201:

[0095] In specific implementations, when inserting a target vector into a vector index, the embodiments of this disclosure employ an in-situ insertion mechanism, that is, after obtaining the target vector to be inserted, the vector index update method provided in this disclosure is used directly to write the target vector to be inserted into the full index on the disk, instead of storing it in memory first and merging it only after the size of the vector index in memory reaches a certain size. In this way, the bandwidth required for batch vector insertion operations can be amortized over time, thereby reducing the interference of the vector index update process on the front-end query.

[0096] When determining the neighboring nodes corresponding to the target vector from the vector index based on the target vector to be merged into the vector index, for example, one can start from a fixed starting vector s in the vector index, and search along a certain search path according to certain search rules to search at least some vectors in the vector index to determine the distance between the searched vectors and the target vector; and based on this distance, maintain a candidate pool P containing the vectors closest to the target vector; when the search cutoff condition is met, the vectors in the candidate pool P are the nodes closest to the target vector among all the searched nodes. At this time, the nodes in the candidate pool P can be used as the neighboring nodes of the target vector, and the target vector can be inserted into the vector index.

[0097] Regarding the above S202:

[0098] In practice, the target disk page is the disk page to be written to the target vector and the updated neighbor nodes; the number of available storage locations in the target disk page is greater than a preset threshold; this preset threshold is usually a configurable parameter. A larger preset threshold value results in more records that can be written to the same disk page and more records that can be merged, effectively allowing for fewer accesses to the disk page during record writing, thus improving update efficiency. However, the number of target disk pages that meet the requirements is relatively small, requiring more time to filter target disk pages from the disk pages. A smaller preset threshold range results in fewer records that can be written to the same disk page and fewer records that can be merged, effectively requiring more accesses to the disk page during record writing. Although the improvement in update efficiency is limited, the larger number of target disk pages that meet the requirements allows for more efficient filtering of target disk pages from various disk pages, thus also improving insertion efficiency. Therefore, the number of records written to the same disk page each time an update operation is performed on the vector index can be controlled by adjusting the specific value of the preset threshold according to actual needs.

[0099] When determining the target disk page for inserting the target vector, at least one of the following methods a1 to a3 can be used:

[0100] a1: Determine the target disk page from the list of blank disk pages.

[0101] Here, for example, the list of blank disk pages can be maintained in memory; during the vector index update process, when all storage locations in a disk page are set to a writable state, the identifier corresponding to that disk page can be added to the list of blank disk pages. When it is necessary to determine the target disk page, it can first be determined from the list of blank disk pages.

[0102] a2: The disk pages corresponding to each node in the search path when determining neighboring nodes are used as candidate disk pages, and the target disk page is determined from the candidate disk pages.

[0103] Here, each time a new target vector is inserted into the vector index, a search process for the nearest neighbor node is performed according to a certain path. During the search process, the accessed disk pages (i.e., the disk pages where the records corresponding to each searched node in the search path are stored) are cached in memory. Therefore, in order to avoid further reducing the update efficiency due to additional read operations on other disk pages during the vector index update process, in this embodiment of the disclosure, the disk pages corresponding to each node in the cached search path can be used as candidate disk pages. From the candidate disk pages, candidate disk pages with a number of available storage locations greater than a preset threshold are selected as the target disk pages.

[0104] a3: Allocate a new blank page on the disk as the target disk page.

[0105] The three methods for determining target disk pages described above can be used individually or in combination. When used in combination, for example, the insertion process described above (a1) can be used first to allocate target disk pages. If the number of free disk pages in the free disk page list is insufficient to satisfy the insertion process, then the insertion process described above (a2) can be used to allocate target disk pages. If the target disk pages allocated using methods a1 and a2 are insufficient, then method a3 can be used to allocate target disk pages. Alternatively, only one of methods a1, a2, or a3 can be used to allocate target disk pages for the target vector. For instance, when inserting a target vector, it can be determined whether there are enough free disk pages in the free disk page queue to satisfy the insertion process; if so, only method a1 can be used to allocate target disk pages for the target vector. Alternatively, we can skip method a1 above and directly use method a2 above to allocate target disk pages for the target vectors; or, in the case of concurrent insertion of multiple target vectors, it is difficult to allocate a sufficient number of target disk pages for all target vectors using the methods described in a1 and a2 above; in order to improve the efficiency of vector insertion, we can skip the allocation methods a1 and a2 above and directly use method a3 above to allocate target disk pages for some of the multiple target vectors, while other target vectors are still allocated target disk pages using the methods described in a1 and / or a2. In this way, we can improve the allocation efficiency of target disk pages, thereby improving the update efficiency of vector indexes during concurrent insertion.

[0106] Regarding the above S203:

[0107] In a specific implementation, the graph-based vector index is split into several records stored on disk, each record including a vector and the identifiers of its outgoing neighbors. Since each record has a fixed length, there is an upper limit to the number of its outgoing neighbors. Furthermore, this embodiment does not limit the storage method of the graph; for example, the vector and the outgoing neighbor numbers can also be stored separately.

[0108] Under this premise, in this embodiment of the disclosure, when generating a record corresponding to the target vector based on the target vector and its corresponding neighbor nodes, the record corresponding to the target vector may include, for example, the identifier of the target vector and the identifiers of each outgoing neighbor node. Then, the record corresponding to the target vector can be written to the target disk page.

[0109] Regarding the above S204:

[0110] In practical implementation, when inserting the target vector into the vector index, it is necessary to establish edges between the target vector and its neighboring nodes. These edges represent the connection relationship between the target vector and its neighboring nodes. Since the target vector and its neighboring nodes have a high similarity, during approximate search, it is possible to find both the target vector and its neighboring nodes. Therefore, the connection relationship between the target vector and its neighboring nodes typically includes connections from the target vector to its neighboring nodes, and connections from its neighboring nodes to its target vector. Because each record in the vector index usually stores the vector's identifier and the identifier of its corresponding outgoing neighbor, the target vector needs to be added as an outgoing neighbor of its neighboring nodes to update the records of the neighboring nodes corresponding to the target vector.

[0111] Furthermore, since the maximum out-degree R of each vector in the vector index is limited, if the number of outgoing neighbors of the target vector's neighboring node does not reach the maximum out-degree limit, the target vector can be directly written into the neighboring node's record as an outgoing neighbor, generating a new record for the neighboring node. If the number of outgoing neighbors of the target vector's neighboring node has reached the maximum out-degree limit, then pruning is required. Outgoing neighbors that are far away from the neighboring node are deleted, and the identifier of the target vector is used as the identifier of the new outgoing neighbor of the neighboring node and added to the neighboring node's record, forming a new record for the neighboring node.

[0112] Specifically, this disclosure provides a method for updating the records of neighboring nodes based on the target vector to obtain new records corresponding to the neighboring nodes, including:

[0113] When the number of outgoing neighbors of the neighboring node reaches the maximum outgoing degree, the target vector is used as an increment to perform a triangle inequality check on the target vector and the outgoing neighbors of the neighboring node to determine the outgoing neighbors to be deleted from the outgoing neighbors of the neighboring node.

[0114] The outgoing neighbor to be deleted is removed from the record corresponding to the neighbor node, and the target vector is used as the new outgoing neighbor of the neighbor node. The identifier of the target vector is added to the record corresponding to the neighbor node to form a new record corresponding to the neighbor node.

[0115] In practice, the process of determining the outgoing neighbors to be deleted is essentially a process of pruning the outgoing neighbors of a neighboring node by utilizing the property of a triangle that "the sum of any two sides is greater than the third side". Assume that the distance function between any two vectors is represented as d(·,·), and any three vectors a, b, and c satisfy: d(a,c)≤d(a,b)+d(b,c).

[0116] When using the target vector as an increment to perform incremental triangle inequality checks on the target vector and the outgoing neighbors of the neighboring nodes, it is assumed that during the construction and updating of the vector index, most of the original outgoing neighbors corresponding to each node in the vector index follow the triangle inequality. To prune the outgoing neighbors of the neighboring nodes and reduce the operational complexity during pruning, in this embodiment, when using the target vector as an increment to perform incremental triangle inequality checks on the target vector and the outgoing neighbors of the neighboring nodes, triangle inequality checks between the original outgoing neighbors of the outgoing neighboring nodes are no longer performed.

[0117] In other words, during this process, for each pair of outgoing neighbors of the target vector and its corresponding neighboring nodes, it is determined whether the triangle inequality is satisfied. If the triangle inequality is satisfied, these two outgoing neighbors are retained. If, for a pair of outgoing neighbors, the distance between the target vector and these two outgoing neighbors does not satisfy the triangle inequality, then the outgoing neighbors to be deleted are determined from these two outgoing neighbors.

[0118] For example, when performing incremental triangle inequality judgment, assume the target vector is represented as a, and the outgoing neighbors of the neighboring nodes are b1, b2 and b3 respectively.

[0119] For the target vector a and its outgoing neighbors b1 and b2, perform a triangle inequality check. That is, check whether the following triangle inequalities are satisfied: d(a,b1)≤d(a,b2)+d(b1,b2), d(a,b2)≤d(a,b1)+d(b1,b2), d(b1,b2)≤d(a,b1)+d(a,b2).

[0120] For the target vector a and its outgoing neighbors b1 and b3, perform a triangle inequality check, that is, check whether the following triangle inequality relationships are satisfied: d(a,b1)≤d(a,b3)+d(b1,b3), d(a,b3)≤d(a,b1)+d(b1,b3), d(b1,b3)≤d(a,b1)+d(a,b3).

[0121] For the target vector a and its outgoing neighbors b2 and b3, perform a triangle inequality check, that is, check whether the following triangle inequality relationships are satisfied: d(a,b2)≤d(a,b3)+d(b2,b3), d(a,b3)≤d(a,b2)+d(b2,b3), d(b2,b3)≤d(a,b2)+d(a,b3).

[0122] In the example above, assuming that k out-neighbors do not follow the triangle inequality, the out-neighbor with the farthest distance (such as Euclidean distance) among the k out-neighbors that do not follow the triangle inequality is selected as the out-neighbor to be deleted.

[0123] In another embodiment, if the incremental triangle inequality judgment on the target vector and the original out-of-neighbors of the neighboring node fails to determine the out-of-neighbor to be deleted from the original out-of-neighbors of the neighboring node, that is, if all out-of-neighbors of the neighboring node and the target vector satisfy the triangle inequality during the incremental triangle inequality judgment, then the triangle inequality judgment between the original out-of-neighbors is performed again to filter out the out-of-neighbor to be deleted.

[0124] After filtering out the outgoing neighbors to be deleted, the outgoing neighbor can be deleted from the original record corresponding to the neighbor node, and the target vector can be added to the record corresponding to the neighbor node as the new outgoing neighbor, forming a new record corresponding to the neighbor node, and the new record can be written to the target disk page.

[0125] By using the incremental triangle inequality judgment process described above, the pruning operation of the outgoing neighbor corresponding to the neighbor node can be reduced from a complexity of O(R²) to O(R²). 2 The computation time is reduced to O(R), thereby enabling fast pruning of neighboring nodes and improving the update process of records corresponding to neighboring nodes.

[0126] Meanwhile, the faster update process for records corresponding to neighboring nodes leads to a reduction in the locking time for those records. This indirectly reduces the impact of update operations on retrieval during concurrent retrieval and update operations, thereby improving retrieval response efficiency during concurrent retrieval and update operations.

[0127] In another embodiment of this disclosure, since the new record corresponding to the neighboring node has been written to the target disk page, while its original record is still stored in the original location of the original disk page, the storage location corresponding to the original record of the neighboring node can be set to an available state. At this time, although the original record and the new record of the neighboring node are stored in two separate storage locations, the storage location corresponding to the original record is set to an available state, so the original record is no longer effective. After the storage location corresponding to the original record is set to an available state, it can be used as an available storage location for rewriting other records.

[0128] Furthermore, in another embodiment of this disclosure, after changing the storage location corresponding to the original record of the neighboring node to an available state, the method further includes:

[0129] Determine whether all storage locations in the disk page where the original record of the neighboring node is located are in an available state;

[0130] If all storage locations in the disk page containing the original record of the neighboring node are available, add the disk page containing the original record of the neighboring node to the list of blank disk pages.

[0131] Here, if all storage locations on the disk page containing the original record of the neighboring node are available, it means that all storage locations on the disk page containing the original record of the neighboring node can be written with a new record.

[0132] At this point, the disk page containing the original record of the neighboring node can be added to the list of blank disk pages (as described in a1 above). When a new target vector is inserted into the vector index, the target disk page for the new target vector can be determined from the list of blank disk pages.

[0133] In another embodiment of this disclosure, to ensure that the record corresponding to a node can be read from the correct location, an "ID-location lookup table" can be maintained in memory, mapping node identifiers (IDs) to storage locations on disk pages. When the storage location of a neighboring node changes, the ID-location lookup table is updated with the new storage location of the neighboring node. Through this lookup table, the correct storage location of any node on the disk page can be found.

[0134] In another embodiment of this disclosure, before updating the records of neighboring nodes based on the target vector, the process may further include:

[0135] The original record of the neighbor node is read, and a locking operation is performed on the original record corresponding to the neighbor node so that the neighbor node enters a locked state; in the locked state, the record corresponding to the neighbor node cannot be accessed by retrieval operations;

[0136] After writing the new record corresponding to the neighbor node to the target disk page, the method further includes:

[0137] An unlock operation is performed on the original record corresponding to the neighbor node so that the record corresponding to the neighbor node can be accessed by the retrieval operation.

[0138] Specifically, during the insertion of the target vector into the vector index, concurrent retrieval operations may occur, which may require accessing the relevant records of the target vector's neighboring nodes. In related technologies, the retrieval and insertion processes lock all records in the query path. This results in the following situation when retrieval and insertion operations occur concurrently: the nodes accessed by the retrieval operation are locked by the insertion operation; the retrieval operation can only continue after the insertion operation is completed, thus reducing the efficiency of the retrieval operation; or the nodes accessed by the insertion operation are locked by the retrieval operation; the insertion operation can only continue after the retrieval operation is completed, thus reducing the efficiency of the insertion operation.

[0139] In this embodiment of the disclosure, to reduce the mutual interference between insertion and retrieval operations during concurrent retrieval and insertion operations, locking is performed at the record granularity. Specifically, before generating a new record corresponding to a neighboring node, the original storage location of the neighboring node can be determined using the ID-location lookup table. After obtaining the original storage location of the neighboring node, the original record corresponding to the neighboring node is read from that location, and a locking operation is performed on the original record corresponding to the neighboring node. Under this locking state, the original record corresponding to the neighboring node cannot be accessed by concurrent retrieval operations.

[0140] After generating a new record for the neighboring node and writing it to the target disk page, an unlock operation is performed on the original record for the neighboring node. At this point, concurrent retrieval operations can use the ID-location lookup table to find the storage location of the new record for the neighboring node, and then access the new record of the neighboring node based on that storage location.

[0141] In another embodiment of this disclosure, the method further includes:

[0142] In the case of concurrent retrieval operations, and the retrieval operation needs to read the record corresponding to the neighbor node, before the retrieval operation reads the record corresponding to the neighbor node, it is determined whether the record corresponding to the neighbor node is in a locked state;

[0143] When the record corresponding to the neighbor node is in a locked state, after waiting for the record corresponding to the neighbor node to be unlocked, the record corresponding to the neighbor node is locked, and after reading the record corresponding to the neighbor node, the record corresponding to the neighbor node is unlocked.

[0144] Thus, the retrieval operation also locks and unlocks the nodes it accesses at the record granularity. Before accessing a node, it checks whether the node is locked by an insertion operation. If the node is locked by an insertion operation, it must wait for the insertion operation to unlock the node before accessing the corresponding record. If the node is not locked by an insertion operation, the locking operation is performed on the node first. At this time, if concurrent insertion operations are performed on the same node, since the node is in a locked state, the insertion operation must wait for the node to unlock before proceeding with the subsequent insertion process. After the retrieval operation locks a node, it reads the corresponding record and generates a snapshot based on the read record, which is then stored in memory. After reading the corresponding record, it performs an unlock operation on the node.

[0145] In this way, by performing approximate concurrency control at the record granularity, that is, regardless of whether it is an insertion operation or a retrieval operation, the memory access process of the record corresponding to the vector index is locked only at the record granularity, thereby reducing the mutual influence between insertion and retrieval operations and improving the efficiency of insertion and retrieval operations.

[0146] like Figure 3 The image shows a specific example of updating a vector index.

[0147] like Figure 3 As shown in (a), in this example, the nine nodes in the graph-based vector index are: nodes 0 to 8. Records corresponding to nodes 0 and 4 are stored in disk page Page0, records corresponding to nodes 1, 2, 3, and 5 are stored in disk page Page1, and records corresponding to nodes 6, 7, and 8 are stored in disk page Page2. Each disk page includes five storage locations, capable of storing a maximum of five records. Taking Page0 as an example, the record corresponding to node 0 is stored in the first storage location of Page0; the record corresponding to node 4 is stored in the fifth storage location of Page0; the second, third, and fourth storage locations of Page0 are all writable, meaning there are three usable storage locations in Page0. The identifier corresponding to node 0 is represented as V0; the identifiers of neighboring nodes are represented as nbr IDs. Therefore, the record corresponding to node 0 is represented as V0|nbr IDs. Similarly, the record corresponding to node 5 is represented as V4|nbr IDs.

[0148] When inserting node 9 into the vector index, assuming node 9's neighboring nodes are nodes 5 and 6, and the preset threshold is 2, this means the target disk page must have more than 2 available storage locations, i.e., at least 3 available storage locations. Since node 9 has 2 neighboring nodes, the target disk page only needs 3 available storage locations to store the record corresponding to node 9, as well as the records corresponding to its neighbors nodes 5 and 6. Therefore, in this example, the determined target disk page is Page 0.

[0149] Then, based on the identifier of node 9 and the identifiers of neighboring nodes 5 and 6, a record corresponding to node 9 is generated and stored in the second storage location of Page 0.

[0150] Furthermore, based on the identifier of node 9, new records for neighboring node 5 and neighboring node 6 will be generated, and the new record for neighboring node 5 will be written to the third storage location of Page 0; the new record for neighboring node 6 will be written to the fourth storage location of Page 0. Meanwhile, since the original records for neighboring nodes 5 and 6 are still stored on Page 1 and Page 2 respectively, these original storage locations can be made available. Although the original records for nodes 5 and 6 are stored in these two locations, they are no longer valid, and these two storage locations can be used as available storage locations for writing other records.

[0151] To ensure that the record of any node can be read from the correct location, an ID-location lookup table, known as the "ID-location lookup table," can be maintained in memory to map the node's ID to its storage location on the disk page. This lookup table allows you to find the correct storage location of any node on the disk page.

[0152] Corresponding to the embodiments of the aforementioned vector index update method, this application also provides embodiments of a vector index update apparatus.

[0153] Embodiments of the vector index updating device of this application can be applied to computer devices. The device embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of the computer device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 5 The diagram shown is a hardware structure diagram of a computer device containing the vector index updating device of this application, except... Figure 5In addition to the processor, memory, network interface, and non-volatile memory shown, the computer device in which the device is located in the embodiment may also include other hardware depending on the actual function of the updating device of the vector index, which will not be described in detail here.

[0154] Please refer to Figure 4 The vector index updating apparatus provided in this embodiment includes:

[0155] The first determining module 41 is used to determine the neighboring nodes corresponding to the target vector from the vector index according to the target vector to be merged into the vector index; wherein the vector similarity between the target vector and the neighboring nodes is greater than or equal to a preset similarity threshold.

[0156] The second determining module 42 is used to determine the target disk page; wherein the number of available storage locations in the target disk page is greater than a preset number threshold.

[0157] The first update module 43 is configured to generate a record corresponding to the target vector based on the target vector and the corresponding neighbor nodes, and write the record to the target disk page; and,

[0158] The second update module 44 is used to update the records of the neighboring nodes based on the target vector, obtain the new records corresponding to the neighboring nodes, and write the new records corresponding to the neighboring nodes to the target disk page.

[0159] Optionally, the second determining module 42, when determining the target disk page to be inserted into the target vector, performs at least one of the following:

[0160] The target disk page is determined from the list of blank disk pages;

[0161] The disk pages corresponding to each node in the search path when determining neighboring nodes are used as candidate disk pages, and the target disk page is determined from the candidate disk pages.

[0162] Allocate a new blank page on the disk as the target disk page.

[0163] The second update module 44, when updating the records of the neighboring nodes based on the target vector to obtain the new records corresponding to the neighboring nodes, is used to:

[0164] If the number of outgoing neighbors of a neighboring node has not reached the maximum outgoing degree, the target vector is used as a new outgoing neighbor of the neighboring node, the identifier of the target vector is added to the record corresponding to the neighboring node, and a new record corresponding to the neighboring node is generated.

[0165] When the number of outgoing neighbors of the neighbor node reaches the maximum outgoing degree, the target vector is used as an increment to perform an incremental triangle inequality judgment on the target vector and the original outgoing neighbors of the neighbor node, so as to determine the outgoing neighbors to be deleted from the original outgoing neighbors of the neighbor node.

[0166] The outgoing neighbor to be deleted is removed from the record corresponding to the neighbor node, and the target vector is used as the new outgoing neighbor of the neighbor node. The identifier of the target vector is added to the record corresponding to the neighbor node, and a new record corresponding to the neighbor node is generated.

[0167] Optionally, the second update module 44 is also used for:

[0168] Change the storage location corresponding to the original record of the neighboring node to an available state; the storage location in an available state can be written with other records.

[0169] Optionally, after changing the storage location corresponding to the original record of the neighboring node to an available state, the second update module is further configured to:

[0170] Determine whether all storage locations in the disk page where the original record of the neighboring node is located are in an available state;

[0171] If all storage locations in the disk page containing the original record of the neighboring node are available, add the disk page containing the original record of the neighboring node to the list of blank disk pages.

[0172] Optionally, the second update module 44 is also used for:

[0173] In the storage location lookup table, the storage location of the record corresponding to the neighbor node is updated to the storage location of the new record corresponding to the neighbor node in the target disk page;

[0174] The storage location lookup table is used to store the correspondence between each node in the vector index and its storage location on the disk.

[0175] Optionally, before updating the records of the neighboring nodes based on the target vector, the second update module 44 is further configured to:

[0176] The original record of the neighboring node is read, and a locking operation is performed on the original record corresponding to the neighboring node so that the original record corresponding to the neighboring node enters a locked state; in the locked state, the original record corresponding to the neighboring node cannot be accessed by the retrieval operation;

[0177] The second update module, after writing the new record corresponding to the neighbor node to the target disk page, is further configured to:

[0178] Perform an unlock operation on the original record corresponding to the neighbor node.

[0179] Optionally, it also includes: a processing module 45, used for:

[0180] In the case of concurrent retrieval operations, and the retrieval operation needs to read the record corresponding to the neighbor node, before the retrieval operation reads the record corresponding to the neighbor node, it is determined whether the record corresponding to the neighbor node is in a locked state;

[0181] When the record corresponding to the neighbor node is in a locked state, after waiting for the record corresponding to the neighbor node to be unlocked, the record corresponding to the neighbor node is locked, and after reading the record corresponding to the neighbor node, the record corresponding to the neighbor node is unlocked.

[0182] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0183] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0184] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0185] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0186] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0187] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0188] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0189] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0190] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0191] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for updating a vector index, characterized in that, The method includes: Based on the target vector to be merged into the vector index, the neighboring nodes corresponding to the target vector are determined from the vector index; wherein the vector similarity between the target vector and the neighboring nodes is greater than or equal to a preset similarity threshold; Identify the target disk page; wherein the number of available storage locations in the target disk page is greater than a preset threshold. Based on the target vector and its corresponding neighbor nodes, a record corresponding to the target vector is generated, and the record is written to the target disk page; and... Based on the target vector, the records of the neighboring nodes are updated to obtain the new records corresponding to the neighboring nodes, and the new records corresponding to the neighboring nodes are written to the target disk page.

2. The method according to claim 1, characterized in that, Determining the target disk page to which the target vector is inserted includes at least one of the following: The target disk page is determined from the list of blank disk pages; The disk pages corresponding to each node in the search path when determining neighboring nodes are used as candidate disk pages, and the target disk page is determined from the candidate disk pages. Allocate a new blank page on the disk as the target disk page.

3. The method according to claim 1, characterized in that, The step of updating the records of the neighboring nodes based on the target vector to obtain the new records corresponding to the neighboring nodes includes: If the number of outgoing neighbors of a neighboring node has not reached the maximum outgoing degree, the target vector is used as a new outgoing neighbor of the neighboring node, the identifier of the target vector is added to the record corresponding to the neighboring node, and a new record corresponding to the neighboring node is generated. When the number of outgoing neighbors of the neighbor node reaches the maximum outgoing degree, the target vector is used as an increment to perform an incremental triangle inequality judgment on the target vector and the original outgoing neighbors of the neighbor node, so as to determine the outgoing neighbors to be deleted from the original outgoing neighbors of the neighbor node. The outgoing neighbor to be deleted is removed from the record corresponding to the neighbor node, and the target vector is used as the new outgoing neighbor of the neighbor node. The identifier of the target vector is added to the record corresponding to the neighbor node, and a new record corresponding to the neighbor node is generated.

4. The method according to claim 1, characterized in that, The method further includes: Change the storage location corresponding to the original record of the neighboring node to an available state; the storage location in an available state can be written with other records.

5. The method according to claim 4, characterized in that, After changing the storage location corresponding to the original record of the neighboring node to an available state, the method further includes: Determine whether all storage locations in the disk page where the original record of the neighboring node is located are in an available state; If all storage locations in the disk page containing the original record of the neighboring node are available, add the disk page containing the original record of the neighboring node to the list of blank disk pages.

6. The method according to claim 1, characterized in that, The method further includes: In the storage location lookup table, the storage location of the record corresponding to the neighbor node is updated to the storage location of the new record corresponding to the neighbor node in the target disk page; The storage location lookup table is used to store the correspondence between each node in the vector index and its storage location on the disk.

7. The method according to any one of claims 1-5, characterized in that, Before updating the records of the neighboring nodes based on the target vector, the process further includes: The original record of the neighboring node is read, and a locking operation is performed on the original record corresponding to the neighboring node so that the original record corresponding to the neighboring node enters a locked state; in the locked state, the original record corresponding to the neighboring node cannot be accessed by the retrieval operation; After writing the new record corresponding to the neighbor node to the target disk page, the method further includes: Perform an unlock operation on the original record corresponding to the neighbor node.

8. The method according to claim 7, characterized in that, The method further includes: In the case of concurrent retrieval operations, and the retrieval operation needs to read the record corresponding to the neighbor node, before the retrieval operation reads the record corresponding to the neighbor node, it is determined whether the record corresponding to the neighbor node is in a locked state; When the record corresponding to the neighbor node is in a locked state, after waiting for the record corresponding to the neighbor node to be unlocked, the record corresponding to the neighbor node is locked, and after reading the record corresponding to the neighbor node, the record corresponding to the neighbor node is unlocked.

9. A vector index updating device, characterized in that, The device includes: The first determining module is used to determine the neighboring nodes corresponding to the target vector from the vector index based on the target vector to be merged into the vector index; wherein the vector similarity between the target vector and the neighboring nodes is greater than or equal to a preset similarity threshold. The second determining module is used to determine the target disk page; wherein the number of available storage locations in the target disk page is greater than a preset number threshold. The first update module is configured to generate a record corresponding to the target vector based on the target vector and the corresponding neighbor nodes, and write the record to the target disk page; and, The second update module is used to update the records of the neighboring nodes based on the target vector, obtain the new records corresponding to the neighboring nodes, and write the new records corresponding to the neighboring nodes to the target disk page.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-8.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it performs the following steps: based on the target vector to be merged into the vector index, it determines the neighboring nodes corresponding to the target vector from the vector index; wherein the vector similarity between the target vector and the neighboring nodes is greater than or equal to a preset similarity threshold. Identify the target disk page; wherein the number of available storage locations in the target disk page is greater than a preset threshold. Based on the target vector and its corresponding neighbor nodes, a record corresponding to the target vector is generated, and the record is written to the target disk page; and... Based on the target vector, the records of the neighboring nodes are updated to obtain the new records corresponding to the neighboring nodes, and the new records corresponding to the neighboring nodes are written to the target disk page.

Images