Vector index construction method, apparatus, and storage medium

Abstract

The application relates to a vector index construction method, a device and a storage medium, wherein the vector index construction method comprises the following steps: setting a highest level of a graph structure and a maximum number M of neighbors of a vector node, and configuring a corresponding vector set for each level of the graph structure; obtaining a to-be-processed vector; determining a target level and a near-neighbor vector list of the to-be-processed vector in the graph structure; in the near-neighbor vector list, if the number of neighbors of any near-neighbor vector exceeds M, it is judged whether there is an isolated vector in the neighbors of the near-neighbor vector; if there is an isolated vector in the neighbors of the near-neighbor vector, the isolated vector is reserved, and the neighbor farthest from the near-neighbor vector is deleted from the neighbors of the near-neighbor vector except the isolated vector; and the to-be-processed vector is inserted into the vector set corresponding to each level of the graph structure from the zeroth level to the target level. Through the application, the problem that an isolated vector is difficult to search is solved.

Description

Technical Field

[0001] This application relates to the field of vector databases, and in particular to methods, apparatus and storage media for constructing vector indexes. Background Technology

[0002] With the rapid development of artificial intelligence and big data technologies, the scale of unstructured data such as images, text, and audio is growing exponentially. This type of data typically needs to be converted into high-dimensional vector form for storage and similarity retrieval. Vector databases have thus become a core technology carrier supporting intelligent retrieval, recommendation systems, computer vision, and other scenarios. In the field of vector databases, the quality of vector index construction directly determines the efficiency and accuracy of similarity retrieval. In related technologies, some vectors, due to distribution issues, are far removed from other vectors. When constructing the index structure, these vectors may become isolated vectors due to parameter issues, lacking any association with other vectors. In similarity search, isolated vectors are difficult to find.

[0003] Currently, no effective solution has been proposed to address the problem that isolated vectors are difficult to find in related technologies. Summary of the Invention

[0004] This application provides a vector index construction method, apparatus, and storage medium to at least solve the problem that isolated vectors are difficult to search in related technologies.

[0005] In a first aspect, embodiments of this application provide a vector index construction method, the method comprising:

[0006] Define the highest level of the graph structure and the maximum number of neighbors M of the vector nodes in the graph structure, and configure the corresponding vector set for each level of the graph structure; where M is a positive integer;

[0007] Obtain the vector to be processed;

[0008] In the graph structure, the target level of the vector to be processed and the list of nearest neighbor vectors of the vector to be processed are determined; wherein, the nearest neighbor vectors in the list of nearest neighbor vectors are the neighbors of the vector to be processed;

[0009] In the list of nearest neighbor vectors, if any of the nearest neighbor vectors has more than M neighbors, then it is determined whether there is an isolated vector among the neighbors of the nearest neighbor vector; the isolated vector is a vector that has only one neighbor.

[0010] If there is an isolated vector among the neighbors of the nearest neighbor vector, then the isolated vector is retained, and the neighbor that is farthest from the nearest neighbor vector, excluding the isolated vector, is deleted from the neighbors of the nearest neighbor vector.

[0011] The vector to be processed is inserted into the vector set corresponding to each level of the graph structure from level zero to the target level.

[0012] In some embodiments, determining the target level of the vector to be processed and the list of nearest neighbor vectors of the vector to be processed in the graph structure includes:

[0013] In each level of the graph structure, the target level of the vector to be processed is determined;

[0014] In the vector nodes of the graph structure, a list of nearest neighbor vectors of the vector to be processed is determined.

[0015] In some embodiments, determining the target level of the vector to be processed in each level of the graph structure includes:

[0016] Generate zero-level random numbers starting from the zeroth level of the graph structure;

[0017] When the zero-level random number is greater than a preset threshold, the vector to be processed is promoted from the zero-level to the next higher level until the random number generated by the target level is less than or equal to the preset threshold. Then, the target level is used as the level of the vector to be processed.

[0018] In some embodiments, determining the list of nearest neighbor vectors of the vector to be processed among the vector nodes in the graph structure includes:

[0019] Starting from the highest level, the distance between the vector to be processed and each vector in the graph structure is calculated layer by layer downwards. The M nearest vectors are saved to the nearest neighbor vector list, and the vectors in the nearest neighbor vector list are sorted according to the distance.

[0020] In some embodiments, inserting the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level includes:

[0021] The vector to be processed is inserted into the vector set corresponding to each level of the graph structure from level zero to the target level. The nearest neighbor vectors of the vector to be processed are saved to the nearest neighbor vector list, and the vector to be processed is added to the nearest neighbor vector list corresponding to each of the nearest neighbor vectors.

[0022] In some embodiments, if any nearest neighbor vector in the nearest neighbor vector list has more than M neighbors, then it is first determined whether there are isolated vectors among the neighbors of the nearest neighbor vector, including:

[0023] If there is no isolated vector among the neighbors of the nearest neighbor vector, then delete the neighbor that is farthest from the nearest neighbor vector.

[0024] If all neighbors of the nearest neighbor vector are isolated vectors, then all neighbors of the nearest neighbor vector are retained.

[0025] In some embodiments, after the vector index is constructed, the method further includes:

[0026] Obtain the vector to be searched;

[0027] Starting from the highest level down to the zeroth level, search layer by layer for the K target vector sets that are closest to the vector to be searched; where K is a positive integer;

[0028] Based on the target vector set, the search results are obtained.

[0029] In some embodiments, the step of searching layer by layer from the highest level down to the zeroth level for the K nearest target vectors to the vector to be searched includes:

[0030] Starting the search from the highest level, calculate the distance between the vector to be searched and each vector in the vector set of the highest level;

[0031] In the highest level, find the highest level target vector that is closest to the vector to be searched, and put the highest level target vector into a preset candidate list; wherein, the candidate list dynamically stores the top K vectors that are closest to the vector to be searched;

[0032] The search proceeds from the highest-level target vector down to the next lower level. In the vector set of the current next lower level, the neighbors of the highest-level target vector are determined and added to the candidate list.

[0033] Calculate the distances between the neighbors of the vector to be searched and the highest-level target vector respectively. Among the neighbors of the highest-level target vector, find the second-highest-level target vector that is closest to the vector to be searched, and continue searching from the second-highest-level target vector to the next lower level until the zeroth level.

[0034] The candidate vectors in the candidate list are used as the target vector set.

[0035] Secondly, embodiments of this application provide a vector index construction apparatus, the apparatus comprising:

[0036] An initialization module is used to set the highest level of the graph structure and the maximum number of neighbors M of the vector nodes in the graph structure, and to configure the corresponding vector set for each level of the graph structure; where M is a positive integer;

[0037] The module for obtaining the vector to be processed is used to obtain the vector to be processed.

[0038] The target level and nearest neighbor list determination module is used to determine the target level of the vector to be processed and the nearest neighbor vector list of the vector to be processed in the graph structure; wherein, the nearest neighbor vectors in the nearest neighbor vector list are the neighbors of the vector to be processed.

[0039] An isolated vector determination module is used to determine whether there is an isolated vector among the neighbors of any of the nearest neighbor vectors if the number of neighbors of any of the nearest neighbor vectors exceeds M; the isolated vector is a vector that has only one neighbor.

[0040] The nearest neighbor vector processing module is used to retain the isolated vector if there is an isolated vector among the neighbors of the nearest neighbor vector, and delete the neighbor that is farthest from the nearest neighbor vector, excluding the isolated vector;

[0041] The vector insertion module is used to insert the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level.

[0042] Thirdly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the vector index construction method as described in the first aspect above.

[0043] Compared to related technologies, the vector index construction method, apparatus, and storage medium provided in this application solve the problem that isolated vectors are difficult to find by setting the highest level of the graph structure and the maximum number of neighbors M of the vector nodes in the graph structure, and configuring corresponding vector sets for each level of the graph structure; where M is a positive integer; obtaining the vector to be processed; determining the target level of the vector to be processed and the list of nearest neighbor vectors of the vector to be processed in the graph structure; where the nearest neighbor vectors in the list of nearest neighbor vectors are the neighbors of the vector to be processed; if there is any nearest neighbor vector whose number of neighbors exceeds M, it is determined whether there is an isolated vector among the neighbors of the nearest neighbor vector; an isolated vector is a vector with only one neighbor; if there is an isolated vector among the neighbors of the nearest neighbor vector, the isolated vector is retained, and the neighbor farthest from the nearest neighbor vector is deleted from the neighbors of the nearest neighbor vector, excluding the isolated vector; inserting the vector to be processed into the vector sets corresponding to each level of the graph structure from the zero level to the target level.

[0044] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0045] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 This is a hardware structure block diagram of a terminal for a vector index construction method according to an embodiment of this application;

[0047] Figure 2 This is a flowchart of a vector index construction method according to an embodiment of this application;

[0048] Figure 3 This is an overall flowchart of a vector index construction method according to an embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the state of vector data after a vector index has been constructed according to an embodiment of this application;

[0050] Figure 5 This is a flowchart of a vector search method according to an embodiment of this application;

[0051] Figure 6 This is a structural block diagram of a vector index construction device according to an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0053] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0054] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0055] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. Taking running on a terminal as an example, Figure 1 This is a hardware structure block diagram of a terminal for a vector index construction method according to an embodiment of this application. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1The different configurations shown.

[0056] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the vector index construction method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0057] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0058] For ease of understanding, exemplary descriptions of concepts related to this application are provided for reference.

[0059] A vector is a mathematical object used to represent data such as text, images, and audio. These vectors are typically represented in a high-dimensional space and can capture the features and semantic information of the data. For example, a text can be represented by a high-dimensional vector, where each dimension corresponds to a feature or attribute (such as word frequency, semantics, etc.).

[0060] Vector search is an information retrieval technique based on a vector space model, primarily used for processing and querying high-dimensional data. Its core idea is to convert data (such as text, images, and audio) into vector representations, and then search and match within this vector space. The similarity between vectors is typically measured by calculating their distance or similarity; commonly used metrics include Euclidean distance, cosine similarity, and Manhattan distance.

[0061] This embodiment provides a method for constructing a vector index. Figure 2 This is a flowchart of a vector index construction method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0062] Step S201: Set the highest level of the graph structure and the maximum number of neighbors M of the vector nodes in the graph structure, and configure the corresponding vector set for each level of the graph structure; where M is a positive integer.

[0063] Specifically, in the initial stage of vector index construction, the basic parameters and hierarchical configuration of the graph structure must be completed first. First, the highest level of the hierarchical graph structure must be clearly defined (i.e., the top-level Layer value, used for the hierarchical distribution and fast navigation of subsequent vector nodes). Simultaneously, the maximum number of neighbors M that each vector node in the graph can be associated with is set (M is a positive integer, and its value needs to be determined based on the actual data scale and search accuracy requirements; for example, for a million-level vector dataset, M can be set to 4-16 to balance index efficiency and association integrity). Based on this, an independent vector set is configured for each level of the graph structure (from level zero to the preset highest level). Each level's vector set will specifically store the vector nodes assigned to that level, and vector sets from different levels form cross-level navigation paths through neighbor associations between nodes. This provides structured support for the hierarchical positioning, nearest neighbor search, and index insertion of the vectors to be processed, ensuring that the entire vector index has clear hierarchical logic and efficient node access capabilities.

[0064] Step S202: Obtain the vector to be processed.

[0065] Specifically, vector data not yet included in the index graph structure is extracted from the vector dataset. These vectors to be processed can be either raw high-dimensional data entering the vector database for the first time, or incremental vector data subsequently added and updated to the current graph structure. During the acquisition process, it is necessary to ensure that the dimensions of the vector data are consistent with the dimensions of the vectors already stored in the current index graph structure to meet the basic conditions for subsequent distance calculations. The current graph structure is constructed by first setting basic parameters such as the highest level and the maximum number of neighbors M of the vector nodes, and then iteratively processing all the vectors to be processed in the initial batch.

[0066] Step S203: In the graph structure, determine the target level of the vector to be processed and the list of nearest neighbor vectors of the vector to be processed; wherein, the nearest neighbor vectors in the list of nearest neighbor vectors are the neighbors of the vector to be processed.

[0067] Specifically, to determine the target level, we need to start from the zeroth level of the graph structure and generate zero-level random numbers for the vector to be processed. If the random number is greater than a preset threshold (such as 0.5), the vector to be processed is promoted from the current level to the next higher level, and random numbers are regenerated at the new level for judgment. This promotion process is repeated until the random number generated at a certain level is less than or equal to the preset threshold. At this time, the level is the target level of the vector to be processed, and the vector to be processed will be inserted into the set of all vectors from the zeroth level to the target level.

[0068] To determine the nearest neighbor vector list, we need to start from the highest level of the graph structure and calculate the distance between the vector to be processed and each vector in the current level's vector set, layer by layer downwards (the distance calculation method can be a common method such as L2 distance or cosine distance, which the user can choose in advance when building the index). At each level, we select the M vectors that are closest to the vector to be processed (if the number of vectors in the level is less than M, then all vectors are included in the candidate range). At the same time, we maintain the list of visited vectors to avoid duplicate calculations. Finally, we integrate and sort the candidate vectors selected from each level to form a nearest neighbor vector list containing the M vectors closest to the vector to be processed. Each nearest neighbor vector in the list is a neighbor of the vector to be processed. We will then use this list to further determine whether there are nearest neighbor vectors with more than M neighbors, providing a basis for subsequent isolated vector judgment and nearest neighbor vector deletion operations.

[0069] Step S204: In the list of nearest neighbor vectors, if there exists a nearest neighbor vector whose number of neighbors exceeds M, then determine whether there is an isolated vector among the neighbors of that nearest neighbor vector; an isolated vector is a vector that has only one neighbor.

[0070] Specifically, after obtaining the list of nearest neighbor vectors for the vector to be processed, the number of neighbors for each nearest neighbor vector in the list needs to be checked first. If it is found that the current number of neighbors of any nearest neighbor vector (such as vector b) exceeds the pre-set maximum number of neighbors M for the vector node, then all neighbors of vector b need to be further analyzed. The core is to determine whether there are any isolated vectors among these neighbors. An isolated vector is a vector that has only one associated neighbor in the entire graph structure, that is, the neighbor list of the vector contains only vector b, and there are no other associated vector nodes. The key significance of this judgment step is that it avoids the problem of isolated vectors being mistakenly deleted due to simply deleting neighbors by distance in conventional algorithms, thus becoming "isolated nodes" that are difficult to locate in subsequent searches. Since the association of isolated vectors is extremely simple, if vector b is deleted because the number of neighbors exceeds the limit, its unique association with the graph structure will be directly broken, ultimately resulting in missing search results. Therefore, by prioritizing the judgment and identification of isolated vectors, while maintaining the basic constraint of M on the number of neighbors to ensure indexing efficiency, special protection is provided for isolated vectors, laying the foundation for subsequent operations of retaining isolated vectors and only deleting non-isolated distant neighbors, ensuring that the index structure is both efficient and complete.

[0071] Step S205: If there is an isolated vector among the neighbors of the nearest neighbor vector, then retain the isolated vector and delete the neighbor that is farthest from the nearest neighbor vector, excluding the isolated vector.

[0072] Specifically, when preliminary checks reveal that the number of neighbors of a nearby vector (such as vector b) in the nearest neighbor vector list exceeds the preset maximum number of neighbors M for a vector node, and it is further determined that there are isolated vectors among vector b's neighbors, the following operations must be performed: prioritizing the retention of isolated vectors and selectively deleting non-isolated distant neighbors. On the one hand, all isolated vectors that meet the definition must be strictly retained because isolated vectors have extremely simple relationships. If they are mistakenly deleted due to the excessive number of neighbors of vector b, their unique connection with the graph structure will be broken, making it impossible to locate the vector through hierarchical navigation or neighbor traversal in subsequent searches, resulting in missing search results. On the other hand, while retaining isolated vectors, other non-isolated neighbors besides isolated vectors are selected from vector b's neighbors. By calculating the distance between these non-isolated neighbors and vector b, the objects to be deleted are determined in order of distance from farthest to closest. Finally, the non-isolated neighbor with the farthest distance is deleted. This controls the number of neighbors of vector b, ensures index efficiency, and prevents isolated vectors from becoming "isolated nodes," ensuring that the index structure meets both performance requirements and search integrity.

[0073] Step S206: Insert the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level.

[0074] After determining the target level and filtering the nearest neighbor vector list for the vector to be processed, the vector to be processed needs to be inserted into the vector sets corresponding to each level of the graph structure from level zero to the target level. Specifically, this involves not only adding the vector to be processed one by one to the vector set specific to each level (for example, if the target level of the vector to be processed is Layer 4, then the vector needs to be entered into the vector sets of Layer 0, Layer 1, Layer 2, Layer 3, and Layer 4 respectively), but also simultaneously saving the nearest neighbor vector list (i.e., the previously filtered M closest vectors), and adding the vector to be processed to each level's vector set. In the list of nearest neighbor vectors corresponding to each nearest neighbor vector, a bidirectional association is established. This method of cross-level insertion and bidirectional association records not only conforms to the hierarchical design logic of graph structures, where lower levels contain all vectors and higher levels contain some navigation vectors, but also ensures that during subsequent searches (searching for the nearest neighbors of the vector to be searched from the highest level down), the vector to be processed can be quickly located through the vector associations between levels. This avoids increasing the search path length due to the vector existing only at a single level. At the same time, combined with the protection mechanism for isolated vectors in the early stage, the accessibility of the vector to be processed and associated isolated vectors in subsequent searches is further guaranteed, ultimately achieving a balance between the integrity of the index structure and search efficiency.

[0075] Steps S201 to S206 above, by setting the highest level of the graph structure, the maximum number of neighbors M for vector nodes, and configuring corresponding vector sets for each level, obtain the vector to be processed, determine its target level and list of nearest neighbor vectors, then judge and protect isolated vectors for nearest neighbor vectors with more than M neighbors in the list, and selectively delete non-isolated distant neighbors, and finally insert the vector to be processed into each vector set from level zero to the target level, effectively solves the problem in related technologies where isolated vectors are difficult to search due to the special vector distribution or parameter limitations that cause the connection between isolated vectors and the graph structure to be broken. It ensures the efficiency of vector index construction through the constraint of the M value, and avoids isolated vectors from becoming "isolated nodes" by prioritizing the retention of isolated vectors. At the same time, by using the cross-level insertion of the vector to be processed and the bidirectional association with the nearest neighbor vectors, it ensures that the vector to be processed and the associated isolated vectors can be quickly located from the highest level down during subsequent searches. Without reducing the overall search efficiency, it achieves a balance between the integrity of the vector index structure and the accuracy of the search, and meets the requirements of efficient and complete similarity retrieval of high-dimensional vector datasets.

[0076] In some embodiments, within the graph structure, determining the target level of the vector to be processed and the list of nearest neighbor vectors of the vector to be processed includes:

[0077] Within each level of the graph structure, determine the target level of the vector to be processed;

[0078] In the vector nodes of the graph structure, determine the list of nearest neighbor vectors of the vector to be processed.

[0079] Specifically, the target level of the vector to be processed is determined in each level of the graph structure. This is achieved by generating zero-level random numbers starting from the zero level of the graph structure and setting a preset threshold (such as 0.5). If the zero-level random number is greater than the preset threshold, the vector to be processed is promoted from the zero level to the next higher level, and the random number generation process is repeated in the new level until the random number generated in a certain target level is less than or equal to the preset threshold. At this point, the level is determined as the target level of the vector to be processed, and the subsequent vectors to be processed will be inserted into the set of all vectors from the zero level to the target level.

[0080] To determine the nearest neighbor list of the vector to be processed among the vector nodes in the graph structure, we need to start from the highest level of the graph structure and calculate the distance between the vector to be processed and each vector node in the current level vector set layer by layer. At the same time, we maintain a list of visited vectors to avoid duplicate calculations. At each level, we select the M vectors that are closest to the vector to be processed (if the number of vector nodes in the level is less than M, then all vector nodes are included in the candidate range). Finally, we integrate the candidate vectors selected from each level and sort them from closest to furthest, forming a nearest neighbor list containing the M nearest neighbor vectors of the vector to be processed.

[0081] Through the above steps, on the one hand, the logic of generating random numbers from level zero and comparing them with a preset threshold until the random number is less than or equal to the threshold to determine the target level ensures that the vectors to be processed are reasonably distributed within the range from level zero to the target level. This conforms to the hierarchical design of the graph structure, where lower levels contain all vectors and higher levels contain navigation vectors. This lays the foundation for subsequent cross-level vector insertion and ensures that vectors can be quickly located layer by layer from the highest level during the search. On the other hand, the nearest neighbor vector list is determined by calculating the distance layer by layer from the highest level, selecting the M nearest vectors, and sorting them. This ensures the relevance and accuracy of the nearest neighbor vectors and provides accurate basic data for subsequent operations such as checking whether the number of neighbors of a nearest neighbor vector exceeds M and identifying and protecting isolated vectors. Ultimately, this achieves reasonable hierarchical positioning and efficient neighbor matching of the vectors to be processed in the graph structure, providing key support for the integrity of the vector index construction and the efficiency of subsequent searches.

[0082] In some embodiments, determining the target level of the vector to be processed within each level of the graph structure includes:

[0083] Generate zero-level random numbers starting from the zeroth level of the graph structure;

[0084] When the random number generated at level zero is greater than the preset threshold, the vector to be processed is promoted from level zero to the next level until the random number generated at the target level is less than or equal to the preset threshold. Then the target level is used as the level of the vector to be processed.

[0085] Specifically, the operation of determining the target level of the vector to be processed in each level of the graph structure must follow a random judgment logic from the bottom to the top. First, starting from the zeroth level of the graph structure, a zero-level random number is generated for the vector to be processed. This random number will serve as the core basis for judging whether the vector should be promoted to a higher level. At the same time, the zero-level random number is compared with a preset threshold (such as 0.5). If the zero-level random number is greater than the preset threshold, it indicates that the vector to be processed has the conditions to be promoted to a higher level, and it needs to be moved up from the current zeroth level to the level above the zeroth level (i.e., the first level). Subsequently, in the newly moved level (the first level), a random number corresponding to the level to be processed is generated again for the vector to be processed, and the operation of comparing the random number with the preset threshold is repeated. If the random number of the level is still greater than the preset threshold, the vector to be processed is promoted to the next level. This promotion process is executed cyclically until the random number generated for the vector to be processed in a certain target level is less than or equal to the preset threshold. At this time, the level promotion operation stops, and the target level where promotion stops is finally determined as the level of the vector to be processed in the graph structure.

[0086] The above steps, through randomized hierarchical allocation logic, ensure that the distribution of the vector to be processed in the hierarchy of the graph structure is both random to adapt to the characteristics of different vectors, and avoids infinitely increasing the hierarchy through threshold constraints. This provides a clear hierarchical range basis for subsequently inserting the vector to be processed into each vector set from the zeroth level to the target level.

[0087] In some embodiments, determining a list of nearest neighbor vectors for the vector to be processed within the vector nodes of the graph structure includes:

[0088] Starting from the highest level, calculate the distance between the vector to be processed and each vector in the graph structure layer by layer downwards, save the M nearest vectors to the nearest neighbor list, and sort the vectors in the nearest neighbor list according to the distance.

[0089] First, starting from the highest level of the graph structure, the distance between the vector to be processed and all vector nodes in the vector set of that highest level is calculated (the calculation method is such as L2 distance, cosine distance, etc., which is selected by the user when building the index). Simultaneously, to avoid redundant calculations and filtering, a list of visited vectors is maintained, recording the vector nodes whose distances have been calculated; these nodes will not be processed again during subsequent level searches. After filtering out the vectors closest to the vector to be processed at the highest level, the process continues down to the next level, repeating the above process of "calculating distance - filtering nearest neighbors - recording visited vectors," until the zeroth level of the graph structure is reached. During this process, the candidate nearest neighbor vectors filtered at each level are temporarily aggregated. Finally, the M vectors closest to the vector to be processed are selected from all candidate vectors at all levels (if the total number of candidate vectors at each level is less than M, all candidate vectors are selected) and saved to the nearest neighbor vector list. The vectors in the list are then sorted in order of distance from closest to furthest.

[0090] Through the above steps, on the one hand, starting the search from the highest level can quickly narrow down the nearest neighbor selection range of the vector to be processed by leveraging high-level vectors, avoiding brute-force distance calculation in the entire vector set, greatly reducing the redundancy of distance calculation and improving the efficiency of nearest neighbor vector selection. On the other hand, by traversing downwards layer by layer and continuously selecting the nearest vector, combined with the operation of sorting by distance, it can be ensured that the vectors in the final nearest neighbor vector list have the optimal correlation with the vector to be processed, and the list data is ordered, providing basic data for subsequent operations such as determining whether the number of neighbors of the nearest vector exceeds M, identifying and protecting isolated vectors, etc. It also lays the foundation for establishing stable neighbor associations after the vector to be processed is inserted into the graph structure. Finally, while ensuring the accuracy of nearest neighbor vector selection, the efficiency of index construction is also taken into account, which meets the needs of efficient indexing of high-dimensional vector datasets.

[0091] In some embodiments, inserting the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level includes:

[0092] Insert the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level, save the nearest neighbor vectors of the vector to be processed into the nearest neighbor vector list, and add the vector to be processed to the nearest neighbor vector list corresponding to each nearest neighbor vector.

[0093] Specifically, according to the determined target level range of the vectors to be processed, each vector needs to be inserted one by one into the dedicated vector set corresponding to each level from level zero to the target level in the graph structure (for example, if the target level of the vectors to be processed is Layer 4, then the vectors to be processed need to be added to the vector sets of Layer 0, Layer 1, Layer 2, Layer 3, and Layer 4 respectively). This ensures that the hierarchical distribution of the vectors to be processed in the graph structure conforms to the design logic that lower levels contain all relevant vectors and higher levels contain navigation vectors. Secondly, the nearest neighbor vectors of the previously determined vectors to be processed need to be saved to the nearest neighbor vector list simultaneously. This list records the M vectors closest to the vector to be processed, which is the core basis for maintaining the relationship between vectors. Finally, a bidirectional association mechanism needs to be established to add the vector to be processed to the nearest neighbor list of each nearest neighbor vector, so that the vector to be processed and the nearest neighbor vectors form a mutually related neighbor relationship. This operation method not only ensures the effective existence of the vector to be processed in the vector set at each level, but also ensures the integrity of the neighbor relationship between vectors through bidirectional association. This provides a basis for quickly finding the vector to be processed through the nearest neighbor vector in the subsequent search process (locating from the highest level down layer by layer), and also avoids the risk of vector isolation caused by unidirectional association.

[0094] Through the above steps, on the one hand, the vectors to be processed are inserted according to the range from level zero to the target level, ensuring that the vectors to be processed can be quickly located from the highest level down during subsequent searches. This conforms to the hierarchical design logic of the graph structure, where lower levels contain all relevant vectors and higher levels contain navigation vectors. On the other hand, the nearest neighbor vectors of the vectors to be processed are saved to a list and bidirectional associations are established (the vectors to be processed are added to the lists of each nearest neighbor vector). This provides complete association data support for subsequent checks on whether the number of neighbors of the nearest neighbor vectors exceeds M and for judging and protecting isolated vectors. It also ensures the association between the vectors to be processed and the nearest neighbor vectors, avoiding the problem of isolated vectors. Ultimately, while ensuring the integrity of the graph structure index, it lays the foundation for efficient and complete vector search in the future.

[0095] In some embodiments, in the nearest neighbor vector list, if any nearest neighbor vector has more than M neighbors, then it is first determined whether there are isolated vectors among the neighbors of the nearest neighbor vector, including:

[0096] If there is no isolated vector among the neighbors of the nearest neighbor vector, then delete the neighbor that is farthest from the nearest neighbor vector.

[0097] If all neighbors of a nearest neighbor vector are isolated vectors, then retain all neighbors of the nearest neighbor vector.

[0098] Specifically, if it is found that there are no isolated vectors among the neighbors of the nearest neighbor vector, that is, all neighbors have at least two or more associated vectors, then there is no need to consider the isolated vector protection problem. The neighbor that is farthest from the nearest neighbor vector will be directly selected from all the neighbors of the nearest neighbor vector and deleted to ensure that the number of neighbors of the nearest neighbor vector returns to M.

[0099] If all neighbors of a nearest neighbor vector are isolated vectors, meaning each neighbor has a unique association with that nearest neighbor vector, then to avoid deleting any neighbor and causing it to become an "isolated node" completely broken from the graph structure, the number of neighbors of that nearest neighbor vector will be specially allowed to exceed M. All isolated vectors will be retained as its neighbors. Through this differentiated processing, while maintaining the basic constraint of the value of M on the number of neighbors, the loss of isolated vectors is minimized, ensuring the accessibility of isolated vectors in subsequent searches.

[0100] Through the above steps, on the one hand, when there are no isolated vectors among the neighbors of a nearest neighbor vector, the furthest neighbor is deleted, and the number of neighbors of the nearest neighbor vector is normalized to M. This ensures the efficiency of vector index construction and subsequent search, and also ensures that the remaining neighbors have a stronger correlation with the nearest neighbor vector by deleting distant neighbors, maintaining the effectiveness of the association between vectors. On the other hand, when all neighbors of a nearest neighbor vector are isolated vectors, all isolated vectors are retained, breaking the absolute limit of the value of M. This prevents isolated vectors from being completely broken from the graph structure due to deletion, ensuring that these isolated vectors can be located through their association with the nearest neighbor vector during subsequent searches. Overall, this step achieves a precise balance between ensuring index efficiency and avoiding the loss of isolated vectors. It meets the performance requirements for indexing large-scale high-dimensional vector datasets and solves the pain point of isolated vectors being difficult to search in conventional algorithms.

[0101] To better understand this application, the following explanation is provided in conjunction with the overall flowchart of the vector index construction method. Figure 3 This is an overall flowchart of a vector index construction method according to an embodiment of this application, such as... Figure 3 As shown, the process includes the following steps:

[0102] Step S301, Start. Initiate the vector index construction process, receive the target vector to be processed, and serve as the starting node of the entire vector processing process, marking the imminent commencement of operations such as constructing the graph structure and processing vectors.

[0103] Step S302: Set the highest level of the graph structure and initialize the maximum number of neighbors M for each vector node.

[0104] Step S303: Determine if there are any unprocessed vectors. If so, proceed to the next processing step; otherwise, the process ends. This is used to iterate through all vectors to be processed.

[0105] Step S304: For the unprocessed vector 'a', select its level. For the unprocessed vector, determine its position in the graph structure hierarchy so that subsequent processing can be performed based on the level.

[0106] Step S305: Find the nearest neighbor vectors of the current vector a. Obtain other vectors that are close to vector a and have a correlation relationship, in preparation for constructing the neighbor relationship between vectors.

[0107] Step S306: Determine whether the number of all neighbors of any nearest neighbor vector (taking b as an example) of the current vector a exceeds M. If it exceeds M, proceed to S307 for further processing. If it does not exceed M, skip the relevant operations in S307. This is used to control the number of neighbors from exceeding the limit.

[0108] Step S307: Determine whether any of the nearest neighbor vectors c of vector b is only vector b. If yes, remove vectors that do not meet the above condition (i.e., vector c has other neighbors besides b) from the nearest neighbor vectors of b; if no, remove the farthest vector from the nearest neighbor vectors of b; if all neighbors of vector b are isolated vectors, do not delete the neighbors of vector b.

[0109] Step S308: Insert vector a into the graph, and simultaneously add a to all of a's neighbors. This completes the insertion of vector a into the graph structure and the bidirectional update of neighbor relationships, establishing the association between vectors.

[0110] Step S309, End. The process ends, marking the completion of the current vector processing flow (processing vector a and its related neighbors). If there are still unprocessed vectors, it will return to S303 to continue the loop.

[0111] Figure 4 This is a schematic diagram illustrating the state of vector data after a vector index has been constructed according to an embodiment of this application. For example... Figure 4 As shown, in the zeroth level of the graph structure, the vector topology consists of 9 vectors from vector a to vector i, where M=4. During the index construction process, when preparing to delete a vector from the nearest neighbors of e, it is found that e's nearest neighbors f and i are only e. Therefore, the associations between f and e, and between i and e, need to be preserved. This ensures that vector f can be found through the association from e to f during a search, and vector i can be found through the association from e to i during a search, rather than being an isolated vector that cannot be found.

[0112] In some embodiments, after the vector index is constructed, the following is also included:

[0113] Obtain the vector to be searched;

[0114] Starting from the highest level down to the zeroth level, search layer by layer for the K nearest target vector sets to the vector to be searched; where K is a positive integer;

[0115] The search results are obtained based on the target vector set.

[0116] First, the search vector to be searched is obtained. This vector must maintain the same dimension as the vector to be processed during the index building phase to ensure the effectiveness of the distance calculation. Second, the search operation is performed layer by layer from the highest level of the graph structure down to the zeroth level. The goal is to select the K nearest target vectors (K is a positive integer, set by the user according to the search requirements) to the search vector. The search starts from the highest level to quickly narrow down the scope by leveraging the navigation properties of higher-level vectors. At each level, the distance between the search vector and each vector in the current level's vector set is calculated (the distance calculation method is consistent with the index building phase, such as L2 distance, cosine distance, etc.), and the nearest vector is dynamically selected to be included in the candidate range until the zeroth level is reached. Finally, the selection results from each level are summarized to form a target vector set containing K vectors. Finally, based on the selected target vector set and combined with business requirements, the final search results are generated, completing the entire process from input of the search vector to output of the search results.

[0117] The above steps, which start the search from the highest level, can quickly narrow down the search scope by leveraging high-level vectors, avoiding brute-force distance calculations in the entire vector set. This significantly reduces computational redundancy during the search process and improves search efficiency. At the same time, the hierarchical navigation approach ensures the integrity of the search results. It can accurately locate the vector most similar to the vector to be searched, and it can also rely on the protection mechanism for isolated vectors built during the initial index construction to prevent isolated vectors from being excluded from the search results due to broken associations. Ultimately, while ensuring search efficiency, it achieves both accuracy and completeness of the search results.

[0118] In some embodiments, starting from the highest level down to the zeroth level, the search proceeds layer by layer to find the K nearest target vector sets to the vector to be searched, including:

[0119] Start the search from the highest level and calculate the distance between the vector to be searched and each vector in the highest level vector set;

[0120] At the highest level, find the highest-level target vector that is closest to the vector to be searched, and add the highest-level target vector to a preset candidate list; the candidate list dynamically stores the top K vectors that are closest to the vector to be searched.

[0121] The search proceeds from the highest-level target vector down to the next level. In the vector set of the current next level, the neighbors of the highest-level target vector are determined and added to the candidate list.

[0122] Calculate the distances between the neighbors of the vector to be searched and the highest-level target vector respectively. Among the neighbors of the highest-level target vector, find the second-highest-level target vector that is closest to the vector to be searched, and continue the search from the second-highest-level target vector to the next lower level until the zeroth level.

[0123] Use the candidate vectors in the candidate list as the target vector set.

[0124] First, the search begins at the highest level of the pre-defined graph structure. Using a pre-defined distance calculation method (such as L2 distance, cosine distance, etc., consistent with the index building stage to ensure a unified distance metric), the distance between the vector to be searched and every vector in the highest-level vector set is calculated, providing a quantitative basis for subsequent selection of nearest neighbor vectors. Next, within the highest-level vector set, based on the calculated distances, the vector closest to the vector to be searched is selected as the highest-level target vector and added to a pre-defined candidate list. This candidate list is dynamically updated, always storing only the top K vectors closest to the vector to be searched (K is a positive integer, set by the user according to search requirements, such as Top 5, Top 10, etc.). If a later added vector is closer, the farthest vector in the list is removed, ensuring high relevance among the vectors in the list. Finally, the highest-level target vector serves as the navigation... A node initiates a search to the next lower level. In the vector set of the current next lower level, it first determines all neighbor vectors of the target vector at the highest level (based on the bidirectional association between vectors established during the index building phase), and temporarily adds these neighbor vectors to the candidate list. At the same time, it triggers dynamic sorting and filtering of the candidate list, retaining the top K closest vectors. Then, it calculates the distance between the target vector and these neighbor vectors of the highest level target vector, and selects the vector closest to the target vector as the next higher level target vector. Using this next higher level target vector as the new navigation node, it continues to repeat the process of "determining neighbors - adding to the candidate list - calculating distance - filtering target vectors" to the next lower level until the search traverses to the zeroth level of the graph structure. Finally, after completing the search at all levels, the top K vectors dynamically retained in the candidate list that are closest to the target vector are the final target vector set.

[0125] The above steps involve starting from the highest level and calculating the distance between the target vector and each vector at that level. The nearest target vector at the highest level is then selected and stored in a candidate list of the top K nearest neighbors. The target vector is then used as a navigation tool to determine its neighbors at the next lower level and include them in the candidate list. After calculating the distance between the neighbors and the target vector, the next higher level target vector is selected and the search continues down to the zeroth level. Finally, the candidate list vectors are used as the target vector set. On the one hand, starting the search from the highest level can quickly narrow the search scope by leveraging high-level vectors, avoiding brute-force calculations on the entire vector set, significantly reducing computational redundancy in the search phase, and adapting to the retrieval efficiency requirements of large-scale high-dimensional vector datasets. On the other hand, by using navigation vectors to locate neighbors layer by layer and dynamically filtering the top K nearest neighbors from the candidate list, the logic ensures that the selected target vector is always the closest to the vector to be searched, guaranteeing the accuracy of the search results. Furthermore, relying on the bidirectional vector associations established during the index building phase, it ensures that isolated vectors protected in the early stages can be included in the search process through neighbor associations, preventing them from becoming "isolated nodes" and being unsearchable. Ultimately, while balancing search efficiency and result accuracy, it also achieves the completeness of the search results.

[0126] Figure 5 This is a flowchart of a vector search method according to an embodiment of this application, such as... Figure 5 As shown, the process includes the following steps:

[0127] Step S501: Start the search from the highest level and calculate which vector in the current level is closest to the vector to be searched.

[0128] Step S502: Search down one level from the nearest vector m1 to find the neighbor list of m1, calculate the distance between each neighbor and the vector to be searched, and find the nearest vector.

[0129] Step S503 continues until the number of nearest neighbor vectors of the vector to be searched in level zero reaches the required number.

[0130] This embodiment also provides a vector index construction apparatus for implementing the above embodiments and preferred embodiments, which will not be repeated hereafter. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0131] Figure 6 This is a structural block diagram of a vector index construction device according to an embodiment of this application, such as... Figure 6 As shown, the device includes:

[0132] Initialization module 61 is used to set the highest level of the graph structure and the maximum number of neighbors M of the vector nodes in the graph structure, and to configure the corresponding vector set for each level of the graph structure; where M is a positive integer;

[0133] The vector acquisition module 62 is used to acquire the vector to be processed.

[0134] The target level and nearest neighbor list determination module 63 is used to determine the target level and the nearest neighbor vector list of the vector to be processed in the graph structure; wherein, the nearest neighbor vectors in the nearest neighbor vector list are the neighbors of the vector to be processed.

[0135] The isolated vector judgment module 64 is used to determine whether there is an isolated vector among the neighbors of any neighbor vector in the nearest neighbor vector list if the number of neighbors of any nearest neighbor vector exceeds M; an isolated vector is a vector that has only one neighbor.

[0136] The nearest neighbor vector processing module 65 is used to retain the isolated vector if there is an isolated vector among the neighbors of the nearest neighbor vector, and delete the neighbor that is farthest from the nearest neighbor vector, excluding the isolated vector.

[0137] Vector insertion module 66 is used to insert the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level.

[0138] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination. Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0139] Furthermore, in conjunction with the vector index construction method in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the vector index construction methods in the above embodiments.

[0140] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0141] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0142] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for constructing a vector index, characterized in that, include: Define the highest level of the graph structure and the maximum number of neighbors M of the vector nodes in the graph structure, and configure the corresponding vector set for each level of the graph structure; where M is a positive integer; Obtain the vector to be processed; the vector is obtained by converting unstructured data, and the unstructured data includes images, text, or audio. In the graph structure, the target level of the vector to be processed and the list of nearest neighbor vectors of the vector to be processed are determined; the nearest neighbor vectors in the list of nearest neighbor vectors are the neighbors of the vector to be processed; wherein, determining the list of nearest neighbor vectors of the vector to be processed includes: starting from the highest level, calculating the distance between the vector to be processed and each vector in the graph structure layer by layer downwards, saving the M nearest vectors to the list of nearest neighbor vectors, and sorting the vectors in the list of nearest neighbor vectors according to the distance; In the list of nearest neighbor vectors, if any of the nearest neighbor vectors has more than M neighbors, then it is determined whether there is an isolated vector among the neighbors of the nearest neighbor vector; the isolated vector is a vector that has only one neighbor. If there is an isolated vector among the neighbors of the nearest neighbor vector, then the isolated vector is retained, and the neighbor that is farthest from the nearest neighbor vector, excluding the isolated vector, is deleted from the neighbors of the nearest neighbor vector. The vector to be processed is inserted into the vector set corresponding to each level of the graph structure from level zero to the target level, thus completing the vector index construction; Obtain the vector to be searched; Starting from the highest level down to the zeroth level, search layer by layer for the K target vector sets that are closest to the vector to be searched; where K is a positive integer; Based on the target vector set, the search results are obtained.

2. The vector index construction method according to claim 1, characterized in that, In the graph structure, determining the target level of the vector to be processed and the list of nearest neighbor vectors of the vector to be processed includes: In each level of the graph structure, the target level of the vector to be processed is determined; In the vector nodes of the graph structure, a list of nearest neighbor vectors of the vector to be processed is determined.

3. The vector index construction method according to claim 2, characterized in that, Determining the target level of the vector to be processed in each level of the graph structure includes: Generate zero-level random numbers starting from the zeroth level of the graph structure; When the zero-level random number is greater than a preset threshold, the vector to be processed is promoted from the zero-level to the next higher level until the random number generated by the target level is less than or equal to the preset threshold. Then, the target level is used as the level of the vector to be processed.

4. The vector index construction method according to claim 1, characterized in that, The step of inserting the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level includes: The vector to be processed is inserted into the vector set corresponding to each level of the graph structure from level zero to the target level. The nearest neighbor vectors of the vector to be processed are saved to the nearest neighbor vector list, and the vector to be processed is added to the nearest neighbor vector list corresponding to each of the nearest neighbor vectors.

5. The vector index construction method according to claim 1, characterized in that, In the nearest neighbor vector list, if any nearest neighbor vector has more than M neighbors, then it is first determined whether there are any isolated vectors among the neighbors of the nearest neighbor vector, including: If there is no isolated vector among the neighbors of the nearest neighbor vector, then delete the neighbor that is farthest from the nearest neighbor vector from the neighbors of the nearest neighbor vector; If all neighbors of the nearest neighbor vector are isolated vectors, then all neighbors of the nearest neighbor vector are retained.

6. The vector index construction method according to claim 1, characterized in that, The step of searching for the K nearest target vector sets to the vector to be searched, starting from the highest level down to the zeroth level, includes: Starting the search from the highest level, calculate the distance between the vector to be searched and each vector in the vector set of the highest level; In the highest level, find the highest level target vector that is closest to the vector to be searched, and put the highest level target vector into a preset candidate list; wherein, the candidate list dynamically stores the top K vectors that are closest to the vector to be searched; The search proceeds from the highest-level target vector down to the next lower level. In the vector set of the current next lower level, the neighbors of the highest-level target vector are determined and added to the candidate list. Calculate the distances between the neighbors of the vector to be searched and the highest-level target vector respectively. Among the neighbors of the highest-level target vector, find the second-highest-level target vector that is closest to the vector to be searched, and continue searching from the second-highest-level target vector to the next lower level until the zeroth level. The candidate vectors in the candidate list are used as the target vector set.

7. A vector index construction apparatus, characterized in that, The device includes: An initialization module is used to set the highest level of the graph structure and the maximum number of neighbors M of the vector nodes in the graph structure, and to configure the corresponding vector set for each level of the graph structure; where M is a positive integer; A vector acquisition module is used to acquire vectors to be processed; the vectors are obtained by converting unstructured data, and the unstructured data includes images, text, or audio. The target level and nearest neighbor list determination module is used to determine the target level of the vector to be processed and the nearest neighbor vector list of the vector to be processed in the graph structure; wherein, the nearest neighbor vectors in the nearest neighbor vector list are the neighbors of the vector to be processed. The target level and nearest neighbor list determination module is further configured to calculate the distance between the vector to be processed and each vector in the graph structure, starting from the highest level and proceeding downwards, and save the M closest vectors to the nearest neighbor vector list, and sort the vectors in the nearest neighbor vector list according to the distance. An isolated vector determination module is used to determine whether there is an isolated vector among the neighbors of any of the nearest neighbor vectors if the number of neighbors of any of the nearest neighbor vectors exceeds M; the isolated vector is a vector that has only one neighbor. The nearest neighbor vector processing module is used to retain the isolated vector if there is an isolated vector among the neighbors of the nearest neighbor vector, and delete the neighbor that is farthest from the nearest neighbor vector, excluding the isolated vector; A vector insertion module is used to insert the vector to be processed into the vector set corresponding to each level of the graph structure from level zero to the target level; The vector index construction device is used to obtain the vector to be searched; starting from the highest level down to the zeroth level, it searches layer by layer for the K target vector sets that are closest to the vector to be searched; where K is a positive integer; and obtains the search results based on the target vector sets.

8. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the vector index construction method according to any one of claims 1 to 6 at runtime.

Images