Cross-database approximate nearest neighbor search method and system and computing framework

Abstract

The invention provides a cross-database approximate nearest neighbor search method and system and a calculation framework, and the method comprises the following steps: constructing a graph index, storing the graph index in a relation table, and obtaining a graph index table; acquiring and storing a data set and a query set in a structured relation table form; the dismantling approximate nearest neighbor search process comprises a plurality of SQL operation stages including a candidate node screening stage, a neighbor expansion stage, a distance calculation stage, a result combination stage and a priority queue maintenance stage; and based on the graph index table, executing an iterative search process of the plurality of SQL operation stages on each query point in the query set, and finally outputting an approximate nearest neighbor search result of the query set. According to the method, the graph index is combined with the relational database, and the approximate nearest neighbor search is realized by adopting a plurality of SQL operation stages, so that the high-dimensional vector retrieval efficiency and the cross-database compatibility are remarkably improved, and the large-scale application of the vector data in a multi-element scene is promoted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, in particular to a cross-database approximate nearest neighbor search method, system and computing framework. BACKGROUND

[0002] With the vigorous development of large language models, artificial intelligence applications have experienced explosive growth, and vector data is the core of this transformation. Large models represent multi-modal data through embedding, capturing its semantics in high-dimensional numerical vectors. Due to the high computational cost of high-dimensional vectors, the current mainstream technology for vector data retrieval is index-based approximate nearest neighbor search (ANNS), which avoids full data computation by constructing an index structure to achieve efficient query of approximate nearest neighbor nodes. In existing research, there are four types of vector indexes: tree-based indexes, locality-sensitive hashing-based indexes, quantization-based indexes, and graph-based indexes. Graph-based indexes have outstanding performance for different application scenarios and have become a commonly used index type in modern vector database systems.

[0003] How to design a database management system that adapts to high-dimensional vector data has become a research focus in the scientific and industrial communities in recent years. Existing vector database systems can be broadly divided into two categories: specialized vector databases and generalized vector databases. The former is designed from scratch for large-scale vector data storage and search, while the latter aims to support vector data management in existing databases, especially relational databases. Its core advantages include: unified storage, eliminating data silos, end-to-end processing capabilities, powerful query language capabilities, system reliability and long-term stability, lower learning costs, better scalability, and a more mature ecosystem.

[0004] However, existing generalized vector databases often require customized extension development based on the underlying implementation of the database system, creating vector index types through underlying interfaces, and establishing index construction and search methods. For different database systems, building such vector search extensions requires engineering teams to spend a lot of time and cost to understand the system architecture and internal APIs to construct functional interfaces. At the same time, the rapid iteration of cutting-edge index technologies and optimization methods for different scenarios further exacerbates the complexity and overhead of system extension and maintenance. SUMMARY

[0005] The present application provides a cross-database approximate nearest neighbor search method, system and computing framework, which can solve the problem of high engineering cost of vector function construction in the existing generalized vector database construction method, and overcome the limitation of not being able to quickly integrate cutting-edge index technologies.

[0006] In a first aspect, the present application provides a cross-database approximate nearest neighbor search method, comprising the following steps: obtaining and storing a dataset and a query set in the form of a structured relational table; constructing a graph index and storing it in the relational table to obtain a graph index table; decomposing an approximate nearest neighbor search process into multiple SQL operation stages including a candidate node screening stage, a neighbor expansion stage, a distance calculation stage, a result merging stage, and a priority queue maintenance stage; based on the graph index table, performing an iterative search process of the multiple SQL operation stages for each query point in the query set, and finally outputting an approximate nearest neighbor search result of the query set.

[0007] Further, the step of constructing a graph index and storing it in a relational table to obtain a graph index table specifically comprises the following steps: preloading an existing graph index into a relational table to obtain a graph index table, or calling an external graph index algorithm through a user-defined table generation function, constructing a relational table in the form of an adjacency list based on vector data in the dataset, and obtaining a graph index table.

[0008] Further, the step of calling an external graph index algorithm through a user-defined table generation function, constructing a relational table in the form of an adjacency list based on vector data in the dataset, and obtaining a graph index table specifically comprises the following steps: calling an external graph index algorithm to process vector data in the dataset to generate an initial graph structure; converting the initial graph structure into an adjacency list form containing a nid primary key field and a nbs neighbor array field, and storing it as a relational table to obtain a graph index table.

[0009] Further, the candidate node screening stage performs a set difference operation with a priority queue table and a visited set table to select all nodes in the priority queue that have not been visited, and for the selected nodes in the priority queue that have not been visited, according to the query node grouping, finding the node with the smallest distance to the corresponding query node as a candidate node table for each group.

[0010] Further, the neighbor expansion stage connects the candidate node table obtained in the previous stage with the graph index table to obtain a corresponding neighbor list array for each candidate node, and decouples the neighbor list array to convert it into a neighbor edge table.

[0011] Further, the distance calculation stage connects the node data table, the query set, and the neighbor edge table to calculate a distance result set between the vector of the candidate node neighbor and the vector of the query point.

[0012] Further, the result merging stage merges the candidate node set and the distance result set into the visited set and the priority queue respectively, and obtains the updated visited set and the priority queue respectively after deduplication processing.

[0013] Further, the priority queue maintenance stage groups the updated priority queue according to the bid field and applies a TOP-K window function, retains the specified data record with the minimum dist in each group, and obtains the candidate node set for the next round of iteration.

[0014] In a second aspect, the present application provides a cross-database approximate nearest neighbor search system, comprising: a data set and query set storage module configured to obtain and store the data set and the query set in the form of a structured relational table; a graph index table obtaining module configured to construct a graph index and store the graph index in the relational table to obtain a graph index table; a process decomposition module configured to decompose the approximate nearest neighbor search process into a plurality of SQL operation stages; an approximate nearest neighbor search module in communication connection with the graph index table obtaining module, the data set and query set storage module, and the process decomposition module, and configured to perform an iterative search process of the plurality of SQL operation stages for each query point in the query set based on the graph index table, and finally output an approximate nearest neighbor search result of the query set In a third aspect, the present application provides a computing framework applied to the cross-database approximate nearest neighbor search system, comprising: an SQL layer comprising an SQL statement parsing module, an approximate nearest neighbor scanning process, and a user-defined table function construction module, configured to convert a query request into a logical execution plan through an SQL optimizer; a data access layer comprising a table access method routine and a basic table interface, wherein the basic table interface provides insertion, deletion, and scanning operations, and is configured to execute the logical execution plan issued by the SQL layer; a storage layer configured to store a data table and a graph index table; The SQL layer converts the query request into the logical execution plan through the SQL optimizer, and then transmits the logical execution plan to the table access method routine of the data access layer, the data access layer performs an iterative search process of a plurality of SQL operation stages on the data table and the graph index table of the storage layer through the basic table interface according to the logical execution plan, and returns the approximate nearest neighbor search result to the SQL layer through the table access method routine, forming a closed-loop processing flow.

[0015] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects: The application provides a general method capable of constructing a vector search function on different relational database systems, realizes cross-database adaptation through unified SQL standards, converts an ANNS search process based on a graph index into a process SQL statement, and calculates by using original operators of a database system, so that cross-system migration of index construction and search logic is realized, seamless integration of advanced algorithms is supported, a development team does not need to deeply understand kernel mechanisms of various databases, and only by using standardized SQL statements, vector function deployment can be completed, engineering development cost is greatly reduced, efficient and accurate vector retrieval services can be provided for users, and the scale application of vector data in multiple scenarios is promoted. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of a cross-database approximate nearest neighbor search method provided by an embodiment of the application is shown. Figure 2 A relational representation of a graph index and other data structures provided by an embodiment of the application is shown. Figure 3 A workflow diagram for performing a single query in an embodiment of the application is shown. Figure 4 A workflow diagram for performing batch queries in an embodiment of the application is shown. Figure 5 A comparison diagram of system computing architectures provided by an existing general vector database architecture and an embodiment of the application is shown. DETAILED DESCRIPTION

[0017] In order to enable personnel in the art to better understand the application scheme, the technical solutions in the embodiments of the application will be described clearly and completely below in conjunction with the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0018] The terms "include" and "have" and any variations thereof in the specification and claims of the application and the above drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally further include steps or units not listed, or can optionally further include other steps or units inherent to the process, method, product or device. The terms "first", "second" and "third" and the like descriptions are used to distinguish different objects, and do not represent the order or limit the types of "first", "second" and "third".

[0019] In the description of the embodiments of the present application, "exemplary", "for example", "for instance" or "like" are used merely to exemplify or illustrate the relevant concept, and any embodiment or design scheme described as "exemplary", "for example", "for instance" or "like" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design schemes. In fact, the words "exemplary", "for example", "for instance" or "like" are intended to present the relevant concept in a specific manner.

[0020] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text merely describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, and in addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0021] In some processes described in the embodiments of the present application, a plurality of operations or steps are included, which appear in a specific order, but it should be understood that these operations or steps can be executed or executed in parallel without the order in which they appear in the embodiments of the present application, and the serial number of the operation is only used to distinguish different operations, and the serial number itself does not represent any execution order. In addition, these processes can include more or fewer operations, and these operations or steps can be executed in sequence or in parallel, and these operations or steps can be combined.

[0022] In order to make the purposes, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0023] In a first aspect, as Figure 1 The present application provides a cross-database approximate nearest neighbor search method, comprising the following steps: Step S1: obtaining and storing the data set and the query set in the form of a structured relational table; Step S2: constructing a graph index and storing it in a relational table, obtaining a graph index table; Step S3: decomposing the approximate nearest neighbor search process into a plurality of SQL operation stages including a candidate node screening stage, a neighbor expansion stage, a distance calculation stage, a result merging stage and a priority queue maintenance stage; Step S4: based on the graph index table, performing an iterative search process of the plurality of SQL operation stages for each query point in the query set, and finally outputting the approximate nearest neighbor search result of the query set.

[0024] This application proposes a general method for building vector search functionality across different relational database systems. It achieves cross-database adaptation through a unified SQL standard, transforming the graph-index-based ANNS search process into procedural SQL statements. Utilizing native database system operators for computation, it enables cross-system migration of index building and search logic, and supports seamless integration of cutting-edge algorithms. Development teams do not need in-depth understanding of the kernel mechanisms of various databases; they can deploy vector functionality simply through standardized SQL statements, significantly reducing engineering development costs. Furthermore, it provides users with efficient and accurate vector retrieval services, promoting the large-scale application of vector data in diverse scenarios.

[0025] In one embodiment, such as Figure 2 As shown, step S1, obtaining and storing the dataset and query set in the form of a structured relational table, specifically includes the following steps: Step S11: Obtain and store the dataset Specifically, both unstructured vector data and structured data are stored in relational tables. In, the field Vector data is used to represent other structured data, while other structured data uses separate fields. To indicate; Step S12: Store the query set using the same method as in step S11. .

[0026] In one embodiment, step S2: constructing a graph index and storing it in a relation table, and obtaining the graph index table, specifically includes either step S2A or step S2B. Steps S2A and S2B are parallel, meaning that steps S2A and S2B are two ways to obtain the graph index table.

[0027] Step S2A: Preload the existing graph index as a relation table and obtain the graph index table. .

[0028] Step S2B: Call the external graph indexing algorithm using the User-Defined Table-Generating Function (UDTF), based on the dataset. The vector data is used to construct an adjacency list-style relation table and obtain the graph index table. Since the graph index in the ANNS (Approximate Nearest Neighbor Search) process has no distance weight, this application uses an adjacency list to store the graph index table. For each node... Using array type This method stores a complete list of its neighbors, which can save a lot of storage space compared to using an edge table.

[0029] In one embodiment, step S2B: calling an external graph indexing algorithm through a user-defined table generation function to construct a relation table in the form of an adjacency table based on the vector data in the dataset, and obtaining the graph index table, specifically includes the following steps: Call the external graph indexing algorithm to process the dataset The vector data in the graph is used to generate the initial graph structure. The initial graph structure is converted into an adjacency list containing the nid primary key field and the nbs neighbor array field, and stored as a relation table. The graph index table is then obtained.

[0030] In a more specific embodiment, the operations implemented in each SQL operation stage of step S3 are as follows: In the candidate node filtering phase, a set division operation is performed between the priority queue list and the access set table to select all unvisited nodes in the priority queue. For all unvisited nodes in the selected priority queue, the query node is then used to further refine the selection process. Group the nodes and find the node with the smallest distance to the corresponding query node in each group as the candidate node table.

[0031] In the neighbor expansion phase, the candidate node table obtained in the previous phase is connected with the graph index table to obtain the corresponding neighbor list array for each candidate node. The neighbor list array is then decoupled and converted into a neighbor edge table.

[0032] In the distance calculation stage, the node data table, query set, and neighbor edge table are connected to calculate the distance result set between the vectors of the candidate point's neighbors and the query point's vector.

[0033] During the result merging phase, the candidate node set and distance result set are merged into the access set and priority queue, respectively. After deduplication, the updated access set and priority queue are obtained.

[0034] During the priority queue maintenance phase, the updated priority queue is grouped by the bid field and the TOP-K window function is applied. The specified data record with the smallest dist in each group is retained, and the candidate node set for the next iteration is obtained.

[0035] This application also adapts relational tables to represent temporary data structures during the search process, and bounded priority queues. This can be represented as a priority list. Its priority bounded property is maintained using SQL operations, while the access set Used to record the nodes visited during the approximate nearest neighbor search process and their distance information, it can also be represented as Both the priority queue list and the access set table are relational tables.

[0036] In one embodiment, step S4: Based on the graph index table, the iterative search process of the multiple SQL operation stages is executed for each query point in the query set, and finally the approximate nearest neighbor search result of the query set is output. Specifically, this is implemented as follows: Starting from the default starting point in dataset P Begin iterative search, proceeding through multiple SQL operation phases in each round of the iterative search process, starting from a bounded priority queue. Select from those that have not yet been visited and match the query set Inquiry point nearest node As a candidate node; subsequently, obtain List of neighboring nodes And calculate the relationship between these neighbor nodes and the query point. The high-dimensional vector distance is calculated; finally, the result is inserted into a priority queue. And update to the nearest one. Each node is used for the next iteration.

[0037] Until bounded priority queue If all nodes in the query set have been visited, stop the iterative search and output the approximate nearest neighbor search result for the query point in the query set.

[0038] In one embodiment, based on the above relational table structure and search process, the iterative search process of the five SQL operation stages of this application supports both single queries and batch queries. The specific implementation of the iterative search process of the five SQL operation stages is as follows: Phase 1 is the candidate node selection phase, referred to as the LOCATE phase: This phase uses a bounded priority queue. Access set The set division operation selects all unvisited nodes in the priority queue, and then, for the selected nodes, it performs a process based on the query node. Group the nodes, and for each group, find the node with the smallest distance to the corresponding query node, and use it as a candidate node table. .

[0039]

[0040] In the formula, For the first Candidate node list for each iteration; Phase 2 is the neighbor expansion phase, also known as the EXPAND phase: based on the candidate node list obtained in the previous phase... , and the index table Perform connections to obtain an array of neighbor lists for each candidate node. Furthermore, the neighbor list array is decoupled and converted into a neighbor edge table adapted for subsequent connection edge tables. :

[0041] Phase 3 is the distance calculation phase, also known as the DISTINCE phase: the node data table... Query set and neighbor's side table The process involves connecting the neighbors of the candidate point to obtain their vector data and the corresponding query point's vector data, then calculating the distance between the vectors of the candidate point's neighbors and the query point's vector to generate a result set. :

[0042] Phase 4 is the result merging phase, also known as the MERGE phase: The results from the previous phases are inserted into the original table, and duplicate results are removed; specifically, the candidate node sets are merged. To access set In the middle, merge the distance result set To priority queue After deduplication, the updated access sets are obtained respectively. and the updated priority queue As shown in the following formula:

[0043] Phase 5 is the priority queue maintenance phase, also known as the LIMIT phase: to ensure the bounded priority queue... Due to the bounded priority property, the relation table needs to be filtered based on the query node. Group the data, sort each group, and retain the group with the smallest distance to the corresponding query node. Each node.

[0044]

[0045] In the formula, For the first The bounded priority queue obtained after rounds of iterations is used for The iteration of the wheel.

[0046] In addition to the basic query structure mentioned above, some variations of near nearest neighbor search, such as cluster search and hybrid search combined with attribute filtering, can also be seamlessly integrated into the framework proposed in this invention through certain operator modifications, demonstrating strong versatility and compatibility.

[0047] In one specific embodiment, Figure 3This describes the specific workflow of a single query within the computing framework. Since there is only one query node, there is no need to utilize... This is used to identify different query nodes, so this item can be omitted in the temporary table structure. The specific steps are as follows: Step 1 (LOCATE): For Figure 3 Diagram showing the initial state of this round. The relation table exists , , , Waiting for nodes, and exist Node. For right After the set division operation, The nodes that have been filtered out are indicated by the red portion, while among the remaining unvisited nodes, of Minimum, therefore Candidate nodes for the current round are represented by the green portion.

[0048] Step 2 (EXPAND): Based on the candidate nodes obtained in the previous stage In the graph index table Searching for The corresponding neighbor list array Its neighbor list has , , Nodes, etc., to process the list UNNEST() Decouple and store in a temporary variable .

[0049] Step 3 (DISTINCE): In the node data table Search in Vector data of all nodes (in a table) (represented by the green part in the middle), and the query node. Distance calculations are performed on the vector data, and the results are stored in a temporary table. middle.

[0050] Step 4 (MERGE): Insert the calculated results into the original table and remove duplicate results. The calculated results are represented in green, and the original table data is represented in white.

[0051] Step 5 (LIMIT): To ensure Due to the bounded precedence property, the relation table needs to be filtered before saving. Find the entry with the smallest distance and delete the rest (the red part).

[0052] For batch queries, such as Figure 4 The detailed steps are as follows: Phase 1 (LOCATE): Through right The set division operation filters all visited nodes, indicated by the red portion. For the selected nodes, based on the query node... The nodes are grouped, with group 1 and group n marked with orange and blue boxes respectively. For each group, the node with the smallest distance to the corresponding query node is found and represented by the green part. The results are stored in a candidate node table. .

[0053] Phase 2 (EXPAND): Based on the candidate node list obtained in the previous phase , and the index table By making connections (as shown in orange and blue in the diagram), we obtain an array of neighbor lists for each candidate node. And obtain the neighbor edge table through decoupling. .

[0054] Phase 3 (DISTINCE): Transfer node data tables Query set and neighbor's side table Perform connections to obtain the vector data of the candidate point's neighbors and their corresponding vectors. Vector data corresponding to the query point And perform distance calculations.

[0055] Phase 4 (MERGE): The results obtained in the previous phases are processed according to... Group and insert into the original table, and remove duplicate results.

[0056] Phase 5 (LIMIT): By analogy The relation table is filtered by grouping, each group is sorted, and the group with the smallest distance to the corresponding query node is retained. Each node.

[0057] This invention designs a novel general framework for building vector search functionality across database systems. It achieves cross-database adaptation through a unified SQL standard, significantly reducing the engineering overhead of building, maintaining, and updating traditional database systems. At the same time, it can adapt to the iteration of new indexing technologies and provide users with accurate and effective approximate nearest neighbor vector search services.

[0058] Secondly, this application provides a cross-database approximate nearest neighbor search system, comprising: The graph index table retrieval module is used to construct the graph index and store it in a relation table, and to retrieve the graph index table. The dataset and query set storage module is used to acquire and store datasets and query sets in the form of structured relational tables; The process decomposition module is used to decompose the near nearest neighbor search process into multiple SQL operation stages; The approximate nearest neighbor search module is communicatively connected to the graph index table acquisition module, the dataset and query set storage module, and the process decomposition module. It is used to perform an iterative search process of the multiple SQL operation stages on each query point in the query set based on the graph index table, and finally output the approximate nearest neighbor search results of the query set.

[0059] The functions of each module in the above-mentioned cross-database approximate nearest neighbor search system correspond to the steps in the above-mentioned cross-database approximate nearest neighbor search method embodiment, and their functions and implementation processes will not be described in detail here.

[0060] Thirdly, such as Figure 2 As shown, this application provides a computational framework for a cross-database approximate nearest neighbor search system, including: The SQL layer includes an SQL statement parsing module, a procedure to ANN scan module, and a user-defined table function (UDTF) building module. It transforms query requests into logical execution plans through the SQL optimizer. The data access layer includes table access method routines (TableAmRoutine) and a basic table interface (Basic Tableinterface). The basic table interface provides insert, delete, and scan operations to execute the logical execution plan issued by the SQL layer. The storage layer is used to store data tables and graph index tables; In this process, the SQL layer transforms the query request into a logical execution plan through the SQL optimizer and then passes it to the table access method routines of the data access layer. The data access layer performs an iterative search process of multiple SQL operation stages on the data tables and graph index tables of the storage layer according to the logical execution plan through the basic table interface, and returns the approximate nearest neighbor search results to the SQL layer through the table access method routines, forming a closed-loop processing flow.

[0061] Fourthly, embodiments of this application provide a cross-database approximate nearest neighbor search device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0062] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting devices within the cross-database approximate nearest neighbor search device, as well as interfaces used for interconnecting the cross-database approximate nearest neighbor search device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0063] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0064] The processor can be a general-purpose processor, which can call a cross-database approximate nearest neighbor search program stored in memory and execute the cross-database approximate nearest neighbor search method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the cross-database approximate nearest neighbor search program is called can refer to the various embodiments of the cross-database approximate nearest neighbor search method of this application, and will not be repeated here.

[0065] Fifthly, embodiments of this application also provide a readable storage medium.

[0066] The present application has a cross-database approximate nearest neighbor search program stored on a readable storage medium, wherein when the cross-database approximate nearest neighbor search program is executed by a processor, it implements the steps of the cross-database approximate nearest neighbor search method as described above.

[0067] The method implemented when the cross-database approximate nearest neighbor search procedure is executed can be referred to in the various embodiments of the cross-database approximate nearest neighbor search method of this application, and will not be repeated here.

[0068] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0069] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0070] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A cross-database approximate nearest neighbor search method, characterized in that, Includes the following steps: Acquire and store datasets and query sets as structured relational tables; Build a graph index and store it in a relation table; retrieve the graph index table. The approximate nearest neighbor search process is broken down into multiple SQL operation stages, including candidate node screening, neighbor expansion, distance calculation, result merging, and priority queue maintenance. Based on the graph index table, an iterative search process involving multiple SQL operation stages is executed for each query point in the query set, ultimately outputting the approximate nearest neighbor search results for the query set.

2. The cross-database approximate nearest neighbor search method as described in claim 1, characterized in that, The process of constructing and storing the graph index in a relational table, and then retrieving the graph index table, specifically includes the following steps: Preload the existing graph index as a relation table, obtain the graph index table or call the external graph index algorithm through a user-defined table generation function, construct the relation table in the form of an adjacency list based on the vector data in the dataset, and obtain the graph index table.

3. The cross-database approximate nearest neighbor search method as described in claim 2, characterized in that, The process of calling an external graph indexing algorithm through a user-defined table generation function to construct an adjacency table based on vector data in the dataset and obtain the graph index table specifically includes the following steps: The external graph indexing algorithm is called to process the vector data in the dataset and generate the initial graph structure. The initial graph structure is converted into an adjacency list containing the nid primary key field and the nbs neighbor array field, and stored as a relation table. The graph index table is then obtained.

4. The cross-database approximate nearest neighbor search method as described in claim 1, characterized in that, The candidate node filtering phase involves performing a set division operation between the priority queue list and the access set list to select all unvisited nodes in the priority queue. For each unvisited node in the selected priority queue, a query node is then used to further refine the selection process. Group the nodes and find the node with the smallest distance to the corresponding query node in each group as the candidate node table.

5. The cross-database approximate nearest neighbor search method as described in claim 4, characterized in that, The neighbor expansion phase connects the candidate node table obtained in the previous phase with the graph index table to obtain the corresponding neighbor list array for each candidate node, and then decouples the neighbor list array to convert it into a neighbor edge table.

6. The cross-database approximate nearest neighbor search method as described in claim 5, characterized in that, The distance calculation stage connects the node data table, query set, and neighbor edge table to calculate and generate a distance result set between the vectors of the candidate point's neighbors and the query point's vector.

7. The cross-database approximate nearest neighbor search method as described in claim 6, characterized in that, The result merging stage merges the candidate node set and distance result set into the access set and priority queue, respectively. After deduplication, the updated access set and priority queue are obtained.

8. The cross-database approximate nearest neighbor search method as described in claim 1, characterized in that, During the priority queue maintenance phase, the updated priority queue is grouped by the bid field and the TOP-K window function is applied. The specified data record with the smallest dist in each group is retained, and the candidate node set for the next iteration is obtained.

9. A cross-database approximate nearest neighbor search system, characterized in that, include: The dataset and query set storage module is used to acquire and store datasets and query sets in the form of structured relational tables; The graph index table retrieval module is used to construct the graph index and store it in a relation table, and to retrieve the graph index table. The process decomposition module is used to decompose the near nearest neighbor search process into multiple SQL operation stages; The approximate nearest neighbor search module is communicatively connected to the graph index table acquisition module, the dataset and query set storage module, and the process decomposition module. It is used to perform an iterative search process of the multiple SQL operation stages on each query point in the query set based on the graph index table, and finally output the approximate nearest neighbor search results of the query set.

10. A computational framework applied to the cross-database approximate nearest neighbor search system as described in claim 9, characterized in that, include: The SQL layer includes an SQL statement parsing module, an approximate nearest neighbor scan process, and a user-defined table function construction module, which transforms query requests into logical execution plans through the SQL optimizer. The data access layer contains table access method routines and basic table interfaces. The basic table interfaces provide insert, delete, and scan operations to execute the logical execution plan issued by the SQL layer. The storage layer is used to store data tables and graph index tables; In this process, the SQL layer transforms the query request into a logical execution plan through the SQL optimizer and then passes it to the table access method routines of the data access layer. The data access layer performs an iterative search process of multiple SQL operation stages on the data tables and graph index tables of the storage layer according to the logical execution plan through the basic table interface, and returns the approximate nearest neighbor search results to the SQL layer through the table access method routines, forming a closed-loop processing flow.

Images