A unified cost model based multi-modal hybrid retrieval method of relations, graphs and vectors

Abstract

This invention discloses a multi-model hybrid retrieval method based on a unified cost model, encompassing relational, graph, and vector queries, and relates to the field of database query and information retrieval technology. The method obtains a multi-model query expression containing relational query conditions, graph association constraints, and vector similarity retrieval conditions, and parses it into a unified logical query plan tree containing relational scan operators, graph adjacency operators, vector retrieval operators, and join operators. Based on cross-model statistical information, the output cardinality and execution cost of each operator are estimated and converted into a unified cost unit. Candidate hybrid join trees are enumerated, and the target join order with the lowest total cost is selected. The target physical execution plan is generated and executed to obtain the multi-model hybrid retrieval result. This invention can reduce intermediate join results and query overhead, improving cross-model retrieval efficiency.

Description

Technical Field

[0001] This invention relates to the field of database query and information retrieval technology, specifically to a multi-model hybrid retrieval method and its implementation that uniformly supports relational models, graph models, and vector models at the database kernel level. Background Technology

[0002] In recent years, with the rapid development of Large Language Models (LLM) and Retrieval Augmentation (RAG) technologies, the need for unified retrieval of cross-model data has become increasingly urgent. In actual business scenarios, structured relational data, graph-structured relational data, and vectorized semantic data often coexist in the same system. For example, in an enterprise knowledge base, user information is stored in relational tables, social relationships between users are expressed in graph structures, and documents and user profiles exist in the form of vector embeddings.

[0003] Current cross-model retrieval implementations have the following drawbacks:

[0004] 1. Application-layer assembly lacks transactional guarantees. Existing solutions often deploy relational, graph, and vector databases independently, making multiple network calls to each system at the application layer and fusing the results. Due to the transaction isolation of each system, the atomicity and consistency of cross-model retrieval cannot be guaranteed, potentially leading to data inconsistency issues.

[0005] 2. Multiple network round trips result in low efficiency. The application-layer aggregation mode requires querying three independent systems separately, involving at least three network round trips. The intermediate results need to be processed again at the application layer, such as merging, filtering, and sorting. It cannot utilize the optimization capabilities of the database kernel, such as operator pushdown and pipelined execution. The overall query latency is the sum of the latencies of each system.

[0006] 3. Semantic gaps limit optimization. Because each system optimizes independently, the query optimizer cannot perceive the statistical characteristics and relationships between cross-model data, making it difficult to generate a globally optimal physical execution plan. For example, it cannot selectively estimate and adjust the JOIN order of relational tables based on vector indexes.

[0007] 4. Vector retrieval accuracy is uncontrollable. Existing vector databases often only provide a single Approximate Nearest Neighbor (ANN) retrieval interface, without clearly distinguishing the use case boundaries between ANN and exact KNN. In scenarios requiring strictly accurate results (such as compliance auditing and financial risk control), approximate retrieval may result in errors; in scenarios requiring high throughput and low latency, exact retrieval will incur unnecessary performance overhead.

[0008] A natural technical approach to solving the above problems is to combine specialized systems, such as installing vector retrieval extensions and graph traversal extensions simultaneously in the same database instance, or performing federated queries on independent relational, graph, and vector databases through the application layer. However, existing combined solutions often still struggle to achieve global query optimization across models. The specific reasons are as follows:

[0009] First, there's the single-kernel extension overlay mode. After installing vector and graph extensions on the same database instance, vector distance operators and graph traversal functions can be directly embedded in SQL statements. The vector extension cost model only perceives index type, vector cardinality, and dimension, while the graph extension cost model only perceives graph degree distribution and traversal depth; furthermore, the two lack a communication mechanism. The optimizer determines the join order based on the order of the FROM clauses or fixed heuristics, failing to equally incorporate the output cardinality of graph adjacency traversal and vector retrieval into the enumeration and evaluation of the join order. The fundamental reason is that the cost estimation logic of each extension lacks a unified interface that simultaneously receives and compares the cardinality and cost of the three heterogeneous operators.

[0010] Second, there is the separate multi-system federated model. Independently deployed relational, graph, and vector databases return intermediate results, which are then merged after being transmitted over the network at the application layer. JOIN, sorting, and filtering are decoupled from the database kernel's indexing and batch processing capabilities, and the MVCC snapshots of the three systems cannot be aligned at all times, resulting in a lack of time consistency guarantees for cross-model JOIN results.

[0011] Even if vector and graph extensions are deployed on the same database kernel, their cost models and measurement units are fundamentally different: vector retrieval is measured by page views and CPU overhead, while graph traversal is calculated by the size of the traversal front and the number of edge visits, lacking a unified normalization process. Simply installing extensions side-by-side cannot solve this problem. Summary of the Invention

[0012] Purpose of the invention: This invention provides a multi-model hybrid retrieval method based on a unified cost model, which supports relational, graph, and vector models at the database kernel level. This allows a single SQL statement to express three retrieval semantics simultaneously and complete the task within the same transaction, thus addressing the shortcomings of existing cross-model retrieval schemes in terms of transaction consistency, query execution efficiency, and controllability of vector retrieval accuracy.

[0013] Technical solution: A multi-modal hybrid retrieval method based on a unified cost model, encompassing relational, graph, and vector models, applied to the database kernel, includes the following steps:

[0014] Obtain a multi-model query expression, which includes relational data query conditions, graph association constraints, and vector similarity retrieval conditions;

[0015] During the query parsing phase, the multi-model query expression is parsed into a unified logical query plan tree. The unified logical query plan tree includes a relation scan operator, a graph adjacency operator, a vector retrieval operator, and a connection operator. The outputs of the graph adjacency operator and the vector retrieval operator both include target identifiers for connection with relational data identifiers.

[0016] Obtain cross-model statistical information corresponding to the unified logical query plan tree. Based on the cross-model statistical information, estimate the output cardinality of the relation scan operator, graph adjacency operator, and vector retrieval operator respectively through the cross-model unified cost model, and estimate the execution cost of the relation scan operator, graph adjacency operator, and vector retrieval operator in a unified cost unit.

[0017] During the query optimization phase, within the scope of satisfying the join conditions and semantic equivalence constraints, multiple candidate hybrid join trees formed by relation scanning operators, graph adjacency operators and vector retrieval operators are enumerated, and the total cost is calculated based on the output cardinality and execution cost of the operators in each candidate hybrid join tree.

[0018] The candidate hybrid connection tree with the lowest total cost is selected as the target connection order, and the target physical execution plan is generated according to the target connection order. The execution positions of the graph adjacency operator and the vector retrieval operator in the connection tree are not restricted by the writing order or generation order in the multi-model query expression.

[0019] The target physical execution plan is used to execute relation scanning operators, graph adjacency operators, vector retrieval operators, and connection operators to obtain multi-modal hybrid retrieval results.

[0020] Furthermore, the multi-model query expression is an extended SQL statement or an internal query expression derived from an extended SQL statement. The extended SQL statement includes relational table references, relational predicate conditions, graph adjacency operators, and vector retrieval operators.

[0021] Relational predicate conditions include relational attribute query conditions and join conditions established based on relational data identifiers;

[0022] The graph adjacency operator includes the source node, edge type, traversal depth, and traversal direction, and is used to return the graph adjacency result including node identifier, graph distance, and path weight.

[0023] Vector retrieval operators include query vector, number of vector retrieval results, and vector retrieval type. Vector retrieval types include approximate vector retrieval type and exact vector retrieval type. Vector retrieval operators are used to return vector retrieval results that include entity identifiers and vector distances.

[0024] During the query parsing phase, graph adjacency operators are parsed into graph adjacency operators, and vector retrieval operators are parsed into approximate vector retrieval operators or exact vector retrieval operators depending on the vector retrieval type.

[0025] The relation scanning operator, graph adjacency operator, and vector retrieval operator exchange data through a unified intermediate result structure. The unified intermediate result structure includes columnar data blocks and optional vector data regions. Node identifiers, entity identifiers, and relation data identifiers adopt matching data types, enabling graph adjacency results and vector retrieval results to be processed by join operators, filter operators, sort operators, or aggregation operators.

[0026] Furthermore, cross-model statistics include the cardinality of relation tables, relation predicate selectivity, average degree of different edge types, size of each jump traversal front, cardinality of vector sets, vector dimensions, and vector index parameters;

[0027] The cross-model unified cost model uses the basic CPU computation cost as the unified cost unit. The tuple processing cost and the single edge access cost are both calibrated using the unified cost unit and estimated in the following way:

[0028] The output cardinality of the relation scan operator is determined based on the statistical cardinality of the relation table, or based on the statistical cardinality of the relation table and the selectivity of the relation predicate. The execution cost of the relation scan operator is determined based on the output cardinality and the cost of tuple processing.

[0029] The output cardinality of the graph adjacency operator is determined based on the average degree of the edge type and the traversal depth. The execution cost of the graph adjacency operator is determined by the sum of the products of the size of the traversal front of each hop from the first hop to the traversal depth and the cost of accessing a single edge.

[0030] The output cardinality of the vector retrieval operator is determined based on the number of vector retrieval results.

[0031] When the vector retrieval operator uses KNN (K-Nearest Neighbors) for exact retrieval, the execution cost is determined by the product of the vector set cardinality, vector dimension, and uniform cost unit. When the vector retrieval operator uses HNSW (Hierarchical Navigable Small World) index retrieval, the execution cost is determined by the product of the logarithm of the vector set cardinality, the search expansion factor, vector dimension, and uniform cost unit. When the vector retrieval operator uses IVFFlat (Inverted File with Flat Vectors) index retrieval, the execution cost is determined by the product of the ratio of the vector set cardinality to the total number of cluster buckets, the number of cluster buckets to be scanned, the vector dimension, and uniform cost unit. When the vector retrieval operator uses IVFPQ (Inverted File with ProductQuantization) index retrieval, the execution cost is determined by the product of the ratio of the vector set cardinality to the total number of cluster buckets, the number of cluster buckets to be scanned, the equivalent compressed dimension after product quantization, and uniform cost unit.

[0032] The output cardinality of the filtering operator is determined based on the output cardinality of the sub-operators and the selectivity of the filtering predicate, while the output cardinality of the connection operator is determined based on the output cardinality of the left sub-operator, the output cardinality of the right sub-operator, and the selectivity of the connection condition.

[0033] Furthermore, during the query optimization phase, dynamic programming is used to enumerate multiple candidate hybrid connection trees;

[0034] When there is no input data dependency among the relation scan operator, graph adjacency operator, and vector retrieval operator, the relation scan operator, graph adjacency operator, and vector retrieval operator are included as peer candidate inputs in the same connection order enumeration framework;

[0035] When the source node of a graph adjacency operator depends on the result of a preceding subquery, the graph adjacency operator is restricted to the period following the preceding subquery that produced the source node; when the query vector or retrieval range of a vector retrieval operator depends on the output result of a preceding operator, the vector retrieval operator is restricted to the period following the preceding operator that produced the corresponding output result.

[0036] For each candidate hybrid connection tree that satisfies the connection conditions, data dependencies and semantic equivalence constraints, the cumulative cost is calculated based on the output cardinality and execution cost of each operator, and the candidate hybrid connection tree with the minimum cumulative cost is determined as the target connection order.

[0037] The target connection order allows the output of a graph adjacency operator or a vector retrieval operator to be placed in the left or right subtree of the connection tree.

[0038] Furthermore, a target physical execution plan is generated based on the target connection order, including:

[0039] Based on the graph data size and traversal depth, the physical implementation method of the graph adjacency operator is selected from adjacency list index scan, CSR (Compressed Sparse Row) row batch adjacency access, and incremental breadth-first traversal.

[0040] When the vector retrieval operator is an exact vector retrieval operator, KNN exact retrieval based on Flat full distance calculation is selected as the physical implementation method;

[0041] When the vector retrieval operator is an approximate vector retrieval operator, the physical implementation method is selected from approximate vector index retrieval, two-stage vector retrieval, and KNN precise retrieval based on Flat full distance calculation, according to the vector set cardinality, vector dimension, existing vector index, index recall rate, and memory budget.

[0042] Approximate vector index retrieval includes HNSW index retrieval, IVFFlat index retrieval, and IVFPQ index retrieval;

[0043] Two-stage vector retrieval obtains more candidate results than the number returned by vector retrieval through approximate vector indexing, calculates the precise distance between the candidate results and reorders them to obtain the top K vector retrieval results.

[0044] Furthermore, when the physical implementation of the approximate vector retrieval operator is approximate vector index retrieval, the method further includes:

[0045] After the approximate vector index retrieval is completed, the distance distribution index is determined based on the ratio of the distance between the farthest neighbor and the nearest neighbor in the retrieval results, and the result coverage index is determined based on the ratio of the actual number of returned results to the number of results returned by the vector retrieval.

[0046] When the distance distribution index is greater than the preset distance threshold, or the result coverage index is less than the preset coverage threshold, KNN precise retrieval using Flat full distance calculation is started in the same transaction context. The approximate vector index retrieval results and KNN precise retrieval results are merged, the precise distance is calculated for the merged candidate results, and the results are reordered according to the precise distance and the top K results are truncated.

[0047] The background recall rate of the approximate vector index is monitored using a sliding window. The retrieval results of the approximate vector index and the K-nearest neighbor exact retrieval results are periodically sampled, and the actual recall rate is determined based on the ratio of the number of overlapping results in the two retrieval results to the number of vector retrieval results returned.

[0048] When the average actual recall rate within the sliding window is consistently lower than the preset proportion of the theoretical recall rate of the corresponding vector index, the selection priority of the corresponding vector index in the query optimization phase is reduced, and asynchronous reconstruction in the background is triggered. During the reconstruction, the original vector index continues to provide retrieval services, and after the reconstruction is completed, the reconstructed vector index is atomically switched to the service index.

[0049] Furthermore, before executing the graph adjacency operator and the vector retrieval operator, a graph-aware vector partitioning joint index is pre-constructed based on the graph data and vector data, specifically including:

[0050] Establish a mapping relationship between vertex identifiers and community identifiers based on the graph community detection results;

[0051] Cluster the vertex vectors within each community, establish a double-key index pointing to the vertex set by both community identifier and cluster identifier, and record the centroid vector corresponding to each cluster; construct a local vector index for the vertex vectors within each cluster; and construct a cross-community fast channel index covering a preset proportion of high-traffic vertices in each community.

[0052] When performing multi-modal hybrid retrieval, the set of reachable nodes is first obtained through the graph adjacency operator. The community scope corresponding to the set of reachable nodes is determined according to the mapping relationship, and vector retrieval is performed in the local vector index corresponding to the community scope. When the number of results returned by the local vector index is less than the number returned by the vector retrieval, candidate results are supplemented by the cross-community fast channel index.

[0053] Furthermore, during the execution phase, the same transaction identifier is assigned to the multi-model query expression, and the creation transaction identifier and deletion transaction identifier are recorded for relation table tuples, graph edge records and vector entries;

[0054] Under the same transaction snapshot, the relation scanning operator, graph adjacency operator, and vector retrieval operator perform visibility checks on relation table tuples, graph edge records, and vector entries according to a unified multi-version concurrency control visibility rule, so that the multi-modal mixed retrieval results correspond to the same transaction snapshot;

[0055] For relational scan operators, graph adjacency operators, and vector retrieval operators that do not have input data dependencies, they are executed in parallel in different threads within the same transaction context based on operator dependencies.

[0056] The present invention also provides an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, wherein when the programs are executed by the processors, they implement the above-described relational, graph, and vector multimodal hybrid retrieval method based on a unified cost model.

[0057] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described multi-modal hybrid retrieval method based on a unified cost model of relations, graphs, and vectors.

[0058] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described multi-modal hybrid retrieval method based on a unified cost model, encompassing relational, graph, and vector data.

[0059] Beneficial effects:

[0060] (1) This invention establishes a unified cost model across models in the database kernel, converting the output cardinality and execution cost of relation scan operators, graph adjacency operators, and vector retrieval operators into a unified cost unit, and incorporating the three types of operators into the connection order enumeration of the candidate hybrid connection tree, enabling the optimizer to select the target connection order based on global cost information. In this way, graph adjacency results and vector retrieval results are no longer restricted by the original SQL writing order or fixed execution rules, but can be adjusted to a more suitable connection position according to selectivity and cost, thereby reducing the scale of intermediate results in the cross-relation, graph, and vector hybrid retrieval process, reducing the overhead of subsequent connection, filtering, and sorting, and improving the overall query efficiency.

[0061] (2) This invention improves the reliability of vector similarity retrieval by distinguishing between KNN exact retrieval and ANN approximate vector index retrieval during the vector retrieval process and implementing adaptive degradation and error correction. When the approximate vector index retrieval results in abnormal distance distribution or insufficient returned results, the system can automatically trigger exact retrieval for verification and merge and reorder the approximate retrieval results and exact retrieval results, thereby maintaining the efficiency advantage of approximate retrieval while reducing the impact of insufficient recall or sorting deviation on the final retrieval results.

[0062] (3) This invention incorporates relational data, graph data, and vector data into a unified query execution process within the database kernel, and performs visibility checks under the same transaction snapshot, ensuring that cross-model retrieval results correspond to consistent data versions. Compared to the method of calling various databases separately at the application layer and then merging the results, this invention reduces multiple network round trips and intermediate result transfers at the application layer, and avoids result deviations caused by inconsistent transaction snapshots of different systems. Attached Figure Description

[0063] Figure 1 A schematic diagram of the five-layer architecture of the multi-modal hybrid retrieval system of relations, graphs and vectors provided in the embodiments of the present invention;

[0064] Figure 2 This is a schematic diagram of a cross-model unified logical query plan tree and connection order rearrangement provided in an embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram of the three-layer structure of the graph-aware vector partitioned joint index provided in an embodiment of the present invention;

[0066] Figure 4 The flowchart illustrates the adaptive degradation and index recovery process for approximate vector retrieval provided in this embodiment of the invention. Detailed Implementation

[0067] The following combination Figures 1 to 4 The technical solution of the present invention will be further explained below.

[0068] This invention provides a hybrid method for relational, graph, and vector operations based on a unified cost model. In the database kernel, relational scan operators, graph adjacency operators, and vector retrieval operators are placed in the same logical query plan tree. Based on relational data statistics, graph data statistics, and vector data statistics, the output cardinality and execution cost of the three types of heterogeneous operators are uniformly estimated. The three types of operators are included as peer candidate inputs in the connection order enumeration of the hybrid connection tree, enabling the optimizer to select the target connection order based on the global cost, without being restricted by the writing order of the FROM clause in the original SQL statement.

[0069] In the following three embodiments, Embodiment 1 fully illustrates the optimization and execution process of multi-modal hybrid queries based on a unified cost model, using a social recommendation scenario; Embodiment 2 uses the data model and query target of Embodiment 1 to further explain the graph-aware vector partition joint index that can be used in the target physical execution plan generation and execution stage of Embodiment 1; Embodiment 3 also uses the data model and query target of Embodiment 1, and further explains the result quality detection, adaptive degradation, background monitoring and index recovery process when the vector retrieval adopts approximate vector index retrieval.

[0070] Example 1

[0071] like Figure 1 As shown, this embodiment uniformly supports relational models, graph models, and vector models at the database kernel level. The database kernel includes an SQL interface layer, a query parsing and optimization layer, an execution engine layer, a storage engine layer, and a transaction management layer. The SQL interface layer receives extended SQL statements containing relational queries, graph association queries, and vector similarity queries; the query parsing and optimization layer generates a unified logical query plan tree, estimates the output cardinality and execution cost of various operators, enumerates mixed join orders, and generates a target physical execution plan; the execution engine layer executes relational scan operators, graph adjacency operators, vector retrieval operators, and join operators; the storage engine layer manages relational tables, graph adjacency data, and vector indexes; and the transaction management layer enables relational tuples, graph edge records, and vector entries to participate in queries under the same transaction snapshot.

[0072] The data model in this embodiment includes a relational table `users`, a relational table `user_profiles`, and a directed graph `follow_graph`. The `users` table stores structured attributes such as user ID, name, and age; the `user_profiles` table stores user IDs and their corresponding 384-dimensional interest embedding vectors; the `follow_graph` uses `follows` edges to represent the follow relationships between users; and interest vectors are pre-indexed.

[0073] The query requirement is: find the top-10 users with the most similar interests from user A's second-degree connections, excluding users that user A has already directly followed, and sort them by a weighted fusion of graph path weight and interest similarity, where graph path weight accounts for 0.3 and interest similarity accounts for 0.7.

[0074] This embodiment provides a hybrid method for relational, graph, and vector multimodal approaches based on a unified cost model, including the following steps.

[0075] S101, Obtain the multi-model query expression, which includes relational data query conditions, graph association constraints, and vector similarity retrieval conditions.

[0076] According to an embodiment of the present invention, the multi-model query expression is an extended SQL statement or an internal query expression derived from an extended SQL statement. The extended SQL statement includes a relational table reference, relational predicate conditions, a graph adjacency operator, and a vector retrieval operator. The relational table reference includes a relational table identifier and a relational table alias. The relational predicate conditions include relational attribute query conditions and relational data identifiers used to connect with graph adjacency results or vector similarity results. The graph adjacency operator includes a source node, edge type, traversal depth, and traversal direction, and is used to return graph adjacency results, which include node identifiers that can be connected with relational data identifiers. The vector retrieval operator includes a query vector, a vector retrieval return quantity, and a vector retrieval type, and is used to return vector similarity results, which include entity identifiers that can be connected with relational data identifiers. This allows a single extended SQL statement to simultaneously express relational data queries, graph association queries, and vector similarity queries.

[0077] Specifically, to enable a single SQL statement to simultaneously describe relational, graph, and vector retrieval semantics, graph adjacency operators, approximate vector retrieval operators, and exact vector retrieval operators are introduced on top of standard SQL.

[0078] The graph adjacency operator is represented as:

[0079] GRAPH_ADJACENT(source, edge_type, depth, direction)

[0080] Wherein, source represents the source node, edge_type represents the edge type, depth represents the traversal depth, and direction represents the traversal direction. The traversal directions include OUT, IN, and BOTH, which represent traversing along outgoing edges, incoming edges, and bidirectional edges, respectively.

[0081] The graph adjacency operator is used as a table-valued function to perform multi-hop neighbor traversal on the graph and return:

[0082] (target_id, distance, path_weight)

[0083] Where target_id represents the target node identifier, distance represents the graph distance of the target node relative to the source node, and path_weight represents the corresponding path weight.

[0084] The approximate vector retrieval operator is represented as:

[0085] VECTOR_ANN(query, k, index_type, [recall_target])

[0086] Where query represents the query vector, k represents the number of vector retrieval results, index_type represents the vector index type, which can be hnsw, ivfflat, ivfpq or auto, and recall_target represents the minimum recall rate expected to be achieved.

[0087] The approximate vector retrieval operator returns the Top-K similar vectors based on the approximate vector index, and returns:

[0088] (entity_id, distance, rank)

[0089] Where entity_id represents the entity identifier, distance represents the distance between the query vector and the candidate vector, and rank represents the sorting position.

[0090] The exact vector search operator is represented as:

[0091] VECTOR_KNN(query, k, metric)

[0092] Here, metric represents the distance metric, including cosine distance, Euclidean distance, and inner product distance. The precise vector retrieval operator returns the globally guaranteed K nearest neighbors through precise distance calculation.

[0093] The primary or foreign keys in the relational table, the node identifiers returned by the graph adjacency operator, and the entity identifiers returned by the vector retrieval operator all use the same data type, enabling graph adjacency results and vector similarity results to be equi-joined with the relational table. Graph adjacency operators and vector retrieval operators can be located in the FROM clause or WITH clause and, together with relational table references, form cross-model queries.

[0094] The following extended SQL statement is used in this embodiment:

[0095] SELECT u.name, u.age, g.path_weight AS graph_score,

[0096] 1.0 - v.distance AS vector_score,

[0097] g.path_weight * 0.3 + (1.0 - v.distance) * 0.7 AS combined_score

[0098] FROM users u

[0099] JOIN GRAPH_ADJACENT (3001, 'follows', 2, 'OUT') g

[0100] ON u.user_id = g.target_id

[0101] JOIN VECTOR_KNN(

[0102] (SELECT interest_embedding FROM user_profiles WHERE user_id =3001),

[0103] 100, 'cosine') v ON u.user_id = v.entity_id

[0104] LEFT JOIN GRAPH_ADJACENT (3001, 'follows', 1, 'OUT') d

[0105] ON u.user_id = d.target_id

[0106] WHERE d.target_id IS NULL

[0107] ORDER BY combined_score DESC LIMIT 10;

[0108] Where 3001 is the internal user identifier of user A. users u and user_profiles are references to relational tables; u.user_id=v.entity_id and u.user_id=g.target_id are cross-model connection conditions; the two-hop graph adjacency operation is used to obtain user A's second-degree connections; the one-hop graph adjacency operation is used to exclude users that user A has already directly followed; VECTOR_KNN is used to obtain 100 candidate users whose interest vectors are similar to user A's.

[0109] This extended SQL statement enables relational data queries, graph association queries, and vector similarity queries to be parsed, optimized, and executed all at once within the database kernel, eliminating the need for the application layer to access relational databases, graph databases, and vector databases separately and then manually merge the results.

[0110] S102, the multi-model query expression is parsed into a unified logical query plan tree, which includes a relation scan operator, a graph adjacency operator, a vector retrieval operator, and a join operator. The outputs of the graph adjacency operator and the vector retrieval operator both include target identifiers for joining with relational data identifiers.

[0111] The query parser performs lexical, syntactic, and semantic analysis on the extended SQL statement and generates a unified logical query plan tree. In this embodiment, the unified logical query plan tree includes at least:

[0112] TABLE_SCAN(users): Scans all rows in the users table.

[0113] TABLE_SCAN(user_profiles): Scans all rows in the user_profiles table.

[0114] GRAPH_ADJACENT(follows, depth=2): For each row of the source intermediate result set, perform a 2-degree adjacency query on all intermediate result sets.

[0115] GRAPH_ADJACENT(follows, depth=1): For each row of the source intermediate result set, perform a 1-degree adjacency query on all intermediate result sets.

[0116] VECTOR_KNN(cosine, k=100): Performs a K-nearest neighbor lookup on the target vector and returns the 100 vector records of the exact nearest neighbors.

[0117] And join operators, filter operators, projection operators, and sorting operators used to perform equi-joins, relational condition filtering, exclusion of followed users, fusion score calculation, and result sorting.

[0118] Relation table references are parsed as relation scan operators, relation attribute query conditions are parsed as filter operators, join conditions are parsed as join operators, graph adjacency operators are parsed as graph adjacency operators, and exact vector retrieval operators are parsed as vector retrieval operators.

[0119] Traditional relational operators, graph adjacency operators, and vector retrieval operators reside within the same unified logical query plan tree. The leaf nodes of the unified logical query plan tree can be relational scan operators, graph adjacency operators, or vector retrieval operators, while non-leaf nodes can be join operators, filter operators, projection operators, sorting operators, or aggregation operators. The left and right subtrees of a join operator can be formed by any valid combination of relational scan results, graph adjacency results, and vector retrieval results.

[0120] Various operators exchange data through a unified intermediate result structure, HybridTupleBatch. HybridTupleBatch includes columnar data blocks and optional vector data regions. The columnar data blocks support INT64, FLOAT64, VARSTRING, and FLOAT64ARRAY physical types. Relationship attribute columns, node identifier columns, entity identifier columns, path weight columns, and vector value columns share the same memory management, batch processing, and encoding / decoding path, and can be directly processed by filtering, joining, sorting, and aggregation operators without requiring separate data transfer formats for graph and vector results.

[0121] An example HybridTupleBatch structure is as follows:

[0122] HybridTupleBatch {

[0123] ColumnVector columns[]; / / Columnar data blocks, each column is a fixed-length array

[0124] uint32 num_columns; / / Number of columns

[0125] VectorStorage* vector_area; / / Optional, vector data area

[0126] }

[0127] ColumnVector {

[0128] uint32 col_type; / / relation, graph node, vector_element, etc.

[0129] uint32 data_type; / / INT64 | FLOAT64 | VARSTRING | FLOAT64ARRAY

[0130] void* data; / / Array of column values ​​(row-major order)

[0131] }

[0132] VectorStorage {

[0133] uint32 dim; / / Vector dimension (e.g., 384)

[0134] void* data; / / dim dimensional main-order floating-point array

[0135] }

[0136] S103, obtain cross-model statistical information corresponding to the unified logical query plan tree, including relational data statistical information, graph data statistical information and vector data statistical information. Based on the cross-model statistical information, estimate the output cardinality and execution cost of relational scan operator, graph adjacency operator and vector retrieval operator through cross-model unified cost model, and convert the execution cost into a unified cost unit.

[0137] According to embodiments of the present invention, the statistical information of relational data includes the cardinality of relational tables, histograms of attribute value distributions, and relational predicate selectivity. The statistical information of graph data includes the average degree distribution corresponding to different edge types, and historical records of the frontier size of each hop breadth-first traversal. The statistical information of vector data includes the cardinality N of the vector set, the vector dimension dim, the types of existing vector indexes, and the vector index search parameters. The number k returned by vector retrieval is determined by the vector retrieval operator.

[0138] In this embodiment, the following statistical information can be obtained: the statistical cardinality of the relation table users is approximately 1 million; the estimated output of the one-hop adjacency traversal of the follows edge is approximately 500 rows; the estimated output of the two-hop adjacency traversal of the follows edge is approximately 100,000 rows; the vector dimension of the vector set is 384; and the number of exact vector retrieval returns k is 100.

[0139] Cross-model statistics are collected and updated uniformly by the storage engine layer and used as input to the cross-model unified cost model.

[0140] According to an embodiment of the present invention, the database kernel constructs a cross-model unified cost model that simultaneously covers relation scan operators, graph adjacency operators, and vector retrieval operators. The cross-model unified cost model estimates the output card and execution cost for each type of operator, and uses `cpu_operator_cost` as the unified cost unit to determine the execution cost of each type of operator, enabling the execution costs of relation scan, graph adjacency traversal, and vector retrieval to be compared within the same cost space. Specifically, `cpu_operator_cost` is defined as the average time cost of executing one basic arithmetic operation within a single CPU cycle. All physical operations of all operators are ultimately converted to the quantity of this base unit.

[0141] (1) The output cardinality and execution cost of the relation scan operator can be expressed as:

[0142] card = Table statistical base

[0143] cost = card × cpu_tuple_cost

[0144] Where cpu_tuple_cost ≈ cpu_operator_cost × (10 ~ 100), the cost of processing each tuple is approximately 10 to 100 cpu_operator_costs, covering fixed overhead such as tuple deserialization and null checking. The coefficient range depends on the tuple width, deserialization complexity, and additional checksums (10-100 are empirical values ​​and can be adjusted appropriately). When the relation scan results continue to be filtered by relation predicates, the output cardinality of the filtering operator is expressed as:

[0145] card = card(child) × selectivity(predicate)

[0146] Here, card(child) represents the output cardinality of the sub-operator, and selectivity(predicate) represents the selectivity of the filtering predicate.

[0147] (2) The output cardinality of the graph adjacency operator is expressed as:

[0148] card = avg_degree(edge_type) depth

[0149] Here, avg_degree(edge_type) represents the average degree corresponding to the specified edge type, and depth represents the traversal depth specified in the graph adjacency operator.

[0150] The execution cost of the graph adjacency operator is expressed as:

[0151]

[0152] Where BFS_frontier_size(d) represents the number of vertices traversed in the frontier during the d-th hop, and cost_per_edge_access represents the discounted CPU cost of accessing a single edge. For the CSR format: cost_per_edge_access ≈ 1 × cpu_operator_cost. Due to its compact structure and contiguous storage of adjacent edges, a single access only requires one offset read plus one target ID read, resulting in the lowest cost. For the adjacency list format: cost_per_edge_access ≈ 3 × cpu_operator_cost. Due to its flexible structure, the access path is longer, requiring three steps: index lookup, page access discounting, and record decoding.

[0153] (3) When the vector retrieval operator uses KNN exact retrieval, the output cardinality and execution cost are expressed as follows:

[0154] card = k

[0155] cost = N × dim × cpu_operator_cost

[0156] Where N represents the cardinality of the vector set, and dim represents the vector dimension. Each distance calculation involves dim multiply-accumulate operations. Under the FMA fusion instruction of modern CPUs, each multiply-accumulate operation is counted as 1 cpu_operator_cost. It needs to be calculated with each of the full N vectors.

[0157] When the vector retrieval operator uses the HNSW approximate vector index retrieval, the output cardinality and execution cost are expressed as follows:

[0158] card = k

[0159] cost =log(N) × ef_search × dim × cpu_operator_cost

[0160] Here, ef_search represents the search expansion factor, and log(N) reflects the number of search layers in the HNSW hierarchical graph. An index structure is used to avoid a full scan, with only logarithmic vectors participating in the calculation. Compared to KNN, several unmatched rows are skipped.

[0161] When the vector retrieval operator uses the IVFFlat approximate vector index retrieval, the output cardinality and execution cost are expressed as follows:

[0162] card = k

[0163] cost = (N / nlist) × nprobe × dim × cpu_operator_cost

[0164] Where nlist represents the total number of K-Means clustering buckets during index building, and N / nlist represents the average number of vectors in a single inverted index. nprobe represents the number of clustering buckets to be scanned during the query phase, and nprobe / nlist is the proportion of the total data actually scanned.

[0165] When the vector retrieval operator uses IVFPQ approximate vector index retrieval, the output cardinality and execution cost are expressed as follows:

[0166] card = k

[0167] cost = (N / nlist) × nprobe × dim_pq × cpu_operator_cost

[0168] Where dim_pq represents the equivalent compressed dimension after PQ product quantization, which is usually 1 / 4 to 1 / 8 of the original dim.

[0169] (4) The output cardinality of the connection operator is expressed as:

[0170] card =card(left) × card(right)× join_selectivity(condition)

[0171] Here, card(left) and card(right) represent the output cardinality of the left and right sub-operators, respectively, and join_selectivity(condition) represents the join condition selectivity.

[0172] Based on the above estimations, the output cardinality and execution cost of the relation scanning operator, graph adjacency operator, vector retrieval operator, filtering operator, and connection operator can be used to calculate the total cost of the candidate hybrid connection tree.

[0173] S104. In the query optimization phase, within the range of satisfying the connection conditions and semantic equivalence constraints, enumerate multiple candidate hybrid connection trees formed by relation scanning operators, graph adjacency operators and vector retrieval operators, and calculate the total cost based on the output cardinality and execution cost of the operators in each candidate hybrid connection tree. Select the candidate hybrid connection tree with the minimum total cost as the target connection order.

[0174] like Figure 2As shown, the optimizer, within the constraints of connection conditions, data dependencies, and semantic equivalence, employs a dynamic programming framework to enumerate valid hybrid connection orders. Relation scan operators, graph adjacency operators, and vector retrieval operators are input as peer candidates into the same connection order enumeration framework. For each candidate hybrid connection tree, the optimizer accumulates the execution costs of each operator and calculates the total cost of the candidate hybrid connection tree by combining the operator's output cardinality and connection selection rate. The candidate hybrid connection tree with the minimum total cost is selected as the target connection order.

[0175] Regarding KNN exact query and ANN approximate query for vectors, based on user semantic constraints, the operation VECTOR_KNN / VECTOR_ANN(index) is directly generated on the vector index to which the table belongs, producing the corresponding intermediate retrieval result set. Regarding the generation of graph adjacency operators, briefly described below: An equi-join based on foreign keys between three relational tables can be converted into a one-hop graph adjacency operator under certain conditions. Similarly, an equi-join based on primary and foreign keys between five tables can be converted into a two-hop graph adjacency operator under certain conditions. For details, please refer to the patent: A method for creating and querying multi-table hash joins based on graph technology, computer equipment, and storage medium (application number CN202510889315.4). Here, hash joins are used as an example; the same applies to index joins and nested joins.

[0176] The output cardinality of the graph adjacency operator depends on the average degree and traversal depth of the edge type, while the output cardinality of the vector retrieval operator depends on the number of vector retrieval returns and the index type. Both output cardinities are equally incorporated into the join order enumeration framework. The optimizer can place graph adjacency results or vector retrieval results in the left or right subtree of the join tree based on global cost information, regardless of the order of the FROM clause in the extended SQL statement.

[0177] In this embodiment, the relation table users contains approximately 1 million rows of data, the two-hop graph adjacency traversal is estimated to output approximately 100,000 rows, and the vector precise retrieval outputs 100 rows.

[0178] If executed according to the order of the FROM clause: (users ⋈ graph_2hop) ⋈ vector, where ⋈ is the join operator in relational algebra, indicating the meaning of JOIN, the relational table and the adjacency result of the two-hop graph may produce about 100,000 intermediate JOIN results.

[0179] The cross-model unified cost model of this invention identifies that the vector retrieval results contain only 100 rows, and therefore rearranges the connection order to: (vector ⋈ users) ⋈ graph_2hop. First, the 100 rows of vector retrieval results are connected to the relation table, and then connected to the graph adjacency results, reducing the size of the intermediate results from approximately 100,000 rows to approximately 100 rows, thereby reducing the computational overhead of subsequent connections and sorting. This kind of cross-model global connection order optimization capability is generally difficult to achieve in existing solutions: in the single-kernel extension stacking mode, the cost estimation logic of each extension is hard-coded inside the extension, and the database optimizer lacks a unified interface to simultaneously receive and compare the cardinality of the outputs of the three types of operators, thus preventing the exchange of cardinality information; while in the separate multi-system federated mode, the optimization decisions are scattered across different system instances and cannot participate in global enumeration.

[0180] S105, the optimizer selects the physical implementation method for the relation scan operator, graph adjacency operator, vector retrieval operator and connection operator according to the target connection order, and generates the target physical execution plan.

[0181] For graph adjacency operators, the choice depends on the graph size and traversal depth: adjacency list index scan is suitable for small graphs or shallow traversals with one to two hops; CSR batch adjacency access is suitable for high-throughput scenarios of large graphs, locating the outgoing edge range of the target vertex through a continuous offset array; incremental breadth-first traversal is suitable for deep traversals or scenarios that need to terminate early based on the LIMIT condition.

[0182] In this embodiment, the two-hop graph adjacency operator uses CSR batch adjacency access, and the one-hop graph adjacency operator uses adjacency list index scan.

[0183] For vector retrieval operators, selection is based on vector retrieval type, vector set cardinality, vector dimension, existing vector index, index recall rate, and memory budget.

[0184] The Flat precise search operator is selected for precise vector retrieval. Flat precise search calculates the full distance and accelerates it using SIMD instructions, maintaining k current nearest neighbor results to ensure that the global nearest neighbor is returned.

[0185] Approximate vector retrieval operators can be selected from HNSW index retrieval, IVFFlat index retrieval, or IVFPQ index retrieval. When the vector set size is less than 1000, Flat retrieval can be selected directly; when an HNSW index has been established and the recall rate meets the requirements, HNSW index retrieval can be selected; when an IVFFlat index has been established and the recall rate meets the requirements, IVFFlat index retrieval can be selected; when memory budget is tight, IVFPQ compressed index retrieval can be selected.

[0186] The target physics execution plan can also employ a two-stage vector retrieval. This two-stage vector retrieval first obtains k × oversample_factor candidate results through approximate vector indexing, then performs precise distance calculation and reordering on these candidate results, finally returning the Top-K results. This serves as a trade-off between pure approximate retrieval and pure flat precise retrieval. Here, oversample_factor is the candidate set expansion factor, typically ranging from 3 to 5. A larger factor increases the probability that the candidate set covers the true Top-K, and the actual recall rate approaches 100%, but the computational cost of precise reordering also increases linearly.

[0187] S106, the execution engine executes relation scanning operators, graph adjacency operators, vector retrieval operators and connection operators according to the target physical execution plan to complete multi-mode hybrid retrieval.

[0188] Operators without data dependencies can be scheduled in parallel using an operator dependency graph. In this embodiment, the two-hop graph adjacency operator, the one-hop graph adjacency operator, and the vector exact retrieval operator employ CSR batch adjacency access, adjacency list index scan, and Flat exact retrieval, respectively, and can be executed in parallel in different threads within the same transaction context.

[0189] The outputs of each operator are passed to the connection operator via HybridTupleBatch. The execution engine first connects the vector similarity results with the relation scan results, then connects them with the two-hop graph adjacency results. It excludes users that user A has directly followed by through left joins, then calculates the fusion score, sorts them according to the fusion score, and extracts the Top-10 results.

[0190] The total execution latency can be approximated as: max(T_2hop, T_knn) + T_join + T_sort, where max(T_2hop, T_knn) refers to the maximum time for two-hop adjacency queries and precise computation of vector KNN, representing the longer execution time of the left and right subtrees, based on the latest end time; T_join refers to the time spent joining the result set; and T_sort refers to the time spent sorting the final result set.

[0191] When the intersection of the graph adjacency results and the vector similarity results is empty, the number of vector retrieval returns k can be increased and the retrieval can be re-executed; during graph traversal, Bloom Filter can be used to remove duplicates from loops; when the index is missing, the index can be built online; when there is insufficient memory for join processing, Grace Hash Join can be used to write some intermediate results to external storage.

[0192] During the execution phase, a global transaction identifier (XID) is assigned to the extended SQL statement. Relational tuples, graph edge records, and vector entries all carry creation and deletion transaction identifiers. Each physical operator reads data according to the same MVCC visibility rules, ensuring that relational data, graph data, and vector data correspond to the same transaction snapshot.

[0193] Through S101 to S106, relational, graph, and vector retrieval semantics can be expressed in a single extended SQL statement and form a unified logical query plan tree in the database kernel; the output cardinality and execution cost of relational scan operators, graph adjacency operators, and vector retrieval operators can be compared in a unified cost space; the three types of operators can jointly participate in hybrid join order enumeration, thereby selecting the target join order with the lower total cost and completing unified execution.

[0194] Example 2

[0195] This embodiment uses the users relation table, user_profiles vector data, follow_graph graph data, and the query target "find users with similar interests from user A's second-degree connections" from Embodiment 1.

[0196] Unlike the execution method in Embodiment 1, which directly performs graph adjacency retrieval and vector retrieval separately and then finds the intersection through connection, this embodiment further refines S105 and S106 of Embodiment 1: when generating and executing the target physical execution plan, a graph-aware vector partition joint index is used, so that graph adjacency constraints are first used to limit the vector retrieval range. This joint index is an optional physical execution method in the overall process of Embodiment 1.

[0197] In the traditional execution method, the graph adjacency operator outputs the set of reachable nodes R, and the vector retrieval operator outputs the set of candidates S. Then, the join operation is used to obtain: R ∩ S. Under the data scale of Example 1, the two-hop graph adjacency traversal may output about 100,000 nodes, the vector retrieval returns 100 candidates, and the final intersection may only contain dozens of results. Therefore, there are a lot of invalid vector distance calculations and intermediate result processing.

[0198] To address this problem, this invention designs a joint index that collaboratively organizes graph community structure and vector index space. For example... Figure 3As shown, the graph-aware vector partitioning joint index comprises a three-layer structure. The first layer divides graph vertices into multiple connected communities using a community detection algorithm, establishing a CommunityMap[vertexId] → communityId mapping. The second layer independently performs k-means clustering on the vertex vectors within each community, establishing a double-key index of (communityId, clusterId) → vertex set, while maintaining the centroid vector of each sub-cluster. The third layer constructs an independent local vector index (local HNSW or Flat) for each sub-cluster, and additionally builds a cross-community fast-path index covering the top 1% of high-traffic vertices in each community.

[0199] In this embodiment, firstly, the Louvain community detection algorithm is applied to the millions of vertices in the follow_graph, dividing the vertices into approximately 200 communities, each containing approximately 5000 vertices, and establishing a CommunityMap[vertexId] → communityId mapping relationship. Secondly, k-means clustering is performed on the 384-dimensional vertex vectors within each community, with the number of clusters k' set to k' ≈ number of vertices in the community / 10, and a (communityId, clusterId) → vertex set double-key index is established, while maintaining the centroid vector of each sub-cluster. Thirdly, a local Flat index or a local HNSW index is constructed for the vertex vectors in each sub-cluster, and an additional cross-community fast-path index is constructed. The cross-community fast-path index covers the top 1% of high-traffic vertices in each community, used to supplement candidate results when the results returned by the local index are insufficient.

[0200] During the query phase, a graph adjacency operation is first performed to obtain the set of reachable nodes R. The CommunityMap is then queried to determine the set of communities covered by R. Retrieval is performed only on the local vector indices of relevant communities, compressing the search space from the full N to 5%-20% of the target communities. The candidate set naturally satisfies graph adjacency constraints, eliminating the need for additional JOIN filtering. If the local results are insufficient (k), they are automatically supplemented through a fast cross-community channel. Simultaneously, relational filtering conditions (such as age > 18) can pre-filter vertices at the community level, further narrowing the scope.

[0201] In this embodiment, the two-hop graph adjacency operator is first executed according to the graph association constraints of Embodiment 1, resulting in approximately 100,000 reachable nodes. Then, the communities covered by the reachable nodes are determined using a CommunityMap, which in this embodiment can cover 3 to 8 communities. Subsequently, vector retrieval is performed only in the local vector indices corresponding to these communities. Through the above processing, the vector search space is reduced from 1 million vectors to 50,000 to 200,000 vectors. Candidate results are constrained by the graph community range, eliminating the need to first perform a retrieval on all vectors and then perform large-scale connection filtering with the 100,000 graph adjacency results. When the number of candidate results returned by the local vector index is less than k, candidate results are supplemented through a cross-community fast-path index. Relationship filtering conditions, such as age greater than 18, can also pre-filter vertices at the community level to further narrow the vector retrieval range.

[0202] In this embodiment, the community scope of local vector retrieval is determined based on the graph adjacency results, thus creating a data dependency between the graph adjacency operator and the vector retrieval operator. When enumerating the valid connection order in S104, the optimizer excludes candidate orders that do not satisfy this data dependency, and in S105 generates a target physical execution plan where the graph adjacency operator is executed first, followed by the local vector retrieval. For example, the independent approach requires full distance calculation across 1 million × 384 dimensions, then joins 100 rows of vectors with 100,000 rows of graph results; the combined index only retrieves 50,000-200,000 target vectors, reducing distance calculation by 80%-95% and eliminating the bloat of intermediate JOIN results.

[0203] When vertices, graph edges, or vectors change, you can choose periodic incremental synchronization or real-time updates, each with its own advantages and disadvantages. Real-time updates cannot guarantee efficiency, while scheduled updates can balance efficiency and user experience. The window period can be set by the user, ranging from hundreds of milliseconds to several hours.

[0204] Example 3

[0205] This embodiment uses the same data model, user recommendation scenario, and overall query optimization process as Embodiment 1. Unlike Embodiment 1, which uses VECTOR_KNN for exact vector retrieval in its extended SQL, this embodiment sets the vector retrieval operator to VECTOR_ANN, or, as in S105 of Embodiment 1, the optimizer selects HNSW, IVFFlat, or IVFPQ approximate vector index retrieval for the vector retrieval operator. Therefore, this embodiment further explains the quality detection, adaptive degradation, background monitoring, and index recovery process following the selection of approximate vector index retrieval in S105 and the execution of the approximate vector index retrieval in S106 of Embodiment 1.

[0206] like Figure 4 As shown, the adaptive degradation and error correction mechanism includes three levels: query quality detection, background actual recall rate monitoring, and optimizer precision awareness selection.

[0207] After the approximate vector index retrieval is completed, the distance ratio (distance_ratio) and the result return density (edge_density) are calculated based on the vector similarity results. The distance ratio is expressed as: distance_ratio = distance to the farthest neighbor / distance to the nearest neighbor. When the nearest neighbor distance is 0, a precise check is triggered directly, or a preset positive lower limit is used to replace the nearest neighbor distance in the ratio calculation. This metric reflects the distribution span of the retrieval results in the distance space. The result return density is expressed as: edge_density = actual number of returned results / k. This metric reflects the coverage of the approximate vector index on the results of the current query. When distance_ratio exceeds a preset distance threshold, such as 5.0, or edge_density is lower than a preset density threshold, such as 0.5, the system triggers degradation processing.

[0208] The downgrade process includes initiating Flat precise search to perform precise verification, taking the union of the approximate vector index search results and the Flat precise search results, recalculating the precise distance between the candidate vectors in the union, and reordering them according to the precise distance before extracting the Top-K results.

[0209] In the first scenario, user B's query vector is located at the edge of a high-density region in the interest space. The Top-10 distances returned by the HNSW index are: [0.12, 0.13, 0.15, 0.18, 0.21, 0.48, 0.51, 0.55, 0.59, 0.62]. The system calculates: distance_ratio = 0.62 / 0.12 = 5.17. Since 5.17 is greater than the preset distance threshold of 5.0, the system initiates a precise Flat retrieval in another thread within the same transaction context. Precise verification reveals that the HNSW index missed two true nearest neighbors with distances of 0.32 and 0.41. The system merges the HNSW retrieval results with the precise Flat retrieval results and reorders them, thereby correcting the approximate retrieval results.

[0210] In the second scenario, user C's query vector is located in a sparse edge region. The HNSW index only returns 4 results when ef_search=64, while the query requires 10 results. In this case, edge_density = 4 / 10 = 0.4. Since 0.4 is lower than the preset density threshold of 0.5, the system initiates Flat precise retrieval to complete the Top-10 results.

[0211] The system maintains the actual recall rate for each vector index using a sliding window: recall_actual = |ANN ∩ KNN| / k. Here, ANN represents the approximate vector index retrieval results, and KNN represents the flat exact retrieval results.

[0212] In this embodiment, the two downgrade events are recorded in the sliding window corresponding to the HNSW index. The system detected that in the most recent 100 queries, the downgrade rate increased from 3% to 12%, while the actual recall rate decreased from 0.97 to 0.91.

[0213] When the average actual recall rate within the sliding window consistently falls below 90% of the index's theoretical recall rate, the optimizer reduces the selection priority of that vector index during the query optimization phase and triggers an asynchronous background rebuild. During the rebuild, the original vector index continues to provide retrieval services; once the new index is built, it is atomically switched to replace the original index.

[0214] At the optimizer level, the exact vector retrieval operator is bound to the Flat exact retrieval; the approximate vector retrieval operator selects between pure approximate vector index retrieval, a two-stage retrieval combining approximate coarse screening and exact rearrangement, and directly using Flat exact retrieval, based on the estimated accuracy gap.

[0215] After reconstruction, the system atomically switches the new index to the service index. This forms a closed-loop guarantee mechanism encompassing query quality checks, background recall monitoring, index reconstruction, and index switching.

[0216] Through the above implementation methods, the present invention can uniformly support mixed retrieval of relational, graph and vector data models in the database kernel, and improve the consistency, execution efficiency and reliability of vector retrieval results of cross-model queries through unified logical plan tree, cross-model unified cost model, global join order reordering, adaptive physical execution strategy, ANN / KNN error correction mechanism, graph-aware vector partition joint index and unified transaction snapshot mechanism.

[0217] This invention can also be implemented using an electronic device. The electronic device includes a processor and a memory. The memory stores a computer program. When the processor executes the computer program, it implements the relational, graph, and vector multi-modal hybrid retrieval method based on the unified cost model in any of the above embodiments.

[0218] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the relational, graph, and vector multi-modal hybrid retrieval method based on a unified cost model in any of the above embodiments.

[0219] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described multi-modal hybrid retrieval method based on a unified cost model, encompassing relational, graph, and vector data.

Claims

1. A multi-modal hybrid retrieval method based on a unified cost model, encompassing relational, graph, and vector models, applied to a database kernel, characterized in that: Includes the following steps: Obtain a multi-model query expression, which includes relational data query conditions, graph association constraints, and vector similarity retrieval conditions; During the query parsing phase, the multi-model query expression is parsed into a unified logical query plan tree. The unified logical query plan tree includes a relation scan operator, a graph adjacency operator, a vector retrieval operator, and a connection operator. The outputs of the graph adjacency operator and the vector retrieval operator both include target identifiers for connection with relational data identifiers. Obtain cross-model statistical information corresponding to the unified logical query plan tree. Based on the cross-model statistical information, estimate the output cardinality of the relation scan operator, graph adjacency operator, and vector retrieval operator respectively through the cross-model unified cost model, and estimate the execution cost of the relation scan operator, graph adjacency operator, and vector retrieval operator in a unified cost unit. During the query optimization phase, within the scope of satisfying the join conditions and semantic equivalence constraints, multiple candidate hybrid join trees formed by relation scanning operators, graph adjacency operators and vector retrieval operators are enumerated, and the total cost is calculated based on the output cardinality and execution cost of the operators in each candidate hybrid join tree. The candidate hybrid connection tree with the lowest total cost is selected as the target connection order, and the target physical execution plan is generated according to the target connection order. The execution positions of the graph adjacency operator and the vector retrieval operator in the connection tree are not restricted by the writing order or generation order in the multi-model query expression. The target physical execution plan is used to execute relation scanning operators, graph adjacency operators, vector retrieval operators, and connection operators to obtain multi-modal hybrid retrieval results.

2. The method according to claim 1, characterized in that, Multi-model query expressions are either extended SQL statements or internal query expressions derived from extended SQL statements. Extended SQL statements include relational table references, relational predicate conditions, graph adjacency operators, and vector retrieval operators. Relational predicate conditions include relational attribute query conditions and join conditions established based on relational data identifiers; The graph adjacency operator includes the source node, edge type, traversal depth, and traversal direction, and is used to return the graph adjacency result including node identifier, graph distance, and path weight. Vector retrieval operators include query vector, number of vector retrieval results, and vector retrieval type. Vector retrieval types include approximate vector retrieval type and exact vector retrieval type. Vector retrieval operators are used to return vector retrieval results that include entity identifiers and vector distances. During the query parsing phase, graph adjacency operators are parsed into graph adjacency operators, and vector retrieval operators are parsed into approximate vector retrieval operators or exact vector retrieval operators depending on the vector retrieval type. The relation scanning operator, graph adjacency operator, and vector retrieval operator exchange data through a unified intermediate result structure. The unified intermediate result structure includes columnar data blocks and optional vector data regions. Node identifiers, entity identifiers, and relation data identifiers adopt matching data types, enabling graph adjacency results and vector retrieval results to be processed by join operators, filter operators, sort operators, or aggregation operators.

3. The method according to claim 1, characterized in that, Cross-model statistics include cardinality of relation tables, selectivity of relation predicates, average degree of different edge types, size of frontier for each jump traversal, cardinality of vector sets, vector dimension, and vector index parameters. The cross-model unified cost model uses the basic CPU computation cost as the unified cost unit. The tuple processing cost and the single edge access cost are both calibrated using the unified cost unit and estimated in the following way: The output cardinality of the relation scan operator is determined based on the statistical cardinality of the relation table, or based on the statistical cardinality of the relation table and the selectivity of the relation predicate. The execution cost of the relation scan operator is determined based on the output cardinality and the cost of tuple processing. The output cardinality of the graph adjacency operator is determined based on the average degree of the edge type and the traversal depth. The execution cost of the graph adjacency operator is determined by the sum of the products of the size of the traversal front of each hop from the first hop to the traversal depth and the cost of accessing a single edge. The output cardinality of the vector retrieval operator is determined based on the number of vector retrieval results. When the vector retrieval operator uses K-Nearest Neighbors (KNN) for exact retrieval, the execution cost is determined by the product of the vector set cardinality, vector dimension, and uniform cost unit. When the vector retrieval operator uses a hierarchical navigable small-world graph (HNSW) index for retrieval, the execution cost is determined by the product of the logarithm of the vector set cardinality, the search expansion factor, vector dimension, and uniform cost unit. When the vector retrieval operator uses an inverted file planar vector (IVFFlat) index for retrieval, the execution cost is determined by the product of the ratio of the vector set cardinality to the total number of cluster buckets, the number of cluster buckets to be scanned, the vector dimension, and uniform cost unit. When the vector retrieval operator uses an inverted file index based on product quantization (IVFPQ), the execution cost is determined by the product of the ratio of the vector set cardinality to the total number of cluster buckets, the number of cluster buckets to be scanned, the equivalent compressed dimension after product quantization, and uniform cost unit. The output cardinality of the filtering operator is determined based on the output cardinality of the sub-operators and the selectivity of the filtering predicate, while the output cardinality of the connection operator is determined based on the output cardinality of the left sub-operator, the output cardinality of the right sub-operator, and the selectivity of the connection condition.

4. The method according to claim 1, characterized in that, During the query optimization phase, dynamic programming is used to enumerate multiple candidate hybrid connection trees; When there is no input data dependency among the relation scan operator, graph adjacency operator, and vector retrieval operator, the relation scan operator, graph adjacency operator, and vector retrieval operator are included as peer candidate inputs in the same connection order enumeration framework; When the source node of a graph adjacency operator depends on the result of a preceding subquery, the graph adjacency operator is restricted to the period following the preceding subquery that produced the source node; when the query vector or retrieval range of a vector retrieval operator depends on the output result of a preceding operator, the vector retrieval operator is restricted to the period following the preceding operator that produced the corresponding output result. For each candidate hybrid connection tree that satisfies the connection conditions, data dependencies and semantic equivalence constraints, the cumulative cost is calculated based on the output cardinality and execution cost of each operator, and the candidate hybrid connection tree with the minimum cumulative cost is determined as the target connection order. The target connection order allows the output of a graph adjacency operator or a vector retrieval operator to be placed in the left or right subtree of the connection tree.

5. The method according to claim 3, characterized in that, Generate the target physical execution plan based on the target connection order, including: Based on the graph data size and traversal depth, the physical implementation method of the graph adjacency operator is selected from adjacency list index scan, compressed sparse row batch adjacency access, and incremental breadth-first traversal. When the vector retrieval operator is an exact vector retrieval operator, KNN exact retrieval based on Flat full distance calculation is selected as the physical implementation method; When the vector retrieval operator is an approximate vector retrieval operator, the physical implementation method is selected from approximate vector index retrieval, two-stage vector retrieval, and KNN precise retrieval based on Flat full distance calculation, according to the vector set cardinality, vector dimension, existing vector index, index recall rate, and memory budget. Approximate vector index retrieval includes HNSW index retrieval, IVFFlat index retrieval, and IVFPQ index retrieval; Two-stage vector retrieval obtains more candidate results than the number returned by vector retrieval through approximate vector indexing, calculates the precise distance between the candidate results and reorders them to obtain the top K vector retrieval results.

6. The method according to claim 5, characterized in that, When the physical implementation of the approximate vector retrieval operator is approximate vector index retrieval, the method further includes: After the approximate vector index retrieval is completed, the distance distribution index is determined based on the ratio of the distance between the farthest neighbor and the nearest neighbor in the retrieval results, and the result coverage index is determined based on the ratio of the actual number of returned results to the number of results returned by the vector retrieval. When the distance distribution index is greater than the preset distance threshold, or the result coverage index is less than the preset coverage threshold, KNN precise retrieval using Flat full distance calculation is started in the same transaction context. The approximate vector index retrieval results and KNN precise retrieval results are merged, the precise distance is calculated for the merged candidate results, and the results are reordered according to the precise distance and the top K results are truncated. The background recall rate of the approximate vector index is monitored using a sliding window. The retrieval results of the approximate vector index and the K-nearest neighbor exact retrieval results are periodically sampled, and the actual recall rate is determined based on the ratio of the number of overlapping results in the two retrieval results to the number of vector retrieval results returned. When the average actual recall rate within the sliding window is consistently lower than the preset proportion of the theoretical recall rate of the corresponding vector index, the selection priority of the corresponding vector index in the query optimization phase is reduced, and asynchronous reconstruction in the background is triggered. During the reconstruction, the original vector index continues to provide retrieval services, and after the reconstruction is completed, the reconstructed vector index is atomically switched to the service index.

7. The method according to claim 1, characterized in that, Before executing the graph adjacency operator and vector retrieval operator, a graph-aware vector partitioning joint index is pre-constructed based on the graph data and vector data, specifically including: Establish a mapping relationship between vertex identifiers and community identifiers based on the graph community detection results; Cluster the vertex vectors within each community, establish a double-key index pointing to the vertex set by both community identifier and cluster identifier, and record the centroid vector corresponding to each cluster; construct a local vector index for the vertex vectors within each cluster; and construct a cross-community fast channel index covering a preset proportion of high-traffic vertices in each community. When performing multi-modal hybrid retrieval, the set of reachable nodes is first obtained through the graph adjacency operator. The community scope corresponding to the set of reachable nodes is determined according to the mapping relationship, and vector retrieval is performed in the local vector index corresponding to the community scope. When the number of results returned by the local vector index is less than the number returned by the vector retrieval, candidate results are supplemented by the cross-community fast channel index.

8. The method according to claim 1, characterized in that, During the execution phase, the same transaction identifier is assigned to the multi-model query expression. The relation table tuples, graph edge records, and vector entries all record the creation transaction identifier and the deletion transaction identifier. Under the same transaction snapshot, the relation scanning operator, graph adjacency operator, and vector retrieval operator perform visibility checks on relation table tuples, graph edge records, and vector entries according to a unified multi-version concurrency control visibility rule, so that the multi-modal mixed retrieval results correspond to the same transaction snapshot; For relational scan operators, graph adjacency operators, and vector retrieval operators that do not have input data dependencies, they are executed in parallel in different threads within the same transaction context based on operator dependencies.

9. An electronic device, characterized in that, include: One or more processors; Memory; And one or more programs, wherein the programs are stored in memory and configured to be executed by one or more processors, wherein when the programs are executed by the processors, they implement the relational, graph and vector multimodal hybrid retrieval method based on the unified cost model of any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When a computer program is executed by a processor, it implements the multi-modal hybrid retrieval method based on a unified cost model of relations, graphs, and vectors as described in any one of claims 1-8.

Images