Multi-metric vector uniform indexing method and device, electronic equipment and medium

Abstract

This specification provides a unified indexing method, apparatus, electronic device, and medium for multiple metric vectors. The method includes: in the index creation phase, constructing a shared graph topology structure decoupled from similarity metrics; in the query execution phase, enabling the query optimizer to generate an execution plan containing a single index scan node and generating a corresponding metric type identifier; performing graph traversal based on the shared graph topology structure, and if no match is found, calling the underlying calculation kernel function corresponding to the target similarity metric type to perform lazy distance calculation and writing it into a shared memory cache pool; dynamically adjusting the pruning threshold and search width parameters of the graph traversal, continuing to traverse the shared graph topology structure until the termination condition is met, and outputting the final candidate node set; and returning the final candidate node set and its corresponding distance values ​​to the query executor layer of the database system through the index scan callback interface in the custom index access method plugin, and outputting the final query result set.

Description

Technical Field

[0001] This document relates to the field of data retrieval and indexing technology, and in particular to a unified indexing method, device, electronic device and medium for multi-metric vectors. Background Technology

[0002] Currently, PostgreSQL-based vector retrieval extensions (such as pgvector) primarily employ graph / inverted index structures like HNSW or IVFFlat to accelerate nearest neighbor searches for high-dimensional vectors. In real-world business scenarios, different applications have varying requirements for similarity calculations, such as L2 Euclidean distance, cosine similarity, and inner product (IP). Existing technologies typically employ two implementation paths: one is parallel deployment of multiple indexes, where a separate physical index file is created for each similarity metric, and queries are routed to the corresponding index based on the target metric; the other is full runtime computation, which maintains only one native metric index. When querying a non-native metric, the index pruning mechanism is bypassed, and a full vector scan or approximate conversion based on the base distance is performed. Traditional vector indexes (such as HNSW) rely on the mathematical properties of specific metrics (such as triangle inequalities, distance monotonicity, and spatial separability) for edge connection optimization and hierarchical pruning during the construction phase. Therefore, their topology (node ​​connection relationships, edge weights, and hierarchical distribution) is strongly bound to the similarity metric used during construction, making cross-metric reuse impossible.

[0003] Among the aforementioned technologies, the multi-index parallel deployment scheme leads to a significant increase in disk and memory usage, and write and update operations require the simultaneous maintenance of multiple sets of indexes, severely slowing down transaction commit speed. Meanwhile, the runtime full computation scheme, due to the inability to reuse index pruning characteristics, results in low CPU cache hit rates, a surge in the number of traversed nodes, and a significant deterioration in query latency under high concurrency. Furthermore, because the index topology and the mathematical properties of metrics are deeply coupled, the industry has long held the technical bias that "supporting multiple metrics inevitably comes at the cost of sacrificing storage efficiency or query performance." This makes it difficult for vector databases to achieve efficient and unified retrieval of multiple similarity metrics under a single index architecture, hindering their large-scale deployment and operational efficiency in multimodal and multi-task scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide a unified indexing method, apparatus, electronic device, and medium for multi-metric vectors, aiming to solve the aforementioned problems in the prior art.

[0005] This invention provides a unified indexing method for multiple metric vectors, applied to a database system, comprising: Step 1, in the index creation stage, extracting the spatial distribution features of the input vector set and constructing a shared graph topology structure decoupled from the similarity metric, wherein only the node identifier and neighbor pointer relationship are persistently stored in the shared graph topology structure; Step 2, in the query execution stage, returning a unified cost estimation model through the cost estimation callback function in the registered custom index access method plugin, enabling the database system's query optimizer to generate an execution plan containing a single index scan node, and parsing the target similarity metric type corresponding to the query request to generate the corresponding metric type identifier; Step 3, performing graph traversal based on the shared graph topology structure, during which the triple consisting of the source node identifier, target node identifier, and metric type identifier is used as the composite key to calculate the distance value in a preset shared memory cache pool. Step 1: If a match is found, the distance value is read from the shared memory cache pool. If no match is found, the underlying kernel function corresponding to the target similarity metric type is called to perform lazy distance calculation, and the calculated distance value is written to the shared memory cache pool. Step 4: Based on the mathematical boundary characteristics of the target similarity metric type and the real-time cache hit rate of the shared memory cache pool, the pruning threshold and search width parameters of the graph traversal are dynamically adjusted. The shared graph topology structure is traversed again based on the dynamically adjusted pruning threshold and search width parameters until the termination condition is met, and the final candidate node set is output. Step 5: Through the index scan callback interface in the custom index access method plugin, the final candidate node set and its corresponding distance value are returned to the query executor layer of the database system, so that the query executor layer can perform visibility filtering in combination with the multi-version concurrency control snapshot of the current transaction and output the final query result set.

[0006] This invention provides a unified indexing device for multiple metric vectors, comprising: The topology construction module extracts the spatial distribution features of the input vector set during the index creation phase, constructing a shared graph topology structure decoupled from the similarity metric. This shared graph topology structure only persistently stores node identifiers and neighbor pointer relationships. The plan generation and parsing module, during the query execution phase, returns a unified cost estimation model through a cost estimation callback function in a registered custom index access method plugin. This enables the database system's query optimizer to generate an execution plan containing a single index scan node. It also parses the target similarity metric type corresponding to the query request and generates the corresponding metric type identifier. The lazy traversal module performs graph traversal based on the shared graph topology structure. During traversal, it uses a triplet consisting of the source node identifier, target node identifier, and metric type identifier as a composite key to query the distance value in a preset shared memory cache pool. If a match is found, the distance value is read from the shared memory cache pool; otherwise, the underlying calculation kernel function corresponding to the target similarity metric type is called to perform lazy distance calculation, and the calculated distance value is written to the shared memory cache pool. The adaptive pruning module dynamically adjusts the pruning threshold and search width parameters of graph traversal based on the mathematical boundary characteristics of the target similarity metric type and the real-time cache hit rate of the shared memory cache pool. It then continues traversing the shared graph topology according to the dynamically adjusted pruning threshold and search width parameters until the termination condition is met, outputting the final candidate node set. The result return module returns the final candidate node set and its corresponding distance values ​​to the database system's query executor layer via the index scan callback interface in the custom index access method plugin. This allows the query executor layer to perform visibility filtering based on the multi-version concurrency control snapshot of the current transaction, outputting the final query result set.

[0007] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the above-described multi-metric vector unified indexing method.

[0008] This invention also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-described unified indexing method for multi-metric vectors.

[0009] In this embodiment of the invention, during the index creation phase, a shared graph topology structure that only stores node identifiers and neighbor pointer relationships is constructed by extracting the spatial distribution features of the vector set. This decouples the index topology from the similarity metric type, completely avoiding the storage redundancy and maintenance overhead caused by building and persisting multiple sets of indexes independently for each metric in traditional solutions. During the query execution phase, a unified cost estimation model is returned using the cost estimation callback function of the custom index access method plugin, enabling the database query optimizer to generate an execution plan containing only a single index scan node. This eliminates resource waste caused by index route splitting or full table scans in multi-metric scenarios. During graph traversal, the source node... A triple consisting of the identifier, target node identifier, and metric type identifier forms a composite key for dynamic querying in a shared memory cache pool. This enables on-demand calculation and cross-metric reuse of distance values. When a match is missed, the underlying kernel function of the corresponding metric is invoked for lazy computation and the result is cached, balancing efficiency and flexibility. Simultaneously, by combining the mathematical boundary characteristics of the target similarity metric type with the real-time hit rate of the cache pool, the pruning threshold and search width parameters are adaptively and dynamically adjusted to improve traversal efficiency and recall accuracy. Finally, the candidate nodes and distance values ​​are returned to the query executor layer via the index scan callback interface. This layer, combined with multi-version concurrency control snapshots, performs visibility filtering and outputs a final result that conforms to transactional consistency. This effectively solves the problems of strong coupling and inability to share vector index topology and metrics under multiple similarity metrics, resulting in storage redundancy and huge maintenance overhead. It enables a single index structure to efficiently support multi-metric queries, significantly reducing storage consumption, minimizing computational duplication, and improving query performance and system maintainability. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of the multi-metric vector unified indexing method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall system architecture and data flow according to an embodiment of the present invention; Figure 3 This is a detailed flowchart of the shared graph index construction and dynamic weight calculation according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the integration of the PostgreSQL AM interface and the interaction with the execution plan in an embodiment of the present invention; Figure 5This is a timing diagram of the multi-metric cache pool memory structure and concurrent access in an embodiment of the present invention; Figure 6 This is a schematic diagram of a multi-metric vector unified indexing device according to an embodiment of the present invention; Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0013] As described in the background section, current PostgreSQL-based vector retrieval systems, when supporting multiple similarity metrics (such as L2, Cosine, and IP), are forced to choose between "parallel deployment of multiple indexes" and "full runtime computation": the former causes a significant increase in storage and maintenance costs, while the latter leads to a precipitous drop in query performance due to the inability to reuse the pruning capabilities of the index topology. The industry has long been trapped in the technical dilemma that "supporting multiple metrics inevitably sacrifices storage efficiency or query performance," making it impossible to achieve efficient, low-overhead, and high-concurrency multi-metric retrieval under a unified index architecture.

[0014] After long-term research, the inventors discovered that the most fundamental reason for the above-mentioned predicament lies in the fact that existing vector indexes (such as HNSW and IVFFlat) deeply bind the "construction of topology" with the "mathematical properties of distance calculation." Their node connection relationships, hierarchical strategies, and edge weights all rely on the geometric characteristics of specific metrics (such as triangle inequalities and monotonicity) for static fixation. As a result, once the index topology is established, it becomes a "dedicated structure" for a certain metric and cannot be reused by other metrics. This design couples "structure" and "computation" into an inseparable whole, making it necessary to rebuild an independent index or abandon index acceleration for any new metric, thus forming a structural technical shackle—not because multiple metrics are inherently difficult to achieve, but because the existing architecture forces each metric to have an "independent physical topology," ignoring that the "connection relationships between nodes" are essentially structural features of the vector space and are unrelated to how distance is calculated.

[0015] Based on the above findings, the inventors proposed a method to completely separate the "topology" of the index from "distance calculation," constructing a shared graph topology consisting solely of node identifiers and neighbor pointer relationships, independent of any similarity metric. During queries, instead of relying on pre-calculated distance values, the target metric type is dynamically identified by parsing the operators used in the query. During graph traversal, distance values ​​are queried or calculated on demand using only the triplet "source node ID, target node ID, and metric type identifier." Simultaneously, the pruning threshold and search width of the graph traversal are dynamically adjusted based on the mathematical boundary characteristics of the metric (such as value range and monotonicity) and the real-time cache hit rate, thereby achieving efficient retrieval of any metric without rebuilding the topology. This concept breaks through the traditional paradigm of "one metric = one topology," transforming the index from a "statically bound dedicated structure" to a "dynamically adaptable general framework." This allows the same physical topology to be shared by an unlimited number of metrics, and distance calculations only occur when necessary, avoiding redundant storage and repeated calculations.

[0016] Guided by this inventive concept, the inventors proposed a unified indexing method for multi-metric vectors, a unified indexing device for multi-metric vectors, an electronic device, and a computer-readable storage medium.

[0017] Method Implementation Examples According to embodiments of the present invention, a unified indexing method for multi-metric vectors is provided. Figure 1 This is a flowchart of the multi-metric vector unified indexing method according to an embodiment of the present invention, such as... Figure 1 As shown, the multi-metric vector unified indexing method according to an embodiment of the present invention specifically includes: Step S101: In the index creation stage, extract the spatial distribution features of the input vector set and construct a shared graph topology structure decoupled from the similarity metric. In the shared graph topology structure, only the node identifier and the neighbor pointer relationship are persistently stored. In this embodiment, the index creation phase extracts the spatial distribution features of the input vector set to establish a connection structure that relies solely on the original coordinate relationships of the vectors. This structure does not depend on the mathematical properties of any specific similarity metric, and only stores node identifiers and neighbor pointer relationships. The extraction of spatial distribution features determines the connection method between nodes, but this connection method does not involve the calculation of distance values ​​or the assignment of weights; it only forms a topological network of relationships between nodes. Node identifiers are used to uniquely identify each vector, and neighbor pointer relationships record the direct connection objects of each node in the graph structure. Together, they constitute the physical skeleton of the index. When this structure is persisted on disk or shared memory, any distance values, edge weights, or metric-specific parameters related to similarity metrics are strictly excluded, ensuring that it retains only the pure connection form. Thus, this topological structure becomes a universal connection framework independent of the similarity type used in subsequent queries, providing a unified underlying structural foundation for the reuse of different metrics.

[0018] Step S102: During the query execution phase, a unified cost estimation model is returned through the cost estimation callback function in the registered custom index access method plugin, enabling the database system's query optimizer to generate an execution plan containing a single index scan node, and to parse the target similarity metric type corresponding to the query request and generate the corresponding metric type identifier. In this embodiment, a unified cost estimation model is returned to the database system's query optimizer through the cost estimation callback function in the registered custom index access method plugin. This model does not change with different similarity metric types, enabling the query optimizer to generate an execution plan containing a single index scan node, avoiding redundant paths such as creating multiple indexes for different similarity metrics or selecting full table scans. Simultaneously, during the execution of this execution plan, the system parses the target similarity metric type used in the query request and generates a corresponding metric type identifier based on the parsing result. This identifier is used in subsequent stages to identify the specific similarity calculation method used in the current query, ensuring that the execution process remains consistent with the query semantics.

[0019] Step S103: Perform graph traversal based on the shared graph topology. During the traversal, use the triplet consisting of the source node identifier, the target node identifier, and the metric type identifier as the composite key to query the distance value in the preset shared memory cache pool. If a match is found, read the distance value from the shared memory cache pool. If no match is found, call the underlying calculation kernel function corresponding to the target similarity metric type to perform lazy distance calculation and write the calculated distance value into the shared memory cache pool. In this embodiment, when traversing the graph based on the shared graph topology, each time a source node and its adjacent target node are involved in the traversal, a distance value query is initiated against the preset shared memory cache pool using a triple consisting of the source node identifier, the target node identifier, and the metric type identifier as the unique query key. If a matching record exists in the cache pool, the distance value stored therein is read directly to avoid duplicate calculations. If no matching record is found, the underlying calculation kernel function corresponding to the target similarity metric type specified in the current query is immediately called to perform lazy distance calculation. The result is then written into the shared memory cache pool for reuse in subsequent queries with the same triple.

[0020] Step S104: Based on the mathematical boundary characteristics of the target similarity metric type and the real-time cache hit rate of the shared memory cache pool, dynamically adjust the pruning threshold and search width parameters of the graph traversal, and continue to traverse the shared graph topology according to the dynamically adjusted pruning threshold and search width parameters until the termination condition is met, and output the final candidate node set. In this embodiment, based on the mathematical boundary characteristics of the target similarity metric type, the applicable geometric constraints for the current query are determined, such as the value range or monotonicity rules, and the pruning threshold is initialized accordingly. Simultaneously, the cache hit rate of the shared memory cache pool in the current query session is monitored in real time as an indicator of the efficiency of node distance calculation reuse. Based on the dual information of the mathematical boundary characteristics and the cache hit rate, the pruning threshold and search width parameters during the graph traversal process are adjusted synchronously to dynamically adapt the search behavior to the geometric characteristics and cache utilization status of the current metric. After this dynamic adjustment, node traversal continues along the shared graph topology until the preset termination condition is met, and finally, a set of candidate nodes that meet the currently adjusted parameters is output.

[0021] Step S105: Through the index scan callback interface in the custom index access method plugin, the final candidate node set and its corresponding distance value are returned to the query executor layer of the database system, so that the query executor layer can perform visibility filtering in combination with the multi-version concurrency control snapshot of the current transaction and output the final query result set; In this embodiment, the final candidate node set and its corresponding distance values ​​are passed to the database system's query executor layer through the index scan callback interface in the custom index access method plugin. This allows the query executor layer to perform visibility filtering on these candidate nodes and their distance values ​​based on the multi-version concurrency control snapshot of the current transaction. This ensures that only records visible to the current transaction are retained, and the final query result set that meets transaction consistency requirements is output. The index scan callback interface, as a standardized communication channel between the plugin and the executor, is only responsible for transmitting the node IDs and corresponding distance values ​​filtered by the index traversal. Visibility filtering is performed independently by the executor based on the transaction snapshot, ensuring that the returned results strictly adhere to the database transaction isolation and consistency rules.

[0022] The technical solution of this embodiment constructs a shared graph topology structure decoupled from similarity metrics by extracting the spatial distribution features of the vector set during the index creation phase. Only the node identifier and neighbor pointer relationship are persistently stored, thereby achieving the general reuse of a single topology structure for multiple similarity metrics. This completely eliminates the storage redundancy and synchronization overhead caused by building and maintaining physical indexes independently for each metric in traditional technologies. During the query execution phase, a cost estimation model is uniformly registered through a custom index access method plugin, enabling the database optimizer to generate a single index scan execution plan and dynamically drive the graph traversal process in conjunction with the target metric type identifier. During traversal, the triple consisting of the source node, target node, and metric type identifier is used as the composite key. A shared memory cache pool is used to reuse distance values ​​across metrics and perform lazy computation, avoiding redundant computation and reducing latency. At the same time, the pruning threshold and search width are adaptively adjusted in real time based on the mathematical boundary characteristics of different metrics and the cache hit rate to improve traversal efficiency and recall accuracy. Finally, the candidate node set and corresponding distance values ​​are returned to the query executor layer through the index scan callback interface. With the help of multi-version concurrency control snapshots, visibility filtering is completed, and accurate query results are output. Therefore, this solution effectively solves the problems of storage redundancy and huge maintenance overhead caused by the strong coupling and inability to share vector index topology and metrics under multiple similarity measures. It realizes unified indexing, efficient caching, dynamic optimization and seamless integration of multi-metric vector retrieval, significantly reduces system resource consumption and improves query performance and maintainability.

[0023] Specifically, in this embodiment, during the index creation stage, extracting the spatial distribution features of the input vector set and constructing a shared graph topology structure decoupled from the similarity metric includes: extracting the underlying spatial distribution features of the input vector set, generating edge connections between nodes using a random approximate adjacency algorithm or a basic projection algorithm, constructing a multi-layer graph topology structure, and during the construction process, not pre-calculating or persistently storing any distance weight values ​​or edge weight values ​​related to a specific similarity metric, and storing the constructed shared graph topology structure as the core data carrier of the physical index in disk or shared memory. Nodes in the shared graph topology structure are stored according to the original coordinates of the vectors. The data structure of the shared graph topology structure includes: node identifiers, a hierarchy identifier assigned to each node, and a neighbor pointer array, but does not include any distance weight values.

[0024] In this embodiment, by extracting the spatial distribution features of the input vector set, a random approximate adjacency algorithm or a basic projection algorithm is used to generate edge connections between nodes, constructing a multi-layer graph topology with a hierarchical structure. A unique hierarchical identifier is assigned to each node, and a standardized data structure containing only node identifiers, hierarchical identifiers, and neighbor pointers is formed by combining the neighbor pointer array. No distance weights or edge weights related to similarity metrics are pre-calculated or persistently stored. At the same time, nodes are directly stored according to their original coordinates, thereby achieving complete decoupling between the topology and similarity metrics. This structure, as the core carrier of the physical index, not only ensures reusability and storage efficiency in cross-metric scenarios, but also provides a hierarchical and efficient unified structural foundation for dynamic pruning and adaptive search based on cache hit rate and metric mathematical characteristics in the subsequent query stage. This significantly improves the convergence speed and cache reuse rate of graph traversal, effectively solving the problems of storage redundancy and high maintenance costs caused by multi-metric coupling in traditional methods, and achieving efficient, unified, and scalable vector retrieval under multiple similarity metrics.

[0025] Specifically, in this embodiment, the database system is a PostgreSQL database system. During the index scanning phase, parsing the target similarity metric type corresponding to the query request and generating the corresponding metric type identifier specifically includes: receiving the SQL query statement submitted by the user, which contains vector fields and target similarity operators; when the executor triggers the index scan, parsing the target similarity operators in the SQL query statement through the operator family of the PostgreSQL database system, mapping them to the predefined target similarity metric type, locking the corresponding underlying vectorized distance calculation kernel function according to the target similarity metric type, loading the corresponding mathematical boundary constraints, and generating a metric type hash value.

[0026] In this embodiment, by limiting the database system to PostgreSQL and introducing an operator family mechanism during the index scanning phase, the system receives SQL query statements submitted by users that contain vector fields and target similarity operators. When the executor triggers an index scan, the system uses PostgreSQL's built-in operator family to perform semantic parsing on the similarity operators in the query, accurately mapping them to predefined target similarity metric types. Then, based on the type, it dynamically locks the corresponding underlying vectorized distance calculation kernel function, loads its unique mathematical boundary constraints, and generates a unique metric type hash value. This ensures that the triples consisting of "source node identifier, target node identifier, and metric type hash value" in subsequent cache queries have a complete and consistent semantic foundation, achieving a strong binding between the cache key and the distance calculation logic and boundary constraints. This completely avoids kernel function mismatch, boundary misuse, or cache invalidation problems caused by ambiguous metric type identification. As a result, the unified graph topology structure can still efficiently reuse cache, accurately prune, and dynamically optimize search parameters in multi-metric query scenarios. Ultimately, it achieves the goal of supporting seamless access, accurate execution, and maximum performance of multiple similarity metrics without changing the shared index structure.

[0027] Specifically, in this embodiment, step 103 includes: Based on the shared graph topology, the traversal engine starts from the starting node and sequentially visits its neighbor node list. For each pair of source node ID and target node ID, a cache key is constructed, wherein the cache key specifically includes: source node ID, target node ID, and metric type hash value; In the shared memory cache pool, using the triple as the composite key, the hash table is used to search for whether there is a valid record for the cache key. If a match is found and the cached data has not expired, the distance value is directly read; If a match is not found, the underlying calculation kernel function corresponding to the target similarity metric type is called to perform lazy distance calculation, and the calculated distance value is written to the shared memory cache pool; A cache eviction policy is adopted to evict node pairs, wherein the cache eviction policy includes: when the number of visits to a node pair within the sliding time window is greater than or equal to a preset threshold, or when it is in the top predetermined percentage of global access frequency statistics, it is defined as a high-frequency node pair and given priority to reside in the cache.

[0028] In this embodiment, the traversal engine, based on a shared graph topology structure decoupled from similarity metrics, visits the neighbor node list level by level starting from the starting node. For each pair of source node ID and target node ID, a triplet composite key containing the source node ID, target node ID, and metric type hash value is constructed. The corresponding distance value record is accurately searched in the shared memory cache pool through the hash table. If a match is found and the cache has not expired, the existing calculation result is directly reused. Otherwise, the underlying kernel function of the corresponding metric type is called to perform lazy calculation and the result is written to the cache. At the same time, based on whether the access frequency of the node pair within the sliding time window reaches a preset threshold or whether it belongs to a predetermined percentage of the global access frequency, high-frequency node pairs are dynamically identified and preferentially retained in the cache. This enables cache reuse across sessions and queries, effectively suppresses repeated calculation of the same node pair in multi-metric concurrent scenarios, reduces invalid memory replacement and computational resource waste, and significantly improves cache hit rate and overall system response efficiency.

[0029] Specifically, in this embodiment, if a match is not found, the underlying calculation kernel function corresponding to the target similarity metric type is called to perform lazy distance calculation, and the calculated distance value is written to the shared memory cache pool. This includes: calling the corresponding underlying distance calculation kernel function according to the target metric type, using the Single Instruction Multiple Data Stream (SIMD) instruction set to perform parallel vectorization calculation on the multi-dimensional coordinate vector to generate an accurate distance value; after the calculation is completed, the distance value, the current timestamp, and the access counter are initialized to 1, ready to be written to the cache pool; in the shared memory cache pool, the corresponding cache slot is located; if the slot is already occupied, a hybrid eviction policy of LRU list priority and hot-weighted access frequency is used to determine whether to replace and write new data to the slot. The cache fields of the new data include: source node ID, target node ID, metric type hash value, distance value, timestamp, and access counter; if it is the first access, the counter is reset to 1; if it is a repeated access, the counter is incremented, and the updated status of the shared memory cache pool is output.

[0030] In this embodiment, when the distance value of the target node pair is not found in the shared memory cache pool, the underlying computation kernel function matching the target similarity metric type is invoked. The single-instruction multiple-data stream (SIMD) instruction set is used to perform parallel vectorization operations on the multi-dimensional vector, efficiently generating an accurate distance value. This distance value, the current timestamp, and an access counter initialized to 1 are then written into the cache pool. During the writing process, the cache slot is located based on the triple composed of the source node ID, the target node ID, and the metric type hash value. If the slot is already occupied, a hybrid eviction policy combining LRU list priority and access frequency weighting is used. The system makes decisions by comparing the access counter value of the data to be inserted with the access frequency and timestamp of the candidate data. It intelligently determines whether to replace the old record, thereby ensuring that frequently accessed node pairs have a higher resident priority in the cache, while low-frequency or long-unaccessed data is evicted first. For the same node pair that is accessed repeatedly in the future, the system automatically increments its access counter, further increasing its weight in the eviction policy. This realizes the dynamic tilt of cache resources towards high-hot data, effectively alleviating the hit rate fluctuations and redundant calculation overhead caused by limited cache capacity, and significantly improving the response stability and overall system performance under high-concurrency queries.

[0031] Specifically, in this embodiment, step 104 includes: initializing the spatial constraint boundary as the initial pruning threshold according to the mathematical boundary characteristics corresponding to the target metric type; wherein, when the target metric type is a distance metric, the spatial constraint boundary is the safe pruning radius; when the target metric type is a similarity metric, the spatial constraint boundary is the upper bound of the angle or the similarity threshold; and calculating the real-time cache hit rate based on the cache hit count of the current query session and the total number of cumulative distance calculations; wherein, the formula for calculating the real-time cache hit rate is: Hit rate = Cache hit count / The system calculates the total number of distance calculations in the current query session. It compares the real-time cache hit rate with a preset hit rate range and dynamically scales the parameters: if the real-time cache hit rate is lower than the preset lower limit, it expands the search width parameter and relaxes the initial pruning threshold to increase the breadth of graph traversal; if the real-time cache hit rate is higher than the preset upper limit, it shrinks the search width parameter and tightens the initial pruning threshold to accelerate path convergence in graph traversal. The base value of the search width parameter and the upper and lower limits of the hit rate range are loaded into shared memory during index initialization and can be dynamically overridden via session-level configuration commands. In each step of graph traversal, the dynamically scaled search width parameter and pruning threshold are used to compare the distances and assess the potential of the current node's neighboring nodes, retaining only nodes that meet the pruning conditions to enter the next candidate set. When the number of explored nodes reaches the upper limit of the dynamically scaled search width parameter, or the current candidate set is empty, or all candidate branches are eliminated by the pruning threshold, a termination condition is triggered, and the final candidate node set is output.

[0032] In this embodiment, the spatial constraint boundary is initialized as the initial pruning threshold based on the mathematical boundary characteristics of the target similarity metric type. When the metric type is distance, a safe pruning radius is used; when it is similarity, an upper bound of the angle or a similarity threshold is used to ensure that the pruning strategy is accurately adapted to the mathematical essence of various metrics. Simultaneously, the cache hit rate is calculated in real time based on the ratio of the number of cache hits to the total number of cumulative distance calculations in the current query session, and dynamically compared with a preset hit rate range. When the hit rate is below the lower limit, the search width is expanded and the pruning threshold is relaxed to enhance the exploration breadth; when the hit rate is above the upper limit, the search width is reduced and the pruning threshold is tightened to accelerate path convergence. The base value of the search width and... The hit rate threshold is loaded into shared memory during index initialization and supports session-level dynamic overwriting, making parameter adjustment flexible and configurable. In each step of graph traversal, the distance comparison and potential evaluation of neighboring nodes are combined with the dynamically adjusted search width and pruning threshold. Only nodes that meet the pruning conditions are retained in the candidate set. Termination is triggered when the number of explored nodes reaches the upper limit, the candidate set is empty, or all branches are pruned and eliminated. This achieves adaptive optimization of search behavior under different metric types, significantly reducing redundant distance calculation overhead and traversal latency without sacrificing recall. Ultimately, it achieves the synergistic effect of efficient and accurate querying and resource saving in a unified index structure under multi-metric environments.

[0033] Specifically, in this embodiment, step 105 includes: encapsulating the final candidate node set and its corresponding distance values ​​into output records conforming to the current relational database tuple format through the index scan callback interface registered in the custom index access method plugin; passing the output records to the query executor layer of the relational database, whereby the query executor layer obtains a multi-version concurrency control snapshot of the current query transaction; based on the multi-version concurrency control snapshot, performing transaction visibility filtering on the node data in the output records to remove invisible data that does not meet the current transaction isolation level; and globally reordering the node data after transaction visibility filtering according to the distance values ​​corresponding to the target similarity metric type, extracting the top K nodes as the final query result set, and returning it to the upper-layer application according to the standard query protocol.

[0034] In this embodiment, the final candidate node set and its corresponding distance values ​​are encapsulated into output records conforming to the tuple format of a relational database. These records are then passed to the query executor layer via the index scan callback interface of the custom index access method plugin. This allows the intermediate results of vector retrieval to seamlessly adapt to the standard data processing flow of the database kernel. The query executor layer performs transaction visibility filtering on the node data in the output records based on the multi-version concurrency control snapshot of the current transaction, eliminating data that is not visible due to the concurrency transaction isolation level, ensuring that the results meet ACID consistency requirements. Subsequently, the system performs a global reordering of the filtered node data based on the distance values ​​corresponding to the target similarity metric type, and extracts the top K most similar nodes as the final query results. These results are then returned to the upper-layer application through a standard query protocol. This achieves deep integration of vector similarity search results with the native tuple structure, transaction mechanism, and SQL sorting logic of the relational database. This effectively solves the problem of vector retrieval output being disconnected from database query semantics and unable to participate in standard transaction processing and result sorting in traditional methods. It achieves the effect of efficiently and accurately outputting Top-K vector retrieval results that conform to SQL semantics while ensuring data consistency and isolation.

[0035] The technical solutions of the embodiments of the present invention will be illustrated below with reference to the accompanying drawings.

[0036] The multi-metric vector unified indexing method based on shared topology and dynamic weights provided in this invention is applicable to the index management module of a vector database. Figure 2 This is a schematic diagram of the overall system architecture and data flow according to an embodiment of the present invention. Figure 2 This diagram illustrates the data flow path of the technical solution of this invention within the PostgreSQL kernel. The SQL Parser receives the user query statement; it is passed to the operator parsing module to extract the target vector operator (such as <->); a custom AM interface layer is called to trigger an index scan; the shared graph topology manager loads nodes and adjacency relationships; the result is routed to the dynamic weight calculation engine to perform metric identification and lazy evaluation; the result is written to a multi-metric cache pool for reuse; the adaptive pruning scheduler adjusts the traversal parameters in real time based on the hit rate and mathematical boundaries; and the final result is returned to the Executor layer for output. Each module in the diagram is represented by a rectangle, and the data flow is connected by solid lines with arrows, clearly demonstrating the closed-loop process of "parsing-loading-calculation-caching-pruning-return".

[0037] Figure 3 This is a detailed flowchart of the shared graph index construction and dynamic weight calculation according to an embodiment of the present invention. Figure 3As shown, the starting node is the "vector set input". After "topology connection generation (metric-independent)", node IDs and neighbor pointer arrays are generated and written to disk. Upon query triggering, the process enters the "metric type identification" branch. Then, a "cache query" is performed, with two paths: "hit" and "miss". If hit, the distance value is directly read and returned; if miss, the process enters the "SIMD distance calculation" module to calculate as needed, and then backfills the cache. The calculation result enters the "pruning judgment" diamond decision box, where a threshold determines whether to "retain" or "eliminate" this branch. Finally, the process loops along the "next hop traversal" arrow until the graph search terminates. The diagram uses standard flowchart symbols (…). Figure 3 In this diagram, rounded rectangles represent start / end points, rectangles represent processing steps, diamonds represent decision points, and arrows represent control flow. A detailed explanation follows.

[0038] As described above, during the index creation phase, the underlying spatial distribution features of the vector set are extracted to construct a shared graph topology decoupled from specific similarity metrics. Nodes are stored using their original vector coordinates, and edge connections are generated based on fundamental projections or random approximate adjacency. This shared graph topology is the core data carrier of the physical index. The physical index only persistently stores this topology on disk / shared memory, containing node IDs, level identifiers, and neighbor pointer arrays, without pre-calculating or persistently storing any metric-specific distance weights. By decoupling metric dependencies, the same set of physical connections can be reused by any similarity metric.

[0039] Upon receiving an SQL vector query request, the target similarity type is resolved using the PostgreSQL operator family (e.g., <-> corresponds to L2, <#> to IP, and <=> to Cosine). The purpose of identifying the similarity type is to: ① determine which underlying distance calculation kernel function to call subsequently; ② determine the mathematical boundary constraints corresponding to the metric to guide pruning; and ③ generate a hash key for the metric type used in cache lookups. During the index scan, the traversal engine calls the corresponding distance calculation function in real time based on the current query metric, performing lazy evaluation on the node pairs passed through. The calculation process is accelerated by vectorization using the SIMD instruction set, avoiding scalar loop overhead.

[0040] A metric weighted cache area is opened in the PostgreSQL shared buffer, and a hybrid eviction strategy of LRU and access popularity weighting is adopted. The node pair refers to the tuple formed by the current candidate node (source node) and its adjacent node (target node) in the adjacency list during graph traversal. When the number of accesses of a node pair within a sliding time window (e.g., 60 seconds) is greater than or equal to a preset threshold (e.g., 5 times), or it is in the top 20% of the global access frequency statistics, it is defined as a high-frequency node pair and is given priority to reside in the cache. Hit and miss determination mechanism: The cache key is composed of a triple <source node ID, target node ID, metric type Hash>. When querying, the cache slot is located through the hash table. If there is valid data in the slot and it has not expired (within TTL), it is determined to be a hit and the distance value is returned directly; if the slot is empty, the data has been eliminated, or the metric Hash does not match, it is determined to be a miss, triggering the dynamic calculation in step (2), and the result is filled back into the cache pool. The cache table structure includes: <Source Node ID, Target Node ID, Metric Type (Hash), Distance Value, Timestamp, Access Counter> During the traversal, the pruning threshold and search depth (ef_search) are not fixed, but dynamically scaled according to the mathematical boundary characteristics of the current metric type and the real-time cache hit rate. Here, the "metric type" is completely consistent with the similarity type identified in step (2), ensuring that the pruning strategy is aligned with the mathematical properties of the query metric.

[0041] The mathematical boundary characteristics are considered as follows: different metrics have different theoretical ranges and geometric properties (e.g., Cosine ∈ [-1, 1] and satisfies angular monotonicity, L2 ∈ [0, +∞) and satisfies the triangle inequality). The system has built-in boundary functions for each metric, which are used to initialize pruning thresholds (e.g., setting an initial safety radius or an upper bound for the cosine angle), and based on this, determine whether neighboring nodes have the potential to surpass the current optimal solution.

[0042] Cache hit rate calculation: Hit rate = Number of cache hits / Total number of cumulative distance calculations in the current query session, calculated in real time by sliding window or single query period.

[0043] Dynamic adjustment logic: When the hit rate is lower than the preset lower limit (e.g., 60%), it indicates that the cache is not effectively covering the current path. The system automatically increases the width of ef_search and relaxes the pruning threshold to increase the exploration breadth and ensure recall. When the hit rate is higher than the preset upper limit (e.g., 85%), it indicates that the cache has been sufficiently accelerated. The system tightens the pruning boundary and reduces the search depth to reduce invalid traversals. The preset parameters (base ef_search, upper and lower hit rate thresholds) are loaded into shared memory during the index initialization phase and can also be dynamically overridden via session-level SET commands, serving as a baseline for adaptive adjustment.

[0044] The above logic is encapsulated as a custom Index Access Method (AM) plugin. The execution entity for this action is the PostgreSQL database backend process, specifically implemented by a custom shared library (.so module) loaded into the server process space. By overriding callback functions such as `ambuild`, `amscan`, and `amcostestimate`, a unified cost estimation model for multiple metrics is exposed to the query optimizer. This allows the query optimizer to generate a single execution plan that can adapt to queries with different similarity levels, eliminating the need for multi-path index routing or full table rollback.

[0045] The unified indexing method for multi-metric vectors based on shared topology and dynamic weights provided in this invention aims to completely decouple "index topology construction" from "metric distance calculation." The entire execution flow strictly follows the lifecycle of a relational database (taking PostgreSQL as an example), divided into an index construction phase and a query execution phase. The following example, using PostgreSQL, illustrates the above technical solution of this invention: Phase 1: Index Building Phase (Offline Persistence) Step S1: Construct a metric-independent shared graph topology Perform the following actions: 1. When the index creation is triggered by receiving the CREATE INDEX command, extract the underlying spatial distribution features of the input vector set.

[0046] 2. Use basic projection or random approximate adjacency algorithms to generate edge connections between nodes, and construct a shared graph topology that is completely decoupled from specific similarity metrics (such as L2, Cosine, IP).

[0047] 3. Nodes are stored in vector original coordinates. The physical index is only persistently stored on disk or in shared memory for this topology structure. Its data structure is strictly limited to: <node ID, hierarchy identifier, neighbor pointer array>.

[0048] 4. Mandatory Constraints: During the construction process, no distance weights or edge weights related to a specific metric are pre-calculated or persistently stored.

[0049] The above processing breaks the technical bias of traditional vector indexes (such as HNSW) topology being strongly bound to specific metrics. By decoupling metric dependencies, the same set of physical connections can be reused by any similarity metric. This fundamentally eliminates the storage redundancy caused by parallel deployment of multiple indexes (saving the need for massive distance matrix storage) and minimizes the overhead of synchronously maintaining multiple sets of indexes for write / update operations.

[0050] Phase Two: Query Execution Phase (Online Real-Time Processing) Step S2: Query metric identification and unified execution plan generation Perform the following actions: 1. Plan Generation: The PostgreSQL query optimizer obtains a unified cost estimation model through the amcostestimate callback function in the loaded custom AM (index access method) plugin, and directly generates a single index scan execution plan containing the CustomScan node, without the need for multi-path index routing.

[0051] Figure 4 This is a schematic diagram illustrating the integration of the PostgreSQL AM interface and the interaction with the execution plan in an embodiment of the present invention, as shown below. Figure 4 As shown, the left side displays the pg_am system table registration entries, including the AM name, OID, and callback function pointers; the middle section shows the execution plan tree, where the original SeqScan or Multi-Index nodes have been replaced with a unified CustomScan (VectorUnifiedIndexScan) node; the right side shows the data flow of the three major callback functions: ambuild (topology writing during the build phase), amscan (dynamic calculation and cache interaction during the scan phase), and amcostestimate (returning a unified estimated cost to the Cost Model during the optimization phase). The diagram uses dashed boxes to distinguish between the kernel layer and the plugin layer, and solid arrows to represent function calls and metadata transmission, intuitively demonstrating the integrated architecture of "single entry registration, multi-metric adaptation, and optimizer-insensitive."

[0052] 2. Measurement and Recognition: When the executor triggers an index scan, the target vector operators in the SQL are parsed through the PostgreSQL operator family (e.g., `<->` corresponds to L2, `<#>` corresponds to IP, and `<=>` corresponds to Cosine) to accurately identify the target similarity type.

[0053] 3. Context Initialization: Based on the identified metric type, perform three initialization operations: ① Lock the corresponding underlying SIMD (Single Instruction Multiple Data) distance calculation kernel function; ② Load the mathematical boundary constraints (such as value range and monotonicity) corresponding to the metric; ③ Generate a "metric type Hash key" for subsequent cache lookup.

[0054] The above processing implements "single-entry registration and multi-metric adaptation" at the database kernel layer. It shields the optimizer from the differences in underlying metrics, preventing the disaster of full table rollback (SeqScan) caused by the lack of corresponding indexes; at the same time, by locking the SIMD kernel function and mathematical boundaries in advance, it provides a precise rule benchmark for subsequent dynamic calculation and adaptive pruning, avoiding runtime branch judgment overhead.

[0055] Step S3: Lazy evaluation based on distance to the shared memory cache pool Perform the following actions: During graph topology traversal, when it is necessary to calculate the distance between the current candidate node (source node) and its adjacent nodes (target node), a lazy evaluation mechanism (i.e., calculation on demand) is executed: 1. Cache lookup: Using the triple <source node ID, target node ID, metric type Hash> as the composite key, perform a hash lookup in the multi-metric cache pool allocated in the PostgreSQL shared buffer.

[0056] 2. Hit handling: If there is valid data in the slot and it has not expired (TTL check passed), it is determined to be a hit, and the distance value is directly read and returned.

[0057] 3. Miss Handling: If a miss occurs, the SIMD kernel function locked in step S2 is invoked for real-time vectorization computation. After computation, the result is populated back into the cache pool.

[0058] 4. Cache Eviction Policy: The cache table structure includes <Source Node ID, Target Node ID, Metric Type Hash, Distance Value, Timestamp, Access Counter>. A hybrid eviction policy combining LRU and access frequency weighting is adopted: when a node pair is accessed ≥ a preset threshold (e.g., 5 times) within a sliding time window (e.g., 60 seconds), or is in the top 20% of global access frequency statistics, it is defined as a high-frequency node pair and given priority for caching.

[0059] Figure 5 This is a timing diagram of the multi-metric cache pool memory structure and concurrent access in an embodiment of the present invention, as shown below. Figure 5 As shown, the diagram is divided into two parts. The upper part displays the cache slot layout in the Shared Buffer: the left side is the hash bucket array, the middle is the data slot (containing source ID, target ID, metric hash, distance value, timestamp, and counter fields), and the right side is the LRU doubly linked list and hotspot frequency statistics array. The lower part is the concurrent access timeline: showing the process of Session A and Session B simultaneously requesting the same node to calculate the distance. Session A fails the calculation, acquires a read-write lock, writes to the cache, and releases it; Session B subsequently accesses the same key-value pair, hits the cache, and returns directly. The diagram uses a vertical timeline, a horizontal message flow, and lock acquisition / release symbols (…). / It clearly presents the cache consistency mechanism and eviction sequence under multi-session concurrency.

[0060] The above processing postpones distance calculation from "full pre-computation during the construction phase" to "on-demand computation during the execution phase," completely relieving CPU and I / O pressure during the construction phase. Combined with a cross-session shared memory cache pool and a composite hash key design, it effectively eliminates the overhead of duplicate computation of the same node pair in different metric queries under high concurrency scenarios, significantly improving CPU L1 / L2 cache hit rate and reducing memory bandwidth pressure.

[0061] Step S4: Adaptive pruning based on mathematical boundaries and cache hit rate Perform the following actions: At each step of the graph traversal, instead of using a fixed search depth (ef_search) and pruning threshold, dynamic closed-loop adjustment is performed based on the following two dimensions: 1. Initialization of mathematical boundaries: Using the mathematical boundary characteristics obtained in step S2 (such as Cosine∈[-1,1] and satisfying angle monotonicity, L2∈[0,+∞) and satisfying triangle inequality), initialize the safety radius or upper bound of the cosine angle of the current query, which is used to predict whether the neighboring nodes have the potential to surpass the current optimal solution.

[0062] 2. Real-time monitoring of hit rate: Calculate the "cache hit rate" in real time by sliding window or single query cycle (formula: `Hit rate = number of cache hits / total number of distance calculations in the current query session`).

[0063] 3. Dynamic parameter scaling: When the hit rate is lower than the preset lower limit (e.g., 60%), it indicates that the cache does not effectively cover the current path or the data locality is poor. The system automatically increases the width of ef_search and relaxes the pruning threshold to increase the breadth of exploration and ensure the recall rate. When the hit rate is higher than the preset limit (e.g., 85%), it indicates that the cache has been sufficiently accelerated and the path convergence is good. The system tightens the pruning boundary and reduces the search depth to reduce invalid traversals.

[0064] 4. Parameter Management: The basic ef_search and upper and lower hit rate thresholds are loaded into shared memory during the index initialization phase and can be dynamically overwritten via session-level SET commands.

[0065] The above processing constructs an "environment-aware" pruning control system. Using "cache hit rate" as a barometer of system load and data locality, and combining it with the mathematical boundaries of the metric itself, intelligent adaptive pruning strategies are achieved. This completely resolves the contradiction between "high recall" and "low latency" that traditional fixed parameters cannot balance, maximizing the compression of P99 long-tail latency under high concurrency while ensuring query accuracy.

[0066] Step S5: Termination of traversal and return of results Perform the following actions: 1. The scanning will terminate when the graph traversal meets the termination condition (such as the number of explored nodes reaching the dynamically adjusted upper limit of ef_search, or all candidate branches being eliminated by mathematical boundary pruning).

[0067] 2. The selected Top-K node IDs and their corresponding distance values ​​are returned to the PostgreSQL Executor via the amscan callback function.

[0068] 3. The executor combines the MVCC (Multi-Version Concurrency Control) snapshot of the current transaction to perform visibility filtering, final sorting, and output of the result set.

[0069] The above processing ensures that the vector retrieval process can converge safely and controllably, and seamlessly integrates with PostgreSQL's native transaction snapshots and concurrency locking mechanisms. It guarantees that the returned results strictly meet ACID transaction consistency requirements, making this invention not only an algorithmic innovation but also possessing the high availability and high concurrency characteristics of an industrial-grade database.

[0070] Device Example 1 According to embodiments of the present invention, a unified indexing device for multiple metric vectors is provided. Figure 6 This is a schematic diagram of a multi-metric vector unified indexing device according to an embodiment of the present invention, such as... Figure 6 As shown, the multi-metric vector unified indexing device according to an embodiment of the present invention specifically includes: The topology building module 60 is used to extract the spatial distribution features of the input vector set during the index creation stage and construct a shared graph topology structure decoupled from the similarity metric. In the shared graph topology structure, only the node identifier and the neighbor pointer relationship are persistently stored. The plan generation and parsing module 61 is used to return a unified cost estimation model through the cost estimation callback function in the registered custom index access method plugin during the query execution phase, so that the database system's query optimizer can generate an execution plan containing a single index scan node, parse the target similarity metric type corresponding to the query request, and generate the corresponding metric type identifier. The lazy traversal module 62 is used to perform graph traversal based on the shared graph topology. During the traversal, the triple consisting of the source node identifier, the target node identifier, and the metric type identifier is used as the composite key to query the distance value in the preset shared memory cache pool. If a match is found, the distance value in the shared memory cache pool is read. If no match is found, the underlying calculation kernel function corresponding to the target similarity metric type is called to perform lazy distance calculation, and the calculated distance value is written to the shared memory cache pool. The adaptive pruning module 63 is used to dynamically adjust the pruning threshold and search width parameters of the graph traversal based on the mathematical boundary characteristics of the target similarity metric type and the real-time cache hit rate of the shared memory cache pool. Based on the dynamically adjusted pruning threshold and search width parameters, it continues to traverse the shared graph topology until the termination condition is met, and outputs the final candidate node set. The result return module 64 is used to return the final candidate node set and its corresponding distance value to the query executor layer of the database system through the index scan callback interface in the custom index access method plugin. This allows the query executor layer to perform visibility filtering in conjunction with the multi-version concurrency control snapshot of the current transaction and output the final query result set.

[0071] By modularizing topology construction, plan parsing, lazy traversal, adaptive pruning, and result return, this device achieves efficient support for multiple similarity metrics at the database kernel layer. During the index construction phase, it only persists node identifiers and neighbor pointer relationships, eliminating the redundant overhead caused by storing distance matrices in traditional multi-index schemes. During query execution, a unified cost estimation model prompts the optimizer to generate a single index scan plan, avoiding multi-path routing or full table scans. The lazy traversal module uses triples composed of node pairs and metric types as cache keys, reusing cross-session calculation results in shared memory, significantly reducing the CPU load of repeated distance calculations. The adaptive pruning module dynamically adjusts the search width and pruning threshold based on the mathematical characteristics of the metric and the real-time hit rate, effectively suppressing invalid traversals while ensuring recall. Finally, it interfaces with the standard index scan interface and transaction snapshot mechanism to ensure that the output meets ACID consistency requirements. This device achieves low-latency, high-concurrency support for multi-metric queries with a single index structure without changing the original transaction and concurrency control architecture of the database, improving the overall resource utilization and query stability of the system.

[0072] The embodiments of the present invention are device embodiments corresponding to the above method embodiments. The specific operation of each module can be understood with reference to the description of the method embodiments, and will not be repeated here.

[0073] Device Example 2 This invention provides an electronic device, such as... Figure 7 As shown, it includes: a memory 70, a processor 72, and a computer program stored in the memory 70 and executable on the processor 72, wherein the computer program, when executed by the processor 72, performs the steps as described in the method embodiment.

[0074] Device Example 3 This invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor 72, implement the steps described in the method embodiment.

[0075] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A unified indexing method for multi-metric vectors, characterized in that, Applied to database systems, including: Step 1: In the index creation stage, extract the spatial distribution features of the input vector set and construct a shared graph topology structure decoupled from the similarity metric. In the shared graph topology structure, only the node identifier and the neighbor pointer relationship are persistently stored. Step 2: During the query execution phase, a unified cost estimation model is returned through the cost estimation callback function in the registered custom index access method plugin. This enables the database system's query optimizer to generate an execution plan containing a single index scan node and parse the target similarity metric type corresponding to the query request to generate the corresponding metric type identifier. Step 3: Perform graph traversal based on the shared graph topology. During the traversal, use the triplet consisting of the source node identifier, the target node identifier, and the metric type identifier as the composite key to query the distance value in the preset shared memory cache pool. If a match is found, read the distance value from the shared memory cache pool. If no match is found, call the underlying calculation kernel function corresponding to the target similarity metric type to perform lazy distance calculation and write the calculated distance value into the shared memory cache pool. Step 4: Based on the mathematical boundary characteristics of the target similarity metric type and the real-time cache hit rate of the shared memory cache pool, dynamically adjust the pruning threshold and search width parameters of the graph traversal, and continue traversing the shared graph topology according to the dynamically adjusted pruning threshold and search width parameters until the termination condition is met, and output the final candidate node set. Step 5: Through the index scan callback interface in the custom index access method plugin, the final candidate node set and its corresponding distance value are returned to the query executor layer of the database system, so that the query executor layer can perform visibility filtering in combination with the multi-version concurrency control snapshot of the current transaction and output the final query result set.

2. The method according to claim 1, characterized in that, During the index creation phase, the spatial distribution features of the input vector set are extracted, and a shared graph topology structure decoupled from the similarity metric is constructed. This specifically includes: The underlying spatial distribution features of the input vector set are extracted, and edge connections between nodes are generated using a random approximate adjacency algorithm or a basic projection algorithm to construct a multi-layer graph topology. During the construction process, no distance weights or edge weights related to a specific similarity metric are pre-calculated or persistently stored. The constructed shared graph topology is stored as the core data carrier of the physical index on disk or shared memory. The nodes in the shared graph topology are stored according to the original coordinates of the vectors. The data structure of the shared graph topology includes: node identifier, a hierarchy identifier assigned to each node, and a neighbor pointer array, but does not include any distance weights.

3. The method according to claim 1, characterized in that, The database system is: PostgreSQL database system; During the index scanning phase, the target similarity metric type corresponding to the query request is parsed, and the corresponding metric type identifier is generated, specifically including: The system receives a user-submitted SQL query statement containing vector fields and target similarity operators. When the executor triggers an index scan, it parses the target similarity operators in the SQL query statement using the operator family of the PostgreSQL database system, maps them to a predefined target similarity metric type, locks the corresponding underlying vectorized distance calculation kernel function based on the target similarity metric type, loads the corresponding mathematical boundary constraints, and generates a metric type hash value.

4. The method according to claim 1, characterized in that, Step 3 specifically includes: Based on the shared graph topology, the traversal engine starts from the starting node and visits its neighbor node list in sequence. For each pair of source node ID and target node ID, a cache key is constructed. The cache key specifically includes: source node ID, target node ID, and metric type Hash value. In the shared memory cache pool, the triple <source node ID, target node ID, metric type Hash> is used as the composite key. The hash table is used to check if there is a valid record for this cache key. If a match is found and the cached data has not expired, the distance value is read directly. If no match is found, the underlying calculation kernel function corresponding to the target similarity metric type is called to perform lazy distance calculation, and the calculated distance value is written to the shared memory cache pool. A cache eviction strategy is adopted to evict node pairs. The cache eviction strategy includes: when the number of accesses of a node pair within a sliding time window is greater than or equal to a preset threshold, or when it is in the top predetermined percentage of global access frequency statistics, it is defined as a high-frequency node pair and given priority to reside in the cache.

5. The method according to claim 4, characterized in that, If no match is found, the underlying kernel function corresponding to the target similarity metric type is invoked to perform lazy distance calculation, and the calculated distance value is written to the shared memory cache pool. Specifically, this includes: The corresponding underlying distance calculation kernel function is called according to the target metric type. The multi-dimensional coordinate vector is calculated in parallel using the Single Instruction Multiple Data Stream (SIMD) instruction set to generate an accurate distance value. After the calculation is completed, the distance value, the current timestamp, and the access counter are initialized to 1 and prepared to be written to the cache pool. In the shared memory cache pool, the corresponding cache slot is located. If the slot is already occupied, a hybrid eviction policy that combines LRU list priority and access frequency weighting is used to determine whether to replace the slot and write new data into it. The cache fields of the new data include: source node ID, target node ID, metric type hash value, distance value, timestamp, and access counter. If it is the first access, the counter is reset to 1; if it is a repeated access, the counter is incremented, and the updated status of the shared memory cache pool is output.

6. The method according to claim 1, characterized in that, Step 4 specifically includes: Based on the mathematical boundary characteristics corresponding to the target metric type, the spatial constraint boundary is initialized as the initial pruning threshold; wherein, when the target metric type is a distance metric, the spatial constraint boundary is the safe pruning radius; when the target metric type is a similarity metric, the spatial constraint boundary is the upper bound of the angle or the similarity threshold. The real-time cache hit rate is calculated in real time based on the number of cache hits and the total number of cumulative distance calculations in the current query session; wherein, the formula for calculating the real-time cache hit rate is: Hit rate = Number of cache hits / Total number of cumulative distance calculations in the current query session; The real-time cache hit rate is compared with a preset hit rate range, and parameters are dynamically scaled: if the real-time cache hit rate is lower than the preset lower limit, the search width parameter is increased and the initial pruning threshold is relaxed to increase the exploration breadth of graph traversal; if the real-time cache hit rate is higher than the preset upper limit, the search width parameter is reduced and the initial pruning threshold is tightened to accelerate path convergence of graph traversal; wherein, the base value of the search width parameter and the upper and lower limit thresholds of the hit rate range are loaded into shared memory during the index initialization phase and can be dynamically overridden through session-level configuration commands; In each step of the graph traversal, the distance to the neighboring nodes of the current node is compared and the potential is evaluated using the dynamically scaled search width parameter and the pruning threshold. Only nodes that meet the pruning conditions are retained to enter the next layer of the candidate set. When the number of explored nodes reaches the upper limit of the dynamically scaled search width parameter, or the current candidate set is empty, or all candidate branches are eliminated by the pruning threshold, the termination condition is triggered, and the final candidate node set is output.

7. The method according to claim 1, characterized in that, Step 5 specifically includes: By using the index scan callback interface registered in the custom index access method plugin, the final candidate node set and its corresponding distance value are encapsulated into output records that conform to the current relational database tuple format; The output record is passed to the query executor layer of the relational database, and the query executor layer obtains a multi-version concurrency control snapshot of the current query transaction; Based on the multi-version concurrency control snapshot, transaction visibility filtering is performed on the node data in the output record to remove invisible data that does not meet the current transaction isolation level; The node data after being filtered by transaction visibility is globally reordered according to the distance value corresponding to the target similarity metric type, the top K nodes are selected as the final query result set, and returned to the upper layer application according to the standard query protocol.

8. A unified indexing device for multi-metric vectors, characterized in that, include: The topology construction module is used to extract the spatial distribution features of the input vector set during the index creation stage and construct a shared graph topology structure decoupled from the similarity metric. In the shared graph topology structure, only the node identifier and the neighbor pointer relationship are persistently stored. The plan generation and parsing module is used during the query execution phase to return a unified cost estimation model through the cost estimation callback function in the registered custom index access method plugin, so that the database system's query optimizer can generate an execution plan containing a single index scan node; and parse the target similarity metric type corresponding to the query request to generate the corresponding metric type identifier. The lazy traversal module is used to perform graph traversal based on the shared graph topology. During the traversal, a triple consisting of the source node identifier, the target node identifier, and the metric type identifier is used as a composite key to query the distance value in a preset shared memory cache pool. If a match is found, the distance value in the shared memory cache pool is read. If no match is found, the underlying calculation kernel function corresponding to the target similarity metric type is called to perform lazy distance calculation, and the calculated distance value is written into the shared memory cache pool. The adaptive pruning module is used to dynamically adjust the pruning threshold and search width parameters of the graph traversal based on the mathematical boundary characteristics of the target similarity metric type and the real-time cache hit rate of the shared memory cache pool. Based on the dynamically adjusted pruning threshold and search width parameters, the module continues to traverse the shared graph topology until the termination condition is met, and outputs the final candidate node set. The result return module is used to return the final candidate node set and its corresponding distance value to the query executor layer of the database system through the index scan callback interface in the custom index access method plugin, so that the query executor layer can perform visibility filtering in combination with the multi-version concurrency control snapshot of the current transaction and output the final query result set.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the multi-metric vector unified indexing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the steps of the multi-metric vector unified indexing method as described in any one of claims 1 to 7.

Images