Distributed vector graph search method, electronic device, and computer program product

Abstract

This disclosure provides a distributed vector graph retrieval method, electronic device, and computer program product, relating to the field of computer technology. The distributed vector graph retrieval method of this disclosure includes: responding to a user query request, selecting candidate graph nodes from a header index to determine a candidate graph node queue; selecting candidate graph nodes from the candidate graph node queue for affiliation identification to determine remote graph nodes; sending a near-data calculation request to the server corresponding to the remote graph node to obtain the remote graph node queue; merging the remote graph node queue into the candidate graph node queue; calculating the node weights of the candidate graph nodes to be processed; aggregating the node weights based on the servers corresponding to the candidate graph nodes to determine the scheduling weights of each server; determining whether the migration conditions are met based on the scheduling weights; if so, migrating the query state to the target server for graph retrieval; if not, expanding the candidate graph nodes to be processed and re-determining whether the termination conditions are met.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a distributed vector graph retrieval method, electronic device, and computer program product. Background Technology

[0002] Traditional sharded retrieval methods construct and search data independently, making it difficult to leverage global adjacency relationships across shards. This easily leads to redundant computations, redundant I / O accesses, and high result merging overhead. On the other hand, while emerging distributed global graph schemes can retain global pruning capabilities, they often employ fixed strategies when handling cross-node access. For example, directly migrating the entire query state or repeatedly requesting remote computation results in frequent serialization, network transmission, and state takeover overhead. When the query path involves frequent cross-node jumps, communication latency significantly impacts retrieval performance. Summary of the Invention

[0003] This disclosure provides a distributed vector graph retrieval method, an electronic device, and a computer program product.

[0004] According to one aspect of this disclosure, a distributed vector graph retrieval method is provided, wherein the distributed vector graph is a global vector graph in which graph nodes and their adjacency relationships are deployed on different servers. The method includes: responding to a user query request, selecting candidate graph nodes from a header index, determining a candidate graph node queue, wherein the header index is determined by graph traversal of the distributed vector graph, and the candidate graph node queue is sorted based on distance to the query vector of the user query request; selecting candidate graph nodes from the candidate graph node queue for affiliation identification, determining remote graph nodes not located on an execution server, wherein the execution server is the server currently processing the user query request; and sending a near-data computation request to the server corresponding to the remote graph node to obtain the remote graph node queue. The process involves: merging the remote graph node queue into the candidate graph node queue; when the merged candidate graph node queue does not meet the termination condition, calculating the node weights of the candidate graph nodes to be processed in the merged candidate graph node queue based on their arrangement order; aggregating the node weights based on the servers corresponding to the candidate graph nodes to be processed in the merged candidate graph node queue to determine the scheduling weight of each server; and determining whether the migration condition is met based on the scheduling weight of each server. If yes, the query status containing the merged candidate graph node queue is migrated to the target server for graph retrieval; otherwise, the candidate graph nodes to be processed in the merged candidate graph node queue are expanded, and the expanded candidate graph node queue is re-determined to meet the termination condition.

[0005] According to the distributed vector graph retrieval method disclosed herein, the accuracy of query state scheduling judgment is improved by calculating the server scheduling weights of the candidate graph node queue arrangement position weights and judging query state transitions. This more accurately identifies the distribution of the search frontier among the servers, reduces invalid state transitions and cross-node communication overhead, and avoids frequent complete query state transmissions caused by relying solely on fixed rules or the affiliation of a single candidate graph node. Simultaneously, through remote near-data computation and adaptive scheduling strategies, multiple servers can participate in candidate expansion and near-data computation in parallel, reducing single query latency.

[0006] According to at least one embodiment of the distributed vector graph retrieval method of this disclosure, in response to a user query request, candidate graph nodes are selected from a header index, and a candidate graph node queue is determined, including: in response to a user query request, selecting candidate graph nodes from graph nodes in the header index based on the distance between the query vector of the user query request and the graph nodes in the header index; sorting the candidate graph nodes based on the distance between the candidate graph nodes and the query vector of the user query request; and determining the candidate graph node queue by combining the server corresponding to the candidate graph nodes.

[0007] According to at least one embodiment of the distributed vector graph retrieval method of this disclosure, selecting candidate graph nodes from the candidate graph node queue for attribution identification to determine remote graph nodes not located on the execution server includes: selecting candidate graph nodes from the candidate graph node queue whose distance to the query vector of the user query request is within a target range to determine a set of graph nodes to be processed; comparing the server identifier of the server corresponding to the candidate graph node in the set of graph nodes to be processed with the server identifier of the execution server to determine remote graph nodes not located on the execution server.

[0008] The distributed vector graph retrieval method according to at least one embodiment of the present disclosure further includes: calculating the distances between a local graph node in the execution server and its neighboring graph nodes based on the query vector of the user query request; comparing the distances between the local graph node and its neighboring graph nodes with the distances between candidate graph nodes in the candidate graph node queue; and merging the neighboring nodes of the local graph node into the candidate graph node queue.

[0009] According to at least one embodiment of the distributed vector graph retrieval method of this disclosure, when the merged candidate graph node queue does not meet the termination condition, the node weights of the candidate graph nodes to be processed in the merged candidate graph node queue are calculated based on the arrangement order of the merged candidate graph node queue, including: selecting candidate graph nodes to be processed from the merged candidate graph node queue to determine a candidate set to be scheduled; calculating the position scores of the candidate graph nodes in the candidate set to be scheduled based on the arrangement order of the candidate graph nodes in the candidate graph node queue; and normalizing the position scores of the candidate graph nodes in the candidate set to be scheduled to determine the node weights of the candidate graph nodes in the candidate set to be scheduled.

[0010] According to at least one embodiment of the distributed vector graph retrieval method of this disclosure, when the merged candidate graph node queue meets the termination condition, the method includes: sorting the merged candidate graph node queue based on the distance between the candidate graph nodes in the merged candidate graph node queue to determine the target graph node queue; and selecting a target graph node from the target graph node queue as the retrieval result.

[0011] According to at least one embodiment of the distributed vector graph retrieval method of this disclosure, the termination conditions include: there are no candidate graph nodes that can be further expanded in the merged candidate graph node queue, the number of expanded candidate graph nodes is equal to the target upper limit, and / or the query time reaches the target time limit.

[0012] According to at least one embodiment of the distributed vector graph retrieval method of this disclosure, the migration conditions include: the scheduling weight of the execution server is equal to the target value, and the scheduling weight of the target server is greater than the target value; or, the difference between the scheduling weight of the execution server and the scheduling weight of the target server is greater than or equal to a first threshold, and the scheduling weight of the target server is greater than or equal to a second threshold, wherein the first threshold is used to delay the migration of the query state, and the second threshold is used to ensure the concentration of high-priority candidate graph nodes in the target server.

[0013] According to at least one embodiment of the distributed vector graph retrieval method of this disclosure, the query state containing the merged candidate graph node queue is migrated to the target server for graph retrieval, including: serializing and encapsulating the query state containing the merged candidate graph node queue, wherein the query state is used to record intermediate data of the execution of the user query request; and sending the serialized and encapsulated query state to the target server.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, causing the processor to perform a distributed vector graph retrieval method according to any embodiment of this disclosure.

[0015] According to another aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement the distributed vector graph retrieval method of any embodiment of this disclosure.

[0016] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a distributed vector graph retrieval method according to any embodiment of this disclosure.

[0017] The beneficial effects of the distributed vector graph retrieval method disclosed herein are at least as follows: By calculating the server scheduling weights based on the positional weights of candidate graph nodes in the queue and judging query state transitions, the accuracy of query state scheduling judgment is improved, the distribution of the search frontier among servers is more accurately identified, invalid state transitions and cross-node communication overhead are reduced, and frequent complete query state transmissions caused by relying solely on fixed rules or the affiliation of a single candidate graph node are avoided. Simultaneously, by employing remote near-data computation and adaptive scheduling strategies, the retrieval parallelism is enhanced, enabling multiple servers to participate in candidate expansion and near-data computation in parallel, thereby reducing single-query latency. Attached Figure Description

[0018] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0019] Figure 1 This is a schematic diagram of the overall process of a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0020] Figure 2 This is a schematic diagram of the process for determining a queue of candidate graph nodes in a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0021] Figure 3 This is a schematic diagram of the process for determining remote graph nodes in a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0022] Figure 4 This is a schematic diagram of the process for calculating node weights in a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0023] Figure 5This is a schematic diagram of the query state transition process in a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0024] Figure 6 This is a flowchart illustrating the process of satisfying termination conditions in a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0025] Figure 7 This is a flowchart illustrating the process of expanding the candidate graph node queue in a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0026] Figure 8 This is an overall flowchart of a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0027] Figure 9 This is a schematic diagram of the query state adaptive scheduling process in a distributed vector graph retrieval method according to one embodiment of the present disclosure.

[0028] Figure 10 This is a schematic structural block diagram of a distributed vector graph retrieval system according to one embodiment of the present disclosure.

[0029] Figure 11 This is a schematic structural block diagram of a distributed vector graph retrieval device according to one embodiment of the present disclosure.

[0030] Figure 12 This is a schematic structural block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0031] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0032] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] In a vector database cluster comprising hundreds of servers deployed over a standard TCP network, when a user initiates an image search request, a fixed strategy (e.g., immediately migrating the entire query state upon encountering a remote graph node) can lead to frequent node jumps in the query path. This results in a large number of complete query states being repeatedly transmitted across the network, causing high serialization overhead, network latency, and waiting time for remote server takeover, thus slowing down the speed of a single query. Conversely, relying solely on a central node to continuously send remote computation requests to various remote servers results in multiple rounds of request-response communication, similarly causing accumulated latency.

[0034] To address this, this disclosure proposes a distributed vector graph retrieval method. Instead of immediately migrating the entire query state upon encountering a remote graph node, this method first processes all remote graph nodes in parallel through a near-data computing service, utilizing the local resources of each server to achieve local expansion. This avoids invalid migrations triggered by a single remote graph node and eliminates the round-trip latency caused by multiple request-response cycles between the central graph node and multiple remote servers. Based on this, node weights are calculated by analyzing the ranking position of candidate graph nodes in the candidate graph node queue, and the scheduling weights of each server are aggregated. This allows the decision-making process to no longer rely solely on the number of candidate graph nodes, but to accurately identify whether the current search frontier is highly concentrated on a single remote server.

[0035] In large-scale cloud retrieval services, the distributed vector graph retrieval method disclosed herein can be deployed on vector database clusters, supporting hundreds of millions of product image searches on e-commerce platforms, similar content recommendations on short video platforms, and real-time knowledge base question answering in large-scale RAG model applications. In edge computing scenarios, the head index and lightweight scheduling logic can be pushed down to edge nodes, enabling smart terminals to complete preliminary candidate screening and scheduling decisions locally, only initiating targeted near-data computation requests or state transition requests to the cloud, reducing wide area network transmission overhead. In edge-side offline retrieval scenarios, mobile phones or IoT devices can cache partial fragments of the distributed vector graph in local storage, realizing dynamic flow of query states between devices, supporting local semantic search and cross-device federated retrieval in offline environments. Furthermore, in hybrid cloud architectures, enterprises can deploy sensitive data vector graphs on private cloud servers and non-sensitive data on public clouds, achieving intelligent routing of query requests between public and private clouds through the scheduling weight calculation and state transition judgment of the method disclosed herein.

[0036] Figure 1 A schematic diagram illustrating the overall flow of a distributed vector graph retrieval method according to one embodiment of this disclosure is shown. Figure 1The method M100 shown includes steps S110 to S170. A distributed vector graph is a global vector graph that deploys graph nodes and their adjacency relationships across different servers. This method can be executed by electronic devices with computing and communication capabilities, such as servers, smartphones, tablets, in-vehicle terminals, or IoT devices.

[0037] In step S110, in response to the user query request, candidate graph nodes are selected from the header index to determine the candidate graph node queue. The header index is determined by traversing the distributed vector graph, and the candidate graph node queue is sorted based on the distance to the query vector of the user query request.

[0038] In response to a user query request, an initial candidate retrieval is performed based on the locally stored header index. The header index, as a lightweight navigation structure extracted from the full distributed vector graph, can quickly locate the search entry point without accessing the full distributed graph data. The distance between the user's query vector and each graph node in the header index is calculated, and the graph nodes with the closest distances are selected as candidate graph nodes, forming a queue of candidate graph nodes.

[0039] Preferably, a user query request may include a query vector, or it may include text, images, audio, video, or other objects to be retrieved.

[0040] Specifically, when a user query request is not in vector form, the user query request is converted into a query vector.

[0041] Preferably, the candidate graph node queue is sorted from closest to furthest based on the distance to the query vector of the user's query request.

[0042] Preferably, the candidate graph node queue includes at least the candidate graph node identifier, the distance value between the candidate graph node and the query vector, the access status of the candidate graph node, and the server identifier ServerID of the server corresponding to the candidate graph node.

[0043] Preferably, an initial query state is generated based on the query vector, the candidate graph node queue, the set of visited graph nodes, the set of search results, the identifier of the currently executing graph node, and the search parameters.

[0044] Preferably, the retrieval parameters include at least one of the following: the number of returned results K, the length of the candidate graph node queue L, the expansion width per round W, the scheduling judgment window M, the maximum number of graph nodes accessed, or the maximum query latency.

[0045] Preferably, the distance between the query vector of the user's query request and the candidate graph nodes can be determined by approximate distance calculation using the PQ product quantization method, or it can be calculated using Euclidean distance, inner product distance, cosine distance, or other similarity measurement methods consistent with the database construction stage.

[0046] In step S120, candidate graph nodes are selected from the candidate graph node queue for attribution identification to determine the remote graph nodes that are not on the execution server. The execution server is the server currently used to process user query requests.

[0047] From the candidate graph node queue, select the Top-W candidate graph nodes that are closest to the query vector and require expansion, where W represents the candidate expansion width for each round of graph traversal. Based on the mapping relationship between candidate graph nodes and servers, identify the affiliation of the Top-W candidate graph nodes to determine the server corresponding to each candidate graph node. If the server corresponding to a candidate graph node is the same as the current execution server, then the candidate graph node is determined as a local graph node; if the server corresponding to a candidate graph node is not the execution server, then the candidate graph node is determined as a remote graph node.

[0048] Preferably, the candidate expansion width can be preset according to retrieval accuracy requirements, query latency requirements, node load, or system configuration.

[0049] In step S130, a near data calculation request is sent to the server corresponding to the remote graph node to obtain the remote graph node queue.

[0050] The server groups remote graph nodes and sends near data calculation requests to the server. Upon receiving the near data calculation requests, the server performs distance calculations, candidate filtering, and local expansion on the specified remote graph node and its neighboring graph nodes within a limited range, based on its locally stored graph node data and adjacency list. It then returns candidate summary information, i.e., the remote graph node queue, to the currently executing server. The remote graph node queue is determined based on the local expansion performed locally by the server corresponding to the remote graph node.

[0051] Preferably, the near data calculation request may include information such as query identifier QueryID, query vector PQCode, remote graph node set, current distance threshold, maximum expansion depth, maximum number of accessed nodes, and maximum number of returned candidates.

[0052] Preferably, the candidate summary information may include the NodeID of the candidate graph node, the ServerID of the server corresponding to the candidate graph node, the distance value of the candidate graph node, the candidate access status, and necessary neighbor summary information.

[0053] In step S140, the remote graph node queue is merged into the candidate graph node queue.

[0054] The queue of remote graph nodes is added to the queue of current candidate graph nodes in an orderly manner according to a unified rule. The progress of the local search at the remote nodes is incorporated into the global search process.

[0055] Preferably, the remote graph node queue is merged into the current candidate graph node queue, and filtering is performed based on the distance between the candidate graph node and the query vector, the current distance threshold, the capacity of the candidate graph node queue, and the candidate access status. Remote graph nodes that do not meet the filtering criteria are pruned; remote graph nodes that meet the filtering criteria are added to the candidate graph node queue, and the sorting result and access status of the candidate graph node queue are updated.

[0056] Preferably, the access status can be pending and processed, or pending expansion and expanded.

[0057] In step S150, when the merged candidate graph node queue does not meet the termination condition, the node weights of the candidate graph nodes to be processed in the merged candidate graph node queue are calculated based on the arrangement order of the merged candidate graph node queue.

[0058] If the current query does not meet the preset termination condition, select candidate graph nodes to be expanded from the current candidate graph node queue, and calculate the node weight of the candidate graph nodes to be expanded according to their sorting position in the current candidate graph node queue.

[0059] Preferably, the higher the ranking of a candidate graph node, the higher its corresponding node weight, indicating that the candidate graph node has a greater impact on the subsequent search path.

[0060] Preferably, the termination conditions include: the merged candidate graph node queue does not contain any candidate graph nodes that can be further expanded, the number of expanded candidate graph nodes is equal to the target upper limit, and / or the query time reaches the target time limit.

[0061] In step S160, based on the servers corresponding to the candidate graph nodes to be processed in the merged candidate graph node queue, the node weights are aggregated to determine the scheduling weight of each server.

[0062] Based on the server identifier corresponding to the candidate graph node to be processed, the node weights of all candidate graph nodes belonging to the same server are accumulated and aggregated to obtain the scheduling weight corresponding to each server. The higher the server scheduling weight, the more high-priority candidate graph nodes to be expanded in the current search front are concentrated on that server, and the more suitable that server is as the execution server for subsequent query states.

[0063] In step S170, the migration conditions are determined based on the scheduling weights of each server. If yes, the query status of the merged candidate graph node queue is migrated to the target server for graph retrieval. If no, the candidate graph nodes to be processed in the merged candidate graph node queue are expanded, and the expanded candidate graph node queue is re-determined to determine whether the termination conditions are met.

[0064] When the migration conditions are met, the complete query state is migrated to the target server for continued execution. When the migration conditions are not met, the complete query state is not migrated; instead, the local machine continues to expand the candidate graph nodes belonging to the currently executing server, and updates the candidate graph node queue, the current query result set, and the visited node set based on the expansion results.

[0065] Preferably, the query status may include the query vector, the current candidate graph node queue, the current query result set, the visited graph node set, the current distance threshold, retrieval parameters, and the query identifier.

[0066] Specifically, the query status includes the status of the current candidate graph node queue, the current query result set, retrieval parameters, and query vector. The query vector is usually only sent to a server once. The server caches the query vector, and the status messages passed across servers can carry the complete query vector without repeating it, thereby reducing network transmission volume.

[0067] Specifically, the current status of the candidate graph node queue includes the candidate graph nodes in the current candidate graph node queue, the approximate distance of each candidate graph node relative to the query vector, the processing status / expansion status of the candidate graph nodes, and the candidate graph nodes to be processed or expanded that are most worth visiting in the next round of expansion.

[0068] Specifically, the current query result set includes candidate graph node data already read from the solid-state drive (SSD) and their full-precision distances to the query vector. Preferably, the target server is the server with the highest scheduling weight, excluding the execution server.

[0069] Preferably, the migration conditions include: the scheduling weight of the execution server is equal to the target value, and the scheduling weight of the target server is greater than the target value; or, the difference between the scheduling weight of the execution server and the scheduling weight of the target server is greater than or equal to a first threshold, and the scheduling weight of the target server is greater than or equal to a second threshold, wherein the first threshold is used for the migration of delayed query states, and the second threshold is used to ensure the concentration of high-priority candidate graph nodes on the target server.

[0070] Therefore, the method disclosed herein achieves dynamic adaptive scheduling in distributed vector graph retrieval based on server scheduling weight calculation of candidate graph node queue position weights and query state transition judgment. Specifically, by introducing a remote-to-near-data calculation mechanism, lightweight requests are used to replace complete query state transitions, avoiding the serialization overhead and network latency caused by repeated transmission of complete states between servers when the query path frequently crosses nodes.

[0071] By calculating node weights based on the candidate graph node queue positions and scheduling weights by server aggregation, the distribution of the search frontier among servers can be accurately identified. The complete state migration is only triggered when the candidate graph nodes are sufficiently concentrated on the target server and the migration lag threshold and concentration threshold are met, effectively preventing invalid state oscillations.

[0072] This preserves the global graph index's adjacency relationships and pruning capabilities, reducing cross-node communication overhead and single-query latency.

[0073] Regarding step S110, in response to the user query request, candidate graph nodes are selected from the header index to determine the candidate graph node queue. In some embodiments of this disclosure, this may include, for example... Figure 2 Steps S1101 to S1102 are shown.

[0074] In step S1101, in response to a user query request, candidate graph nodes are selected from the graph nodes in the header index based on the distance between the query vector of the user query request and the graph nodes in the header index.

[0075] In response to a user query request, the distance between the query vector and each graph node in the locally stored header index is calculated. The graph nodes are then sorted based on the calculation results, and a group of graph nodes with the closest distances is selected as the initial candidate graph nodes.

[0076] In step S1102, the candidate graph nodes are sorted based on the distance between the candidate graph nodes and the query vector of the user query request, and the candidate graph node queue is determined by combining the server corresponding to the candidate graph node.

[0077] Based on the distance between each candidate graph node and the query vector, the candidate graph nodes are sorted in ascending order to form an ordered candidate set. Simultaneously, by combining the server identifier of the server to which each candidate graph node belongs, the corresponding target server information is labeled for each candidate graph node, thereby constructing a candidate graph node queue containing the candidate graph node identifier, distance value, access status, and server identifier.

[0078] Therefore, initial candidate recall and ranking based on the distance between the query vector and the head index graph nodes can quickly locate high-value search entry points without directly accessing the full distributed graph data, reducing initial retrieval latency. Simultaneously, by labeling candidate graph nodes with server identifiers, the physical distribution of candidate graph nodes is determined at the initial stage of execution, enabling synergistic optimization of retrieval efficiency and cross-node scheduling accuracy.

[0079] Regarding step S120, candidate graph nodes are selected from the candidate graph node queue for affiliation identification to determine remote graph nodes that are not located on the execution server. In some embodiments of this disclosure, this may include, for example... Figure 3Steps S1201 to S1202 are shown.

[0080] In step S1201, candidate graph nodes whose distance from the query vector of the user's query request is within the target range are selected from the candidate graph node queue to determine the set of graph nodes to be processed.

[0081] From the candidate graph node queue, select the top W candidate graph nodes (i.e., Top-W) that are within the target range in order of distance from the query vector, and determine the set of graph nodes to be processed. W is the candidate expansion width for each round of graph traversal.

[0082] In step S1202, the server identifier of the server corresponding to the candidate graph node in the set of graph nodes to be processed is compared with the server identifier of the execution server to determine the remote graph node that is not located on the execution server.

[0083] Based on the mapping relationship between candidate graph nodes and servers, the server identifier of the server to which each candidate graph node belongs in the set of graph nodes to be processed is obtained, and the server identifier is compared one by one with the server identifier of the currently executing server. If the server identifier of the candidate graph node is the same as that of the currently executing server, it is determined to be a local graph node; if the server identifier of the candidate graph node points to another server, it is determined to be a remote graph node.

[0084] Therefore, by selecting the Top-W high-value candidate graph nodes from the candidate graph node queue based on distance for attribution identification, local graph nodes and remote graph nodes can be accurately distinguished. Prioritizing the processing of the closest high-priority candidate graph nodes ensures that limited computing and communication resources are concentrated on the graph nodes most likely to affect the final retrieval results, thereby improving the execution efficiency and resource utilization of retrieval tasks in a distributed environment.

[0085] Regarding step S150, when the merged candidate graph node queue does not meet the termination condition, the node weights of the candidate graph nodes to be processed in the merged candidate graph node queue are calculated based on their arrangement order. In some embodiments of this disclosure, this may include, for example... Figure 4 Steps S1501 to S1503 are shown.

[0086] In step S1501, candidate graph nodes to be processed are selected from the merged candidate graph node queue to determine the candidate set to be scheduled.

[0087] Specifically, the top M candidate graph nodes that are closest to the query vector and need to be expanded are selected from the candidate graph node queue to determine the candidate set to be scheduled: Where L represents the set of candidates to be scheduled, and M represents the size of the scheduling decision window. M can be equal to or greater than the expansion width of each round. This represents the i-th candidate graph node in the candidate set to be scheduled. The earlier a candidate graph node is ranked in the candidate graph node queue, the closer it is to the query direction, and the greater its influence on subsequent search paths.

[0088] In step S1502, the position score of the candidate graph nodes in the candidate set to be scheduled is calculated based on the order of the candidate graph nodes in the candidate graph node queue.

[0089] Specifically, for candidate graph nodes in the candidate set to be scheduled Calculate the position score based on its sorting position in the candidate graph node queue: in, Represents candidate graph nodes Position score, where i represents the candidate graph node The ranking position within the candidate graph node set to be scheduled. Since the candidate graph node set to be expanded is sorted from closest to furthest from the query vector, the candidate graph node ranked first... The position score is 1, and the candidate graph node ranked Mth is selected. Position score In other words, the earlier a candidate graph node is in the queue, the higher its position score.

[0090] In step S1503, the position scores of candidate graph nodes in the candidate set to be scheduled are normalized to determine the node weights of candidate graph nodes in the candidate set to be scheduled.

[0091] Specifically, the node weights of the candidate graph nodes to be expanded are: This represents the node weight of the candidate graph node. This represents the j-th candidate graph node in the candidate set to be scheduled. This represents the position influence factor, which is used to adjust the degree of influence of the candidate graph node queue position on the node weight. The larger the position influence factor, the greater the influence of the position on the node weight.

[0092] Therefore, by introducing a node weighting mechanism based on sorting position, a refined evaluation of the importance of candidate graph nodes is achieved. Instead of treating all candidate graph nodes as equally important, higher-ranking nodes in the queue are assigned higher weights, thus more accurately reflecting the influence of each candidate graph node on subsequent search paths. This approach can identify the core regions of the current search frontier, effectively avoiding misjudgments caused by a large number of low-relevance candidate graph nodes at distant points.

[0093] Regarding step S170, the query state containing the merged candidate graph node queue is migrated to the target server for graph retrieval. In some embodiments of this disclosure, it may include, for example... Figure 5 Steps S510 to S520 are shown.

[0094] In step S510, the query status containing the merged candidate graph node queue is serialized and encapsulated. The query status is used to record intermediate data of the user query request execution.

[0095] In step S520, the serialized and encapsulated query status is sent to the target server.

[0096] Therefore, by serializing and migrating the query state, a seamless relay of distributed retrieval tasks is achieved. Serializing and encapsulating the query state ensures that all execution contexts can be transmitted completely and accurately.

[0097] In some embodiments of this disclosure, when the merged candidate graph node queue meets the termination condition, it may include, for example: Figure 6 Steps S610 to S620 are shown.

[0098] In step S610, the merged candidate graph node queue is sorted based on the distance between the candidate graph nodes in the merged candidate graph node queue to determine the target graph node queue.

[0099] The candidate graph nodes in the current candidate graph node queue or query result set are identified as the candidate set to be rearranged. The original vector, high-precision vector representation, or full-precision distance calculation result of the candidate graph nodes is obtained.

[0100] In step S620, target graph nodes are selected from the target graph node queue as retrieval results.

[0101] Based on the full-precision distance calculation results (i.e., the distance between candidate graph nodes and query vectors), the candidate set to be rearranged is sorted to determine the target graph node queue, and the top K candidate graph nodes with the best distance or the highest similarity are selected as the final Top-K retrieval results output.

[0102] Therefore, by introducing a full-precision rearrangement mechanism, the accuracy of the final retrieval results can be improved.

[0103] In some embodiments of this disclosure, it may also include, for example Figure 7 Steps S710 to S720 are shown.

[0104] In step S710, based on the query vector of the user query request, the distances between the local graph node in the execution server and its adjacent graph nodes are calculated.

[0105] For a local graph node whose affiliation has been determined, its locally stored graph node data and adjacency table are read to obtain the identifiers of all its neighboring graph nodes. Based on the query vector of the user's query request, the distances between the query vector and the local graph node and each of its neighboring graph nodes are calculated respectively, thereby exploring the neighbor space of the local graph node, discovering new candidate graph nodes that are closer to the query vector, and realizing the candidate expansion of local graph traversal.

[0106] In step S720, the distance of the local graph node and the distance of the local graph node's neighboring graph nodes are compared with the distance of the candidate graph nodes in the candidate graph node queue, and the neighboring nodes of the local graph node are merged into the candidate graph node queue.

[0107] The distance calculation results of the local graph node and its adjacent graph nodes are compared with the distances of existing candidate graph nodes in the candidate graph node queue. Based on the comparison results, combined with the current distance threshold, the capacity of the candidate graph node queue, and the candidate access status, the new candidate graph nodes that are closer to the query vector and have not been visited are added to the candidate graph node queue, and the queue sorting results and access status are updated. At the same time, the expanded local graph nodes are marked as visited to avoid duplicate processing, thereby completing the local expansion of the candidate graph node queue, enriching the search frontier and advancing the graph traversal process.

[0108] Therefore, retrieval efficiency is improved by performing graph traversal locally in parallel. This fully utilizes the local computing and storage resources of the current execution server to quickly calculate distances to local graph nodes and their neighbors, and expand the candidate pool, avoiding centralized processing of all computational tasks or reliance on remote communication. By comparing and filtering newly discovered candidate graph nodes with the existing queue, it ensures that the candidate graph node queue always contains the optimal nodes to be processed, effectively advancing the search process.

[0109] In one specific embodiment, the node weight at each position in the candidate graph node queue can be a pre-calculated fixed value, resulting in a score table between queue positions and node weights. When aggregating node weights, the scheduling weight corresponding to each server can be directly obtained by looking up the table and performing addition.

[0110] In one specific embodiment, during the offline preparation phase, a distributed vector graph index is built or loaded, and the distributed vector graph index is deployed to multiple servers.

[0111] Preferably, the distributed vector graph index may include graph nodes, an adjacency list, vector compression encoding, the original vector storage location, a query entry index (i.e., a head index), and a mapping table between graph nodes and servers. The query entry index is used to generate an initial candidate graph node queue.

[0112] Preferably, the distributed vector graph index can be generated using centralized graph construction, distributed graph construction, local graph fusion, or other graph index construction methods; this disclosure does not impose specific limitations on this. As long as the index can support adjacency access, distance calculation, and graph node affiliation identification for candidate graph nodes, it can serve as the basic index structure for the online query state adaptive scheduling of this disclosure.

[0113] The technical solution of this disclosure will be further explained below with a specific implementation example.

[0114] like Figure 8 As shown, the method disclosed herein includes offline preparation stages S10-S20 and online retrieval stages S30-S60. The offline preparation stage is used to construct or load a distributed vector graph index and generate a mapping relationship between graph nodes and retrieval nodes, providing a basic indexing environment for online retrieval.

[0115] Specifically, a distributed vector graph index may include graph nodes, an adjacency list, vector compression encoding, original vector positions, a query entry index (i.e., a head index), and a mapping relationship from nodes to servers. The distributed vector graph index can be generated using centralized graph construction, distributed graph construction, local graph fusion, or other graph index construction methods; this disclosure does not impose specific limitations on these methods. As long as the index can support the identification of the server to which candidate graph nodes belong during the online retrieval phase, and can access the adjacency information and vector data of candidate graph nodes on the corresponding server, it can serve as the basic index structure of this embodiment.

[0116] During the online retrieval phase, after receiving a user's query request, the system sends the request to the current execution node (i.e., the execution server). The current execution node generates an initial query state based on the query entry index (i.e., the header index). This initial query state includes at least the query vector, a candidate graph node queue, a set of visited nodes, a result set, retrieval parameters, and the current execution node identifier. The current execution node then selects candidate graph nodes to be expanded from the candidate graph node queue and determines the corresponding server for each candidate graph node based on the mapping relationship between graph nodes and servers.

[0117] The current execution node no longer determines whether to migrate the query state solely based on whether the candidate graph nodes to be expanded are located locally. Instead, it calculates the node weight based on the position of each candidate graph node in the candidate graph node queue, and aggregates the node weights according to the server to which the candidate graph nodes belong, obtaining the server scheduling weight corresponding to each server. The higher the ranking of a candidate graph node in the candidate graph node queue, the higher its corresponding node weight; the more high-weight candidate graph nodes a server has, the more the search front of the current query is inclined towards that server.

[0118] The current execution node determines, based on the scheduling weights of each server, whether subsequent searches should continue on the current execution node or the complete query state should be migrated to the target server for continued execution. If the server scheduling weight of the current execution node meets the condition for continued local execution, the current execution node continues to expand its local candidate graph nodes; if the server scheduling weight of the target remote server meets the condition for state migration, the current execution node sends the complete query state to the target remote server, which then becomes the new current execution node to continue subsequent graph traversal searches.

[0119] During the query state transition, the complete query state can include the query vector, candidate graph node queue, visited node set, result set, current distance threshold, retrieval parameters, and query identifier. After the new executing node takes over the query state, it continues to expand the candidate graph node queue, calculate server scheduling weights, and determine state transitions until the retrieval termination condition is met. Once the termination condition is met, the candidate results are fully reordered, and the final Top-K retrieval results are output.

[0120] Therefore, this embodiment enables the query state to adaptively flow across multiple servers based on the current search frontier by calculating node weights based on the candidate graph node queue position, calculating scheduling weights by server aggregation, and judging query state transitions. Compared to a fixed strategy that determines state transitions solely based on the presence of a local node in the Top-W candidate graph nodes, this approach more accurately determines which server is more suitable for the current query to continue execution, thereby reducing invalid state transitions and cross-server access overhead, and improving the throughput and retrieval stability of distributed vector graph retrieval.

[0121] Specifically, during the offline preparation phase, a distributed vector graph index is built or loaded, and then deployed to multiple retrieval nodes. The distributed vector graph index may include graph nodes, an adjacency list, vector compression encoding, the original vector storage location, a query entry index, and a mapping table between graph node identifiers (NodeID) and server identifiers (ServerID). The mapping table between NodeID and ServerID is used to determine the retrieval node (i.e., the server) to which a candidate graph node belongs during the online retrieval phase, and the query entry index is used to generate an initial queue of candidate graph nodes.

[0122] The distributed vector graph index can be generated using centralized graph construction, distributed graph construction, local graph fusion, or other graph index construction methods; this disclosure does not impose specific limitations on this. As long as the index can support adjacency access, distance calculation, and node affiliation identification of candidate graph nodes, it can serve as the basic index structure for the online query state adaptive scheduling of this invention.

[0123] Specifically, during the online retrieval phase, the system receives user query requests and distributes them to the current execution node (i.e., the execution server). User query requests can include query vectors, as well as text, images, audio, video, or other objects to be retrieved; when a user query request is not in vector form, the system converts it into a query vector. The current execution node generates initial candidate graph nodes based on the locally stored header index and determines the retrieval node (i.e., the server) to which each initial candidate graph node belongs by combining the graph node identifier NodeID with the server identifier ServerID mapping table, thus forming an initial candidate graph node queue.

[0124] The initial candidate graph node queue includes at least the candidate graph node identifier, the distance between the candidate graph node and the query vector q, the access status of the candidate graph node, and the server identifier ServerID corresponding to the candidate graph node. The currently executing node further generates an initial query state based on the query vector q, the initial candidate graph node queue, the set of visited nodes, the result set, the current executing node identifier, and retrieval parameters. The retrieval parameters include at least one of the following: the number of returned results K, the length of the candidate graph node queue L, the expansion width W per round, the scheduling judgment window M, the maximum number of accessed nodes, or the maximum query latency. Therefore, stage S30 provides the initial input for subsequent node scheduling based on the position weights of the candidate graph node queue, server scheduling weight calculation, and query state transition judgment.

[0125] Specifically, the current execution node selects candidate graph nodes that are close to the query vector and have not yet been expanded, based on the candidate graph node queue generated in stage S30. It then identifies the affiliation of these candidate graph nodes according to the mapping relationship between the graph node identifier (NodeID) and the server identifier (ServerID). For local nodes, the current execution node performs local graph traversal; for remote nodes, it calls the near data calculation service of the corresponding remote retrieval node, enabling the remote retrieval node to complete candidate expansion and distance calculation within a limited range based on its locally stored graph node data.

[0126] Specifically, local traversal and remote near data calculation include the following: The currently executing node selects the Top-W candidate graph nodes from the candidate graph node queue that are closest to the query vector q and have not yet been expanded. Here, W represents the candidate expansion width for each round of graph traversal, which can be preset according to retrieval accuracy requirements, query latency requirements, node load, or system configuration.

[0127] The current execution node identifies the top-W candidate graph nodes based on the mapping relationship between graph node identifiers (NodeID) and server identifiers (ServerID), determining the target retrieval node corresponding to each candidate graph node. If the ServerID corresponding to a candidate graph node is the same as that of the current execution node, the candidate graph node is identified as a local node; if the ServerID corresponding to a candidate graph node points to another retrieval node, the candidate graph node is identified as a remote node.

[0128] For local nodes, the currently executing node reads the locally stored graph node data and adjacency table, and performs distance calculations and candidate expansion on the local node and its neighboring nodes based on the query vector q. For new candidate graph nodes obtained from the expansion, the currently executing node filters them according to the distance results, the capacity of the candidate graph node queue, and the current distance threshold, and adds the new candidate graph nodes that meet the conditions to the candidate graph node queue. At the same time, the currently executing node updates the access status of the candidate graph nodes to avoid duplicate expansion.

[0129] For remote nodes, the currently executing node does not immediately migrate the complete query state. Instead, it groups the remote nodes according to the target ServerID and sends a near data calculation request to the corresponding remote retrieval node. The near data calculation request may include information such as the query state identifier QueryID, the query vector PQCode, the set of remote nodes, the current distance threshold, the maximum expansion depth, the maximum number of accessed nodes, and the maximum number of returned candidates.

[0130] After receiving a near-data calculation request, the remote retrieval node, based on its locally stored graph node data and adjacency list, performs distance calculation, candidate filtering, and local expansion on the specified remote node and its neighboring nodes within a limited range, and returns candidate summary information to the currently executing node. The candidate summary information may include the candidate graph node identifier (NodeID), the serverID to which the candidate graph node belongs, the candidate distance value, the access status of the candidate graph node, and necessary neighbor summary information.

[0131] In the above process, the remote retrieval node only performs near-data computation within a limited scope for the specified remote node. It does not maintain a complete candidate graph node queue, does not determine global termination conditions, and does not take over the complete query state. Therefore, remote near-data computation is different from query state migration. Its role is to perform parallel exploration and local expansion of remote nodes without transmitting complete graph node data or migrating the complete query state.

[0132] Through the steps described above, stage S40 implements local graph traversal for local nodes and near-data computation invocation for remote nodes. During this process, the local graph traversal of the currently executing node and the near-data computation of one or more remote retrieval nodes can be executed in parallel, thereby fully utilizing the local storage and computing capabilities of multiple retrieval nodes, reducing the cross-network transmission of complete graph nodes, and avoiding premature migration of the complete query state. This stage provides the foundation for the subsequent S50 stage's adaptive scheduling based on remote return results, candidate graph node distribution, and the current query state.

[0133] The current execution node receives candidate summary information returned by one or more remote retrieval nodes in phase S40 and merges the candidate summary information into the current candidate graph node queue. Based on the candidate graph node queue status, preset termination conditions, and the distribution of unexpanded candidate graph nodes, the current execution node determines whether to continue executing the retrieval and call the remote near data computation service, or to migrate the complete query status to the target remote retrieval node for continued execution.

[0134] Specifically, the remote result merging and state transition judgment includes: The current execution node receives candidate summary information returned by each remote retrieval node. The candidate summary information is calculated from the remote near data in stage S40, used to supplement the current candidate graph node queue, and provides a basis for subsequent candidate graph node queue updates, termination judgments, and state transition judgments.

[0135] The currently executing node merges remote candidate summary information into the current candidate graph node queue and filters them based on the distance between the candidate graph node and the query vector, the current distance threshold, the capacity of the candidate graph node queue, and the candidate access status. For candidate graph nodes that do not meet the filtering criteria, the currently executing node performs pruning; for candidate graph nodes that meet the filtering criteria, the currently executing node adds them to the candidate graph node queue and updates the sorting result and access status of the candidate graph node queue.

[0136] The currently executing node determines whether the current query meets the preset termination conditions based on the current candidate graph node queue status and retrieval budget. Preset termination conditions may include: no candidate graph nodes remain in the queue for further expansion; the number of expanded nodes reaches a preset upper limit; the current Top-K candidate results meet stability conditions; the query time reaches a preset upper limit; or a combination of the above conditions. If the current query meets the preset termination conditions, the graph traversal process stops, and the process enters the S60 full-precision rearrangement and result output stage.

[0137] If the current query does not meet the preset termination conditions, the current execution node determines the set of candidates to be scheduled based on the unexpanded candidate graph nodes in the candidate graph node queue, and calculates the server scheduling weight corresponding to each retrieval node based on the set of candidates to be scheduled, thereby determining whether to perform query state migration.

[0138] Specifically, such as Figure 9 As shown, the currently executing node selects the top M candidate graph nodes that are closest to the query vector q and have not yet been expanded from the candidate graph node queue, forming a candidate set to be expanded (i.e., a candidate set to be scheduled): Where L represents the set of candidates to be scheduled, and M represents the size of the scheduling decision window. M can be equal to or greater than the expansion width of each round. This represents the i-th candidate graph node in the candidate set to be scheduled. The earlier a candidate graph node is ranked in the candidate graph node queue, the closer it is to the query direction, and the greater its influence on subsequent search paths.

[0139] For candidate graph nodes in the candidate set L to be scheduled The currently executing node calculates its weight based on its sorting position i in the candidate graph node queue: in, Represents candidate graph nodes Position score, where i represents the candidate graph node The ranking position within the candidate graph node set to be scheduled. Since the candidate graph node set to be expanded is sorted from closest to furthest from the query vector, the candidate graph node ranked first... The position score is 1, and the candidate graph node ranked Mth is selected. Position score In other words, the earlier a candidate graph node is in the queue, the higher its position score.

[0140] After obtaining the position score, the currently executing node determines the position based on the position influence factor. The position scores are adjusted, and candidate graph nodes are obtained through normalization. Node weights: This represents the node weight of the candidate graph node. This represents the j-th candidate graph node in the candidate set to be scheduled. This indicates the location influence factor.

[0141] After the above normalization process, the sum of the node weights of all candidate graph nodes in the candidate set to be scheduled is 1, and the aforementioned position influence factor... Used to adjust the degree to which the position of a candidate graph node queue affects the node weight: The larger the value, the greater the influence of position on node weight.

[0142] Furthermore, the currently executing node determines each candidate graph node based on the graph node-to-server mapping table. The node to which it belongs is denoted as: ,express Belongs to the Each search node.

[0143] The currently executing node aggregates the node weights according to the retrieval node to which the candidate graph node belongs, to obtain the server scheduling weight corresponding to each retrieval node: in, Indicates the search node The scheduling weight of the server indicates that a larger server scheduling weight means that more high-priority, unexpanded candidate graph nodes are concentrated on the current candidate graph node queue, making the current candidate graph node more suitable as the execution node for subsequent queries. Let the current execution node be... The server scheduling weight corresponding to the currently executing node is The currently executing node selects the search node with the highest server scheduling weight from among the search nodes other than itself as the candidate target search node. : in, This indicates a candidate target retrieval node.

[0144] The current execution node is determined by its server scheduling weight. Server scheduling weights of candidate target retrieval nodes Determine whether query state transition is necessary. Query state transition is determined to be necessary when any of the following conditions are met: and or: and in, This represents the migration lag threshold (i.e., the first threshold), used to prevent the query state from frequently migrating between multiple retrieval nodes due to slight changes in server scheduling weights. This represents the server concentration threshold (i.e., the second threshold), which ensures that query state migration is only performed when high-priority, unexpanded candidate graph nodes are sufficiently concentrated in the candidate target retrieval node.

[0145] When the above query state transition conditions are met, the current execution node will retrieve the candidate target node. Once the target remote retrieval node is identified, the complete query state is migrated to the target remote retrieval node to continue execution. The complete query state may include the query vector. The database includes candidate graph node queue, result set, visited node set, current distance threshold, retrieval parameters, and query identifier QueryID.

[0146] When the above query state transition conditions are not met, the current execution node does not transition the complete query state, but continues to expand the unexpanded candidate graph nodes belonging to the current execution node on the local machine, and updates the candidate graph node queue, result set, and visited node set based on the expansion results; if the updated query state still does not meet the termination conditions, it will enter the next round of candidate set determination, server scheduling weight calculation, and state transition judgment.

[0147] In this way, the current execution node does not simply determine whether to migrate the query state based on the distribution of the number of unexpanded candidate graph nodes, but rather comprehensively considers the ranking position of the candidate graph node in the candidate graph node queue and the retrieval node to which the candidate graph node belongs. The higher the ranking of the candidate graph node, the greater its contribution to the server scheduling weight. Therefore, the server scheduling weight can more accurately reflect the distribution of the current search frontier among multiple retrieval nodes. Thus, this invention can adaptively determine whether to migrate the query state and to which retrieval node based on the current query state, thereby reducing invalid state migrations and improving the execution efficiency and stability of distributed vector graph retrieval.

[0148] When the S50 stage determines that the current query meets the preset termination conditions, the system stops expanding candidate graph nodes and determines the candidate graph nodes in the current candidate graph node queue or result set as the candidate set to be rearranged. The system obtains the corresponding original vector, high-precision vector representation, or full-precision distance calculation result based on the candidate graph node's identifier NodeID or vector data identifier VectorID.

[0149] Full-precision distance calculation can employ Euclidean distance, inner product distance, cosine distance, or other similarity metrics consistent with the database construction phase. The system sorts the candidate set to be rearranged based on the full-precision distance calculation results and selects the top K candidate graph nodes with the best distance or highest similarity as the final Top-K retrieval results.

[0150] Through the above steps, stage S60 completes the transformation from the approximate candidate set to the final retrieval results. This stage, together with stages S40 and S50, forms a complete retrieval process of "approximate graph traversal to recall candidates and full-precision rearrangement to output results," thereby improving the ranking accuracy of the final results while ensuring retrieval efficiency.

[0151] Furthermore, the method disclosed herein also provides a distributed vector graph retrieval system based on query state adaptive scheduling. For example... Figure 10 As shown, system 10 includes a collaborative graph building module 100, a logical single graph node organization module 200, a graph traversal and retrieval module 300, and a node scheduling module 400.

[0152] The collaborative graph building module 100 is used to partition the original vector dataset and build a multi-node collaborative graph. The collaborative graph building module 100 may include an overlapping clustering unit, a data distribution unit, a PQ vector compression unit, a local Vaman graph building unit, and an edge set fusion and pruning unit. After receiving the original vector dataset, the collaborative graph building module 100 performs overlapping clustering and node distribution on the original vector dataset, enabling multiple retrieval nodes to construct local Vaman graphs respectively. Based on the common nodes corresponding to the overlapping vectors, the module performs edge set fusion and pruning on the multiple local Vaman graphs to obtain a logical single graph.

[0153] The logical single-graph node organization module 200 is used to index and organize the logical single graph output by the collaborative graph building module 100, partition the graph nodes, and deploy the index data. The logical single-graph node organization module 200 may include a header index construction unit, a node partitioning unit, an affinity node allocation unit, a route mapping generation unit, and an index data deployment unit. Specifically, the header index construction unit selects a subset of graph nodes from the logical single graph using a breadth-first traversal approach to construct a header index; the node partitioning unit and the affinity node allocation unit determine the target deployment location of graph nodes based on the adjacency relationships between graph nodes, node affinity, and the capacity and load constraints of the retrieval nodes; the route mapping generation unit generates a mapping relationship from NodeID to ServerID; and the index data deployment unit distributes graph node shards, adjacency lists, PQCodes, original vector positions, header indexes, and mapping tables to the corresponding retrieval nodes.

[0154] The graph traversal retrieval module 300 is used to perform candidate generation, graph traversal, remote near data calculation invocation, and result output during the online query process. The graph traversal retrieval module 300 may include a user query access unit, a header index retrieval unit, a candidate graph node queue maintenance unit, a candidate attribution identification unit, a local graph traversal unit, a near data calculation invocation unit, a query status maintenance unit, and a candidate result fine-ranking and return unit. Upon receiving a query request, the graph traversal retrieval module 300 generates an initial candidate graph node queue based on the header index and identifies the attribution of candidate graph nodes according to the mapping relationship from NodeID to ServerID; for local nodes, it performs local graph traversal; for remote nodes, it invokes the near data calculation service of the corresponding remote retrieval node; after the retrieval meets the termination condition, it performs a full-precision re-ranking of the candidate set and outputs the final Top-K retrieval results.

[0155] The node scheduling module 400 assists the graph traversal and retrieval module 300 in distributing query entry points, calculating candidate attribution statistics, and controlling query state transitions. The node scheduling module 400 may include a query entry point distribution load balancing unit, a candidate attribution statistics unit, a state transition condition determination unit, a query state encapsulation unit, and a state takeover and transmission unit. Based on the current query state, the attribution distribution of nodes in the unexpanded candidate graph, the quality of remote candidates, and the node load, the node scheduling module 400 determines whether subsequent retrieval should continue to be executed by the currently executing node, continue to call the remote near-data computation service, or migrate the complete query state to the target remote retrieval node for continued execution.

[0156] During system operation, the collaborative graph building module 100 generates logical single graphs; the logical single graph node organization module 200 divides the logical single graph into nodes and deploys indexes; the graph traversal retrieval module 300 performs graph traversal retrieval based on user query requests, header indexes, and the mapping relationship from NodeID to ServerID; and the node scheduling module 400 schedules and controls the cross-node retrieval process according to the query status and the distribution of candidate graph nodes. Through the collaborative work of these modules, the system can achieve multi-node collaborative graph building, distributed deployment of logical single graphs, parallel candidate expansion both locally and remotely, and adaptive retrieval scheduling based on query status.

[0157] Based on any of the above embodiments, this disclosure also provides a distributed vector graph retrieval device.

[0158] Figure 11 This is a schematic block diagram of a distributed vector graph retrieval device according to one embodiment of the present disclosure.

[0159] As shown in Figure 11, the distributed vector graph retrieval device includes: The node selection module 1102 responds to the user's query request by selecting candidate graph nodes from the header index and determining the candidate graph node queue. The header index is determined by traversing the distributed vector graph, and the candidate graph node queue is sorted based on the distance to the query vector of the user's query request. The attribution identification module 1104 selects candidate graph nodes from the candidate graph node queue for attribution identification, and determines the remote graph nodes that are not in the execution server. The execution server is the server currently used to process user query requests. The request sending module 1106 sends a near data calculation request to the server corresponding to the remote graph node to obtain the remote graph node queue. The node merging module 1108 merges the remote graph node queue into the candidate graph node queue. The weight calculation module 1110 calculates the node weights of the candidate graph nodes to be processed in the merged candidate graph node queue based on the arrangement order of the merged candidate graph node queue when the merged candidate graph node queue does not meet the termination condition. The weight aggregation module 1112 aggregates the node weights based on the servers corresponding to the candidate graph nodes to be processed in the merged candidate graph node queue, and determines the scheduling weight of each server. The state transition module 1114 determines whether the migration conditions are met based on the scheduling weight of each server. If so, the query state containing the merged candidate graph node queue is migrated to the target server for graph retrieval. If not, the candidate graph nodes to be processed in the merged candidate graph node queue are expanded, and the expanded candidate graph node queue is re-determined to meet the termination conditions.

[0160] The aforementioned distributed vector graph retrieval device can be in the form of computer software, and each module of the aforementioned distributed vector graph retrieval device can be implemented through computer software modules.

[0161] The implementation process of the functions and roles of each module in the above distributed vector graph retrieval device is detailed in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0162] The executing entity of the distributed vector graph retrieval method in the specific embodiments of this disclosure can be an electronic device with computing and communication capabilities, such as a server, smartphone, tablet computer, vehicle terminal, or Internet of Things device.

[0163] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the distributed vector graph retrieval method of any of the embodiments described above.

[0164] Figure 12 This is a schematic block diagram of an electronic device 1000 according to one embodiment of the present disclosure.

[0165] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0166] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, this diagram uses only one connection line, but this does not imply that there is only one bus or one type of bus.

[0167] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0168] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. A computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0169] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, a computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. A readable storage medium can be any accessible medium or a data storage device such as a server or data center that integrates one or more accessible media. The accessible medium can be magnetic media, such as floppy disks, hard disks, and magnetic tapes; optical media, such as digital video discs; or semiconductor media, such as solid-state drives. The computer-readable storage medium can be volatile or non-volatile, or it can include both types of storage media.

[0170] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0174] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0175] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0176] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A distributed vector graphic search method, characterized by, The distributed vector graph is a global vector graph in which graph nodes and their adjacency relationships are deployed on different servers. The method includes: In response to a user query request, candidate graph nodes are selected from the header index to determine a candidate graph node queue. The header index is determined by performing graph traversal on the distributed vector graph, and the candidate graph node queue is sorted based on the distance to the query vector of the user query request. Candidate graph nodes are selected from the candidate graph node queue for attribution identification to determine remote graph nodes that are not located on the execution server, which is the server currently used to process the user query request; Send a near data calculation request to the server corresponding to the remote graph node to obtain the remote graph node queue; Merge the remote graph node queue into the candidate graph node queue; When the merged candidate graph node queue does not meet the termination condition, the node weights of the candidate graph nodes to be processed in the merged candidate graph node queue are calculated based on the order of the merged candidate graph node queue. Based on the servers corresponding to the candidate graph nodes to be processed in the merged candidate graph node queue, the node weights are aggregated to determine the scheduling weight of each server; and Based on the scheduling weights of each server, determine whether the migration conditions are met. If so, migrate the query status of the merged candidate graph node queue to the target server for graph retrieval. If not, expand the candidate graph nodes to be processed in the merged candidate graph node queue and re-determine whether the expanded candidate graph node queue meets the termination conditions.

2. The distributed vector map retrieval method of claim 1, wherein, In response to a user query request, candidate graph nodes are selected from the header index to determine the candidate graph node queue, including: In response to a user query request, candidate graph nodes are selected from the graph nodes in the header index based on the distance between the query vector of the user query request and the graph nodes in the header index. Based on the distance between the candidate graph nodes and the query vector of the user query request, the candidate graph nodes are sorted, and a candidate graph node queue is determined by combining the server corresponding to the candidate graph nodes.

3. The distributed vector map retrieval method of claim 1, wherein, Selecting candidate graph nodes from the candidate graph node queue for attribution identification, and determining remote graph nodes not located on the execution server, includes: From the candidate graph node queue, select candidate graph nodes whose distance to the query vector of the user query request is within the target range, and determine the set of graph nodes to be processed; The server identifier of the server corresponding to the candidate graph node in the set of graph nodes to be processed is compared with the server identifier of the execution server to determine the remote graph node that is not located on the execution server.

4. The distributed vector map retrieval method of claim 1, wherein, Also includes: Based on the query vector of the user query request, the distances between the local graph node in the execution server and the adjacent graph nodes of the local graph node are calculated respectively. The distance of the local graph node and the distance of its neighboring graph nodes are compared with the distances of the candidate graph nodes in the candidate graph node queue, and the neighboring nodes of the local graph node are merged into the candidate graph node queue.

5. The distributed vector map retrieval method of claim 1, wherein, When the merged candidate graph node queue does not meet the termination condition, the node weights of the candidate graph nodes to be processed in the merged candidate graph node queue are calculated based on the order of the merged candidate graph node queue, including: Select candidate graph nodes to be processed from the merged candidate graph node queue to determine the candidate set to be scheduled; Based on the order of the candidate graph nodes in the candidate set to be scheduled in the candidate graph node queue, calculate the position score of the candidate graph nodes in the candidate set to be scheduled; The position scores of the candidate graph nodes in the candidate set to be scheduled are normalized to determine the node weights of the candidate graph nodes in the candidate set to be scheduled.

6. The distributed vector map retrieval method of claim 1, wherein, When the merged candidate graph node queue meets the termination conditions, including: Based on the distance between candidate graph nodes in the merged candidate graph node queue, the merged candidate graph node queue is sorted to determine the target graph node queue; Select target graph nodes from the target graph node queue as search results.

7. The distributed vector graph retrieval method as described in claim 1, characterized in that, The termination conditions include: the merged candidate graph node queue does not contain any candidate graph nodes that can be further expanded, the number of expanded candidate graph nodes is equal to the target upper limit, and / or the query time reaches the target time limit.

8. The distributed vector map retrieval method of claim 1, wherein, The query state, including the merged candidate graph node queue, is migrated to the target server for graph retrieval, including: The query status, which includes the queue of merged candidate graph nodes, is serialized and encapsulated. The query status is used to record intermediate data of the user query request execution. The serialized and encapsulated query status is sent to the target server.

9. An electronic device, comprising: include: The memory stores execution instructions; as well as A processor that executes the execution instructions stored in the memory, causing the processor to perform the distributed vector graph retrieval method according to any one of claims 1 to 8.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the distributed vector graph retrieval method according to any one of claims 1 to 8.

Images