A graph data query method and system for super large degree nodes

Abstract

The application discloses a kind of graph data query method and system for super large degree node, method includes the following steps: parsing query request, obtaining target node and corresponding query condition;Based on target node and query condition, the virtual adjacency list directory corresponding to target node is inquired to determine the target adjacency fragment satisfying query condition;Based on target adjacency fragment, the adjacency query result of target node is obtained.In the above method, the mapping relationship between the virtual adjacency table corresponding to target node and multiple adjacency fragments is maintained by virtual adjacency list directory, and the target adjacency fragment that needs to be actually accessed is determined from multiple adjacency fragments according to query condition, reduces the adjacency edge reading irrelevant to current query condition, reduces read amplification and data access overhead in query process, improves the graph data query efficiency of super large degree node.

Description

Technical Field

[0001] This application relates to the field of graph query technology, specifically to a graph data query method and system for ultra-large degree nodes. Background Technology

[0002] With the rapid development of graph database technology, graph data has been widely applied in scenarios such as social networks, device relationship networks, financial risk control, the Internet of Things (IoT), and knowledge graphs. In these applications, nodes are typically connected by edges to describe relationships such as following, interaction, communication, transfer, access, and ownership. As the scale of graph data continues to expand, some nodes may be connected to a huge number of adjacent edges, thus forming ultra-large degree nodes. Since ultra-large degree nodes are widely present in star users in social networks, gateway devices in IoT, and device identifiers or account nodes in risk control graphs, improving the adjacency query efficiency of ultra-large degree nodes has become an important research direction in the field of graph databases.

[0003] Existing graph databases typically organize the relationships between nodes using adjacency lists, edge lists, edge indexes, or key-value mappings. A common approach is to store all adjacent edges of a given node in a single adjacency list. Furthermore, to improve query performance, existing technologies employ hash sharding based on node or edge identifiers to distribute edge data, support relation filtering through relation type indexes, and reduce access latency to hot nodes through caching mechanisms. When executing a query, the set of adjacent edges corresponding to a node is usually read first, and then filtering is performed based on conditions such as relation type, time range, edge attributes, and access direction.

[0004] However, the above-mentioned technical solutions still have significant drawbacks in scenarios with extremely large nodes. When the number of adjacency edges of a node is large, a single adjacency list can easily lead to severe read amplification problems. Even if the query only involves edge data within a partial relation or a specific time range, the system still needs to scan a large number of irrelevant edges, resulting in significant input / output overhead. Meanwhile, while simple hash sharding can achieve balanced data storage, it destroys the semantic locality of edge data, causing data for the same query scenario to be stored in a scattered manner, increasing the complexity of cross-shard access and result aggregation. Furthermore, existing technologies typically place filtering operations during the query execution phase, requiring the reading of a large number of candidate edges before filtering, further increasing the consumption of computational and storage resources. Summary of the Invention

[0005] This application addresses the shortcomings of existing technologies that use a single adjacency list for storing and querying ultra-large degree nodes, which can easily lead to read amplification, low query filtering efficiency, and high query resource consumption. It provides a graph database query method and system for ultra-large degree nodes.

[0006] To solve the above-mentioned technical problems, this application provides the following technical solution: In a first aspect, this application provides a graph data query method for ultra-large degree nodes, comprising the following steps: parsing a query request to obtain a target node and corresponding query conditions; the target node is an ultra-large degree node; wherein, the ultra-large degree node is a node whose node feature information meets preset activation conditions, and the node feature information is used to characterize at least one of the target node's adjacency data scale, adjacency access load, data growth status, and query performance; based on the target node and the query conditions, querying the virtual adjacency table directory corresponding to the target node to determine the target adjacency segment that meets the query conditions; wherein, the virtual adjacency table directory is used to indicate the mapping relationship between the target adjacency segment and the corresponding physical storage location; and obtaining the adjacency query result of the target node based on the target adjacency segment.

[0007] It is understood that the above implementation determines the target adjacency segment from the virtual adjacency list directory by querying the conditions, so that the query process does not need to scan the complete adjacency list of the target node, thereby reducing the actual data access range, reducing the reading of irrelevant adjacency edges, reducing read amplification and query resource consumption, and improving the adjacency query efficiency of ultra-large degree nodes.

[0008] As one possible implementation, the establishment of the virtual adjacency table directory includes: semantically dividing the adjacent edges of the target node to form multiple adjacency segments; determining the physical storage locations corresponding to the multiple adjacency segments respectively; establishing a mapping relationship between the segment identifier of each adjacency segment and the corresponding physical storage location; and establishing the virtual adjacency table directory based on the mapping relationship and the physical storage locations corresponding to the multiple adjacency segments.

[0009] It is understood that the above implementation proposes an organizational method that decouples logical adjacency relationships from physical adjacency segments and establishes a mapping through a directory, so that queries can locate specific physical segments from logical adjacency entry points, reducing indiscriminate access to the complete adjacency table.

[0010] As one possible implementation, the semantic partitioning includes partitioning the adjacent edges according to at least one of relation type, time window, and access scenario to form multiple adjacent segments, each with corresponding semantic attributes.

[0011] It is understood that the above implementation proposes a partitioning method for adjacent segments based on the semantic attributes of adjacent edges, which organizes adjacent edges with similar query semantics in a concentrated manner, making it easier to directly locate relevant segments according to query conditions and reducing the scanning of irrelevant adjacent edges.

[0012] As one possible implementation, determining the physical storage location corresponding to each of the multiple adjacent segments includes: determining the basic storage layer corresponding to the adjacent segments based on the access frequency and time freshness of the adjacent segments; and determining the scene overlay layer corresponding to the adjacent segments based on the access scenario of the adjacent segments.

[0013] It is understood that the above implementation proposes a way to determine the physical layout by combining access popularity, time freshness, access scenario and fragment size, so that fragments with different access characteristics adopt appropriate storage locations and reduce storage and access hotspots caused by excessively large single fragments.

[0014] As one possible implementation, determining the target adjacent segment that satisfies the query conditions includes: matching the query conditions with the semantic attributes corresponding to each adjacent segment in the virtual adjacent table directory to obtain a matching result; and determining the target adjacent segment from multiple adjacent segments corresponding to the virtual adjacent table based on the matching result.

[0015] It is understandable that the above implementation proposes a fragment filtering method that matches query conditions with directory semantic attributes, which narrows the range of data to be accessed before actually reading adjacent edges, thereby transforming post-query filtering into targeted access after directory filtering and reducing read amplification.

[0016] As one possible implementation, an access plan is generated based on the target adjacent segment, and the adjacent segment is accessed based on the access plan to obtain the adjacency query result of the target node. The method of generating the access plan includes the following steps: obtaining the segment attributes corresponding to the target adjacent segment and the execution query conditions corresponding to the query request; determining the access order and access control information of the target adjacent segment based on the segment attributes and the execution query conditions to generate the access plan; wherein, the segment attributes include query condition matching degree, segment priority, segment popularity, expected scan cost and consistency status, and the access control information includes sorting strategy, pagination strategy, maximum number of adjacent edges allowed to be scanned, early termination condition and rollback strategy.

[0017] It is understood that the above implementation proposes a control method for generating access plans based on fragment attributes and query execution requirements, so that queries prioritize accessing fragments with high matching degree and low access cost, and control query resource consumption through scanning restrictions, early termination and rollback strategies.

[0018] As one possible implementation, when the target adjacent segment includes multiple adjacent segments, the segment query results corresponding to the multiple target adjacent segments are strategically merged to obtain the adjacent query result; wherein, the strategic merging includes full merging, conditional merging, priority merging, pagination merging, deduplication merging, rate limiting merging, and consistency merging.

[0019] It is understandable that the above implementation proposes a strategic merging method for multi-fragment query results, which maintains the integrity, sorting stability, pagination continuity, result uniqueness and data consistency of query results when the underlying adjacent data is stored in a distributed manner.

[0020] As one possible implementation, before establishing the virtual adjacency list directory, the method further includes: obtaining node characteristic information corresponding to the node, wherein the node characteristic information includes node degree, adjacency query frequency, edge growth rate, and query latency; determining whether the node meets the preset activation conditions based on the node characteristic information; when the node meets the preset activation conditions, identifying the node as an ultra-high degree node, and establishing a corresponding virtual adjacency list directory for the ultra-high degree node.

[0021] It is understandable that the above implementation proposes a way to selectively enable the virtual adjacency table mechanism based on node size and access load, and only establishes the virtual adjacency table directory for nodes that meet the conditions, thereby reducing the additional overhead caused by fragmenting and maintaining the directory for ordinary nodes.

[0022] As one possible implementation, it further includes: obtaining the running status information corresponding to the adjacent segments; when the running status information meets the preset adjustment conditions, performing fine-grained splitting, segment merging, segment migration, and query processing strategy adjustment on the adjacent segments; and updating the virtual adjacency table directory according to the adjusted adjacent segments.

[0023] It is understood that the above implementation proposes a mechanism for dynamically adjusting the segment organization method based on the running status of adjacent segments, so that the granularity of adjacent segment partitioning, physical storage layout and query strategy can adapt to changes in data scale and access mode, and avoid the fixed splitting strategy from gradually becoming ineffective.

[0024] Secondly, this application provides a graph data query system for ultra-large degree nodes, including: The parsing module is used to parse the query request and obtain the target node and the corresponding query conditions; the target node is an ultra-large node; wherein, the ultra-large node is a node whose node feature information meets the preset activation conditions, and the node feature information is used to characterize at least one of the target node's adjacent data scale, adjacent access load, data growth status and query performance; The query module is used to query the virtual adjacency table directory corresponding to the target node based on the target node and the query conditions, so as to determine the target adjacency segment that meets the query conditions; wherein, the virtual adjacency table directory is used to indicate the mapping relationship between the target adjacency segment and the corresponding physical storage location; The acquisition module is used to obtain the adjacency query result of the target node based on the target adjacency fragment.

[0025] Understandably, in the above system, the target adjacent segments that meet the query conditions are determined by querying the virtual adjacency table directory, thereby avoiding indiscriminate scanning of the complete adjacency table of ultra-large degree nodes, reducing the amount of query data read and resource consumption.

[0026] This application, by adopting the above technical solution, has significant technical effects: By representing the complete adjacency relationships of ultra-large nodes as a virtual adjacency list at the logical level, and organizing adjacent edges into multiple adjacency segments according to semantic attributes, a virtual adjacency list directory is used to maintain the mapping relationship between the virtual adjacency list, adjacency segments, and physical storage locations. This allows query requests to determine the target adjacency segment before actually reading the adjacency data based on the query conditions. Therefore, this avoids the extensive unnecessary scanning of the complete adjacency list of ultra-large nodes by traditional adjacency queries, reducing read amplification, input / output overhead, cross-segment access overhead, and query resource consumption during the adjacency query process, thus improving the efficiency and stability of adjacency queries for ultra-large nodes. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a graph data query method for ultra-large degree nodes provided in this application; Figure 2 This is a structural diagram of a graph data query system for ultra-large degree nodes provided in this application. Detailed Implementation

[0029] The present application will be further described in detail below with reference to the embodiments. The following embodiments are explanations of the present application, but the present application is not limited to the following embodiments.

[0030] Currently, with the rapid development of internet recommendation systems and network security applications, the scale of graph data is constantly expanding, and graph databases are widely used for the storage, management, and querying of massive amounts of related data. In these application scenarios, graph data typically describes the relationships between entities in the form of nodes and edges. Nodes represent entities such as users, products, devices, or data objects, while edges represent relationships such as attention, interaction, transaction, access, or dependence between entities.

[0031] In practical applications, some nodes establish connections with a large number of other nodes, forming ultra-large nodes. For example, in social networks, users with a large number of followers form ultra-large nodes; in e-commerce platforms, popular products are associated with a large number of users; and in knowledge graphs, core entities typically establish connections with a large number of related entities. When performing adjacency queries on ultra-large nodes, traditional adjacency list storage methods usually store all adjacency edges of the node in a unified data structure. Queries require traversing and filtering a large number of adjacency edges, leading to problems such as amplified storage reads, high input / output resource consumption, and low query response efficiency.

[0032] To facilitate understanding of this application, the relevant concepts involved in this application are explained below: (1) A virtual adjacency list is a set of logical adjacency relationships established for a target node, used to represent the complete adjacency relationship between the target node and its associated nodes. The virtual adjacency list is a logical adjacency relationship organization structure, and the adjacency edges corresponding to the adjacency relationships can be divided into multiple adjacency segments.

[0033] (2) An adjacency segment is a data set formed by dividing the adjacent edges of a target node according to at least one semantic attribute. Semantic attributes may include one or more of relation type, time window, and access scenario. For example, the interaction relationship of the target node can be further divided into different adjacency segments according to the time window.

[0034] (3) Physical adjacency segments are the actual storage form of adjacency segments in the physical storage medium. Different adjacency segments can correspond to different physical storage locations, thereby avoiding storing all the adjacency edges of the target node in the same data structure.

[0035] (4) The virtual adjacency table directory is used to record the mapping relationship between the virtual adjacency table and multiple adjacency segments, and to record the semantic attributes and physical storage location of the adjacency segments. By querying the virtual adjacency table directory, the target adjacency segment to be accessed can be determined according to the query conditions without traversing all the adjacency edges of the target node.

[0036] (5) The access plan is a data access scheme generated after the target adjacent segments are determined based on the query conditions. It is used to determine the access range, access order and processing method of the query results of the target adjacent segments.

[0037] (6) Strategic merging is the process of combining query results returned by multiple target adjacent segments according to preset rules. Preset rules may include full merging, conditional merging, priority merging, pagination merging, or deduplication merging.

[0038] To address the issues of large storage reads, high input / output resource consumption, and low query efficiency in adjacency queries for ultra-large nodes, this application provides a graph data query method for ultra-large nodes. The method aims to construct a virtual adjacency table directory, divide the adjacent edges of the target node into multiple adjacency segments, and establish a mapping relationship between the virtual adjacency table and these segments. Upon receiving a query request, the method quickly locates the target adjacency segment that meets the query conditions, reducing unnecessary data scanning and improving the query efficiency of graph data.

[0039] like Figure 1 As shown, it includes the following steps: S101. Parse the query request, obtain the target node and the corresponding query conditions. The target node is a super-large node.

[0040] In one implementation, the system extracts a unique identifier for the target node from the query request. The target node refers to the graph node from which the query operation originates; in a social network scenario, it could be a user account node; in a device management scenario, it could be a gateway device node; and in a risk control scenario, it could be an account or device fingerprint node. The target node identifier typically exists as a string or integer in the conditional clauses or path start positions of the query statement. The system uses the target node identifier as the base key for subsequent queries of the virtual adjacency table directory and for locating physical adjacency fragments.

[0041] Secondly, the system extracts query direction conditions from the query request. In graph databases, edges between nodes have directional attributes. A query operation may only focus on outgoing edges originating from the target node, incoming edges pointing to the target node, or bidirectional edges containing both. The parsing phase needs to explicitly identify the restrictions on edge directions in the query statement so that only adjacent segments that meet the direction requirements are retained in subsequent segment filtering, avoiding the reading of edge data with irrelevant directions.

[0042] Secondly, the system extracts relation type conditions. Relationship types are used to distinguish different semantic connections between nodes, such as follow, interaction, connection, or affiliation relationships. A query request may explicitly specify one or more relation types, or it may be unrestricted and request the return of all types of relations. For very large nodes, relation type is the most fundamental dimension for splitting the adjacency list. Therefore, accurately identifying relation type conditions during the parsing phase directly determines which relation type fragments can be hit subsequently, thus significantly reducing invalid scans.

[0043] The system also needs to extract time-related conditions. Many graph query operations are inherently time-sensitive, such as querying interaction records from the last seven days, data reported in the last hour, or connections generated during a specific activity. The parsing phase needs to identify filtering conditions related to the time attributes of edges from the query statement. These time attributes can be edge creation time, update time, or event occurrence time. The system converts the time conditions into a standardized time range expression, which is then used to match the time windows registered in each segment of the virtual adjacency list directory, thereby filtering out candidate segments covering that time range.

[0044] The system also extracts access scenario identifiers. An access scenario is a semantic marker of the business context to which a query request belongs. The system can automatically identify the scenario category of the current query based on the query source, interface type, user operation type, or query execution plan characteristics. For example, a query from the recommendation service is marked as a recommendation recall scenario, a query from the risk control engine is marked as a risk diffusion scenario, and a query from the operations and maintenance console is marked as a device topology query scenario. After being explicitly extracted during the parsing phase, the access scenario identifier will serve as the matching basis for the scenario field in the virtual adjacency table directory, enabling the system to prioritize the selection of dedicated fragments optimized for specific scenarios.

[0045] After extracting the above information, the system integrates the results to form a structured query description object. This object contains complete information such as target node identifier, direction constraints, set of relation types, time range, target label, access scenario, sorting and pagination parameters, path hop count, and consistency level. This object, as the output of the parsing step, is passed to the access plan generation stage to drive the querying of the virtual adjacency table directory and the filtering of candidate segments.

[0046] S102. Based on the target node and query conditions, query the virtual adjacency table directory corresponding to the target node to determine the target adjacency segment that meets the query conditions; wherein, the virtual adjacency table directory is used to indicate the mapping relationship between the target adjacency segment and the corresponding physical storage location.

[0047] In one implementation, before establishing the virtual adjacency list directory corresponding to the target node, the process also includes identifying oversized nodes.

[0048] Specifically, the node feature information corresponding to each node in the graph database is obtained, the nodes that meet the preset activation conditions are determined based on the node feature information, and the nodes that meet the preset activation conditions are identified as super-large nodes, so as to establish a virtual adjacency list directory for super-large nodes.

[0049] The node characteristic information may include one or more of the following: node degree, adjacency query frequency, edge growth rate, access scenario indicators, and resource consumption indicators.

[0050] Node degree characterizes the size of the adjacent edges corresponding to a node, including out-degree, in-degree, total degree, and degree under a specified relation type. Adjacency query frequency characterizes how frequently an adjacency query is executed on a node within a preset time range, including the number of adjacency queries, the number of multi-hop path query hits, the number of visits per unit time, the 95th percentile response time for adjacency queries, and the 99th percentile response time for adjacency queries. Edge growth rate characterizes how the number of adjacent edges corresponding to a node changes over time, including the number of new edges per unit time, the growth rate for relation types, and the percentage of edge increments within a time window.

[0051] Access scenario metrics are used to characterize the query hit rate of target nodes in different business access scenarios and the activity level of the corresponding access scenarios, including the number of social interaction query hits, the number of device topology query hits, the number of risk control path query hits, the number of recommendation recall query hits, and the number of recent time window query hits.

[0052] Resource consumption metrics are used to characterize the frequency with which a target node is queried or hit in different business access scenarios, in order to reflect the usage characteristics of the target node's corresponding adjacency relationship in different access scenarios, including adjacency read I / O volume, cache usage, cross-shard access count, and query merge memory usage.

[0053] For example, for a given node, if it exceeds a preset score, it is classified as an overly large node. The score for an overly large node can be calculated using the following formula: SVS(v)=a1·norm(degree(v))+a2·norm(adjQueryCount(v))+a3·norm(pathHitCount(v))+a4·norm(edgeGrowth(v))+a5·norm(P99Latency(v)), Where SVS(v) is the super-degree node score, degree(v) represents the total degree of the node; adjQueryCount(v) represents the number of adjacency query visits; pathHitCount(v) represents the number of path query hits; edgeGrowth(v) represents the edge growth rate; P99Latency(v) represents the P99 latency of the node's adjacency query; a1, a2, a3, a4, and a5 are preset weight coefficients; and norm is the normalization function.

[0054] It should be noted that the weight parameters are determined as follows: the system is first initialized according to the business default values, such as a1=0.35, a2=0.20, a3=0.15, a4=0.15, a5=0.15; when the business is more concerned about the risk of multi-hop path expansion, a3 is increased; when the business is more concerned about the risk of write growth, a4 is increased; when the business is more concerned about the stability of online queries, a5 is increased.

[0055] A node is marked as an over-degree node when it meets any of the following conditions: 1. Degree(v) is greater than the preset degree threshold; 2. The degree(v) is higher than the 99th percentile of the degree of nodes with the same label; 3. SVS(v) is greater than the scoring threshold for ultra-large degree nodes; 4. The number of edges scanned in a single adjacency query exceeds the threshold; 5. The size of the node adjacency list exceeds the threshold for a single storage block or cache block.

[0056] It is understandable that the total degree of a node, the size of the node's adjacency list, and the number of edges scanned in a single adjacency query reflect the scale of the node's adjacency data; the number of adjacency query visits and the number of path query hits reflect the node's adjacency access load; the edge growth rate reflects the node's data growth status; and the adjacency query latency reflects the node's query performance.

[0057] Based on the above node characteristic information, determine whether the target node meets the preset activation conditions. When the target node meets the preset activation conditions, the target node is identified as a super-large degree node, and the virtual adjacency table mechanism is activated for the target node. Its adjacent edges are divided into multiple adjacency segments, and a corresponding virtual adjacency table directory is established. When the target node does not meet the preset activation conditions, the original adjacency data organization method can continue to be used for storage and query.

[0058] Therefore, the virtual adjacency table mechanism is enabled only for nodes with large adjacent edge sizes or high query loads, without having to perform adjacency segment partitioning and directory maintenance on all nodes in the graph database, thereby reducing the maintenance overhead of the virtual adjacency table directory.

[0059] The virtual adjacency table directory is used to maintain the mapping relationship between the virtual adjacency table corresponding to the target node and multiple adjacency segments, and to record the semantic attributes, physical storage location and query control information of each adjacency segment.

[0060] In one implementation, the virtual adjacency list directory may include one or more of the following: virtual adjacency list identifier, adjacency segment identifier, relation type, edge direction, time window, access scenario, physical storage location, merge priority, and consistency status. The virtual adjacency list identifier is used to determine the virtual adjacency list to which the adjacency segment belongs; the adjacency segment identifier is used to distinguish different adjacency segments corresponding to the same virtual adjacency list; the relation type, edge direction, time window, and access scenario are used to characterize the semantic range of the adjacency edges carried by the adjacency segment; the physical storage location is used to indicate the actual data area where the adjacency segment is stored; and the merge priority and consistency status are used to provide control for subsequent access plan generation and multi-segment query result processing.

[0061] For the target node v, its virtual adjacency list is logically composed of multiple adjacency segments, which can be represented as: , in, Let represent the i-th adjacent segment of node v, and n represent the number of adjacent segments.

[0062] It's important to note that a virtual adjacency list does not require all adjacent edges of the target node to be stored contiguously in the physical storage layer. Instead, it represents the complete adjacency relationship of the target node logically as a single virtual adjacency list, and divides and stores its corresponding adjacent edges in multiple adjacency segments according to preset semantic attributes. These adjacency segments can reside in different physical storage areas. The virtual adjacency list directory records the mapping relationship between the virtual adjacency list and the adjacency segments, as well as the semantic attributes and physical storage locations of each adjacency segment, ensuring that multiple physically dispersed adjacency segments logically still constitute the complete adjacency relationship of the target node.

[0063] The creation of a virtual adjacency table directory includes the following steps: (1) Semantically divide the adjacent edges of the target node to form multiple adjacent segments.

[0064] Adjacent edges can be divided according to at least one semantic attribute to form multiple adjacent segments; the semantic attribute may include at least one of relation type, time window, and access scenario. Different adjacent segments correspond to different semantic attribute ranges, so that subsequent queries can determine the target adjacent segment to be accessed based on the query conditions.

[0065] Example 1

[0066] In one implementation, the adjacent edges of the target node can be divided according to the relationship type of the adjacent edges. It should be noted that dividing according to relationship type does not mean creating a separate virtual adjacency table for each relationship type. Instead, a logical virtual adjacency table is maintained for the same target node, and multiple adjacency segments corresponding to different relationship types are registered in the virtual adjacency table directory corresponding to this virtual adjacency table. Thus, one virtual adjacency table can correspond to multiple adjacency segments, and these adjacency segments logically constitute the complete adjacency relationship of the target node.

[0067] For the target node v, its set of adjacent edges can be represented as: , Where Adj(v) represents the set of adjacent edges of the target node v, e m This represents the m-th adjacent edge of the target node v, where m represents the total number of adjacent edges of the target node v.

[0068] After partitioning the set of adjacent edges r according to relation type, it can be represented as: , in, This indicates that the relation type in the target node v is r. k The set of adjacent edges, k represents the number of relation types involved in the target node v. Each relation type can correspond to one or more adjacent segments; when the number of adjacent edges under the same relation type is large, or when it is necessary to further divide according to other semantic attributes such as time window and access scenario, the same relation type can also correspond to multiple adjacent segments.

[0069] For example, when a target node has follow, interaction, mention, and share relationships, adjacent segments corresponding to different relationship types can be formed respectively. For interaction relationships with large data scales or significant differences in access patterns, multiple adjacent segments can be formed based on time windows or access scenarios on the basis of relationship type division.

[0070] By dividing adjacent edges according to relation type, when a query request only involves a specific relation type, the target adjacent segment corresponding to the relation type can be directly determined based on the virtual adjacency table directory, without having to scan all the adjacent edges corresponding to the target node, thereby reducing the reading of data unrelated to the current query.

[0071] In one implementation, based on the division of relation types, the adjacent edges corresponding to the same relation type can be further divided according to time windows to form multiple adjacent segments corresponding to different time ranges.

[0072] The set of adjacent edges of relation type r in the target node v can be represented as: , in, This indicates that the relation type in the target node v is r. k And the set of adjacent edges corresponding to the p-th time window, where p represents the set of edges based on relation type r. k The number of time windows formed by the division.

[0073] It should be noted that the time window can be determined based on the creation time, update time, event occurrence time, last access time, or other time information that can characterize the temporal features of the adjacent edge. Different time windows correspond to different time ranges, so that adjacent edges under the same relation type can form multiple adjacent segments based on time characteristics.

[0074] The fixed time window can divide adjacent edges according to a preset fixed time length, such as a one-minute window, an hourly window, a one-day window, or a one-month window; the sliding time window can determine the dynamically changing time range based on the current time, such as a time window for the most recent five minutes, the most recent hour, the most recent seven days, or the most recent thirty days.

[0075] Furthermore, the business time window can be determined based on the time range corresponding to the specific business process, such as the activity period, equipment online cycle, risk control observation period, billing cycle, or session cycle; the hot and cold time windows can be divided according to the data access frequency or data timeliness of the time range corresponding to the adjacent edge, such as hot time window, warm time window, cold time window, and archiving time window.

[0076] It should be noted that the above different time window division methods are only examples. Those skilled in the art can set the corresponding time window range according to the actual business scenario, query frequency, and time distribution characteristics of adjacent data.

[0077] By further dividing the adjacent edges under the same relation type according to the time window, when the query request contains specific time conditions, the target adjacent fragments covering the time conditions can be directly filtered according to the virtual adjacency table directory without reading all historical adjacent edges under the relation type, thereby further narrowing the query scope of the adjacent data.

[0078] In one implementation, adjacent edges can be further divided according to access scenarios based on the actual query pattern corresponding to the target node, to form adjacent segments corresponding to different business query needs. Access scenarios can be determined based on one or more of historical query statements, business request sources, query execution plans, and access tags, and are used to characterize typical query methods or high-frequency access needs for the target node.

[0079] For the set of adjacent edges of relation type r and time window t in target node v, it can be further divided according to different access scenarios: , in, This indicates that the relation type in the target node v is r. k The time window is t p And the set of adjacent edges corresponding to the q-th access scenario, where q represents the number of access scenarios.

[0080] In one implementation, the scenario-based adjacency segment can be organized using a combination of primary adjacency segments and scenario reference segments. The primary adjacency segment stores the complete edge record of the adjacent edge; the scenario reference segment stores the edge identifier, sorting information, time information, and necessary filtering fields required for a specific access scenario. This approach avoids repeatedly storing complete data for the same adjacent edge in multiple scenario adjacency segments because it serves multiple access scenarios simultaneously, while enabling high-frequency access scenarios to quickly locate corresponding candidate adjacent edges through the scenario reference segment.

[0081] It's important to note that the division based on access scenarios is not simply a mechanical binning based on the attributes of adjacent edges themselves. Instead, it's based on the query needs and access patterns of target nodes in actual business operations, forming adjacent segments that can be directly matched by the query process. Therefore, access scenarios reflect how different business requests use the adjacent data, allowing subsequent queries to directly filter target adjacent segments that match the current query requirements based on the access scenarios.

[0082] For example, in a social graph scenario, access scenarios may include friend list query scenarios, follower list query scenarios, recent interaction query scenarios, recommendation recall query scenarios, mutual friend query scenarios, and hot event propagation path query scenarios, etc.

[0083] For example, in a device graph scenario, access scenarios may include device topology query scenario, gateway-attached device query scenario, abnormal communication query scenario, device alarm association query scenario, online status query scenario, and regional device aggregation query scenario, etc.

[0084] For example, in a risk control graph scenario, access scenarios may include account association extended query scenarios, device fingerprint association query scenarios, fund path query scenarios, blacklist diffusion query scenarios, and high-risk edge screening query scenarios, etc.

[0085] For example, in a knowledge graph scenario, access scenarios may include entity relationship expansion scenarios, specific relationship path query scenarios, entity recommendation scenarios, and attribute association query scenarios, etc.

[0086] By dividing adjacent edges according to the access scenario, the system can prioritize determining the target adjacent segments that match the actual business scenario corresponding to the current query request, thereby reducing access to adjacent data that is irrelevant to the current business query and improving the query hit efficiency in high-frequency scenarios.

[0087] In the above embodiments, a partitioning method based on the semantic attributes of adjacent edges to form adjacent segments is proposed, which enables adjacent edges with similar query semantics to be organized in a concentrated manner, making it easier to directly locate relevant segments according to query conditions and reducing the scanning of irrelevant adjacent edges.

[0088] (2) Determine the physical storage location corresponding to multiple adjacent segments.

[0089] Example 2

[0090] In one implementation, after forming multiple adjacent segments, the physical storage layout is determined according to the segment attributes corresponding to each adjacent segment. The segment attributes may include one or more of the following: popularity attribute, capacity attribute, timeliness attribute, scene attribute, consistency attribute, and load attribute.

[0091] Specifically, the physical storage location of adjacent segments can be determined by combining a base temperature layer, a scene overlay layer, and a capacity balancing layer.

[0092] First, the base temperature layer is determined based on the access frequency and time freshness of adjacent segments. Adjacent segments with high access frequency or relatively recent corresponding times can be identified as high-frequency segments and stored in physical storage areas with high access performance; adjacent segments that meet regular access requirements can be identified as ordinary segments and stored in regular physical storage areas; adjacent segments with low access frequency corresponding to historical time ranges or in archived state can be identified as cold segments and stored in data storage areas suitable for low-frequency access.

[0093] Secondly, based on the basic temperature layer, a scenario coverage strategy is determined according to the access scenarios corresponding to adjacent segments. For adjacent segments corresponding to specific high-frequency access scenarios, corresponding scenario tags can be set, and independent caching strategies, access priorities, or access rate limiting strategies can be configured for adjacent segments. Therefore, the same adjacent segment can simultaneously possess both basic temperature attributes and scenario attributes. For example, an adjacent segment can belong to both high-frequency segments and scenario-specific segments corresponding to a specific scenario. Consequently, scenario-specific segments are an additional access organization method above the basic temperature layer, rather than a mutually exclusive classification independent of high-frequency segments, ordinary segments, and cold segments.

[0094] Furthermore, when the data size or access load corresponding to an adjacent segment meets the preset secondary splitting conditions, the adjacent segment can be split into multiple sub-adjacent segments, and these sub-adjacent segments can be allocated to different physical storage areas. The preset secondary splitting conditions can be determined based on one or more of the following: the data capacity of the adjacent segment, the number of adjacent edges, query latency, access frequency, storage resources occupied during the query process, and the processing load of the physical storage area where the adjacent segment is located.

[0095] For example, when the query mainly requires reading in chronological order or pagination, the adjacent segments can be further divided according to time sub-windows; when the query mainly filters based on the category or group attribute of the target node, it can be divided according to the target node attribute or the group to which the target node belongs; when the query does not have obvious semantic filtering conditions and is mainly used to achieve storage load balancing, hash partitioning can be performed based on the target node identifier, so that different adjacent edges are assigned to multiple sub-adjacent segments.

[0096] Understandably, after determining the physical storage location of each adjacent segment or sub-adjacent segment, the corresponding physical storage location information can be associated with the adjacent segment identifier and recorded in the virtual adjacency table directory, so that the actual physical storage location of the target adjacent segment can be quickly determined during subsequent queries.

[0097] In one implementation, the physical storage layout of adjacent segments can be configured based on the access frequency, time freshness, access scenario, and capacity load of the adjacent segments. For example, the physical storage layout may include a high-frequency segment storage area, a normal segment storage area, a cold segment storage area, a scenario-specific segment area, and a cross-segment balancing area.

[0098] The high-frequency segment storage area can be used to store adjacent segments corresponding to relatively recent time windows, frequently accessed relation types, or high-frequency access scenarios. To improve the read efficiency of such adjacent segments, the high-frequency segment storage area can be set in a storage area with high access performance, such as a memory table, a high-performance storage area in a solid-state storage medium, or a high-priority cache area.

[0099] The general fragment storage area can be used to store adjacent fragments that meet the needs of regular graph queries and whose access frequency is within a normal range. The general fragment storage area can adopt conventional data storage structures, such as ordinary column families, ordinary edge tables, or ordinary key-value storage areas, to achieve a balance between storage cost and query performance.

[0100] Cold segment storage areas can be used to store adjacent segments that correspond to historical time windows, have low access frequency, or are in an archived state. Since the real-time access requirements for these adjacent segments are low, compressed storage methods or low-cost storage media can be used to store them, thereby reducing the storage resource consumption caused by storing a large number of historical adjacent edges for a long time.

[0101] Furthermore, for adjacent segments with high hit frequency in specific business scenarios such as recommendation, risk control, and device alarms, a scenario-specific segment area can be set up. This scenario-specific segment area can be configured with independent caching or access rate limiting strategies for the corresponding access scenarios, ensuring that queries related to specific business scenarios prioritize accessing the corresponding adjacent segments. It should be noted that the scenario-specific segment area is not an independent category mutually exclusive with high-frequency segment storage areas, ordinary segment storage areas, and cold segment storage areas, but rather serves as an additional storage and access strategy above the basic storage hierarchy. For example, an adjacent segment can belong to both high-frequency segments and simultaneously possess scenario attributes corresponding to a specific scenario.

[0102] Furthermore, when the data size or access load of a certain adjacent segment is still large, the adjacent segment can be further split, and the resulting multiple sub-adjacent segments can be allocated to different physical storage areas to achieve cross-shard load balancing. Secondary splitting can be performed based on the hash result of the target node identifier, the group to which the target node belongs, or a preset attribute range. For example, when the primary purpose is to achieve load balancing, a hash operation can be performed based on the target node identifier, and the adjacent edges can be allocated to different sub-adjacent segments based on the hash result; when queries are mainly filtered based on the target node category or group, the segmentation can also be based on the corresponding category, group, or attribute range.

[0103] In this embodiment, the physical storage layout described above can select appropriate physical storage locations for adjacent segments based on their access characteristics and data size. Furthermore, cross-segment adjustments can be performed when the access pressure or data size of adjacent segments changes, thereby reducing the access pressure caused by a single storage area carrying a large amount of adjacent data.

[0104] (3) Establish the mapping relationship between the segment identifier of each adjacent segment and the corresponding physical storage location.

[0105] In one implementation, after determining the physical storage location corresponding to each adjacent segment, a segment identifier is assigned to each adjacent segment, and the segment identifier is associated with the corresponding physical storage location identifier to form a mapping item. The physical storage location identifier indicates the actual physical partition, storage block, data bucket, or file segment where the adjacent segment is located. During subsequent queries, the corresponding physical storage location identifier can be obtained based on the adjacent segment identifier, thereby locating and accessing the corresponding adjacent edge data.

[0106] (4) Establish a virtual adjacency table directory based on the mapping relationship and the physical storage location of multiple adjacent segments.

[0107] Specifically, after establishing the mapping relationship between the virtual adjacency table and each adjacent segment, corresponding directory entries are generated according to each mapping relationship, and the directory entries are written into the virtual adjacency table directory corresponding to the target node to form directory data that can be retrieved in the subsequent query process.

[0108] In one implementation, a corresponding directory entry is generated for each adjacent segment. The directory entry records at least the virtual adjacency table identifier, the adjacent segment identifier, and the physical storage location corresponding to the adjacent segment. The virtual adjacency table identifier is used to determine the logical adjacency relationship to which the adjacent segment belongs, the adjacent segment identifier is used to distinguish different adjacent segments under the same virtual adjacency table, and the physical storage location is used to indicate the actual data area where the adjacent segment is stored.

[0109] Furthermore, based on the semantic segmentation results during the formation of adjacent segments, at least one of the following can be recorded in the directory entries: the relation type, time window, and access scenario corresponding to the adjacent segment. Thus, each directory entry not only represents the affiliation relationship between the virtual adjacency table and the adjacent segment, but also represents the semantic range of the adjacent edges contained in the adjacent segment and their corresponding physical storage locations.

[0110] For example, for a virtual adjacency table corresponding to the same target node, multiple directory entries can be formed in the virtual adjacency table directory. The first directory entry can correspond to an adjacency segment of the first relation type, the second directory entry can correspond to an adjacency segment of the second relation type, and the third directory entry can correspond to an adjacency segment corresponding to a specific time window under the first relation type. Each directory entry records the semantic attributes and physical storage location of the corresponding adjacency segment. When a query request for the target node is received subsequently, the directory entries in the virtual adjacency table directory can be filtered according to the query conditions to determine the target adjacency segment that meets the query conditions, and the target adjacency segment can be accessed according to the physical storage location recorded in the directory entry.

[0111] Furthermore, the virtual adjacency table directory can also record one or more of the following: merge priority, consistency status, version status, or access status of adjacent segments. Merge priority can be used to determine the order of subsequent access or result processing when multiple adjacent segments meet the query conditions; consistency status and version status can be used to characterize whether an adjacent segment is currently in a valid, migrated, split, or reorganized state; access status can be used to characterize whether the corresponding adjacent segment can be accessed normally. This information can be further used to generate subsequent access plans.

[0112] It's important to note that the virtual adjacency table directory doesn't just record the physical addresses of adjacent segments; it also records their logical ownership, semantic attributes, and physical storage information. Therefore, in subsequent queries, the corresponding virtual adjacency table directory can be determined first based on the target node. Then, the query conditions are matched against the semantic attributes recorded in the directory entries to identify the target adjacent segment. Finally, targeted access is performed based on the physical storage location of the target adjacent segment.

[0113] Through the above implementation method, multiple physically dispersed adjacent segments can be uniformly organized in the virtual adjacent table directory corresponding to the target node. This allows subsequent queries to first filter adjacent segments that match the query conditions through directory information, and then perform data reading based on the corresponding physical storage location, thereby reducing invalid scanning of all adjacent edges of the target node.

[0114] In another implementation, an access plan can be generated based on the target adjacency segment, and the target adjacency segment can be accessed according to the access plan to obtain the adjacency query result of the target node.

[0115] It should be noted that after determining one or more target adjacent segments from the virtual adjacency table directory based on the query conditions, the target adjacent segments are not accessed directly in an unordered manner. Instead, an access plan is generated based on the segment attributes of the target adjacent segments and the query requirements corresponding to the query request. The access plan is a data structure used to control the execution process of this graph data query. It describes the target adjacent segments that need to be accessed in this query, the access order of each target adjacent segment, and the access control method used when executing the query on the target adjacent segments.

[0116] Example 3

[0117] An access plan is generated based on the target adjacency segment, and the adjacency query result of the target node is obtained by accessing the target adjacency segment based on the access plan. The method of generating the access plan includes the following steps: (1) Parse the query request and obtain the query conditions.

[0118] In one implementation, when a query request for graph data is received, the query request is parsed to obtain the target node and query conditions corresponding to this query.

[0119] The target node represents the starting object for this adjacency query, while the query conditions limit the range of data to be retrieved from the adjacency relationships of the target node and the subsequent query execution method. By parsing the query request, the graph query statement submitted by the user or the upper-layer business system can be converted into structured query information that can be used for virtual adjacency table directory retrieval and access plan generation.

[0120] Specifically, query conditions may include semantic query conditions for filtering adjacent segments, and execution query conditions for controlling the subsequent query execution process. Semantic query conditions may include one or more of the following: query direction, relation type, time condition, target node label, edge attribute filtering conditions, and access scenario; execution query conditions may include one or more of the following: sorting requirements, pagination requirements, path expansion requirements, and query consistency requirements.

[0121] For example, when parsing a query request, the following can be extracted: starting node identifier, query direction, relationship type, time condition, target node label, edge attribute filtering condition, access scenario, sorting field, pagination parameters, path hop count, and query consistency requirements.

[0122] Among them, the starting node identifier is used to determine the target node corresponding to this query, and further determine the virtual adjacency table corresponding to the target node; the query direction is used to limit the incoming edge, outgoing edge, or bidirectional adjacency edge of the target node involved in this query; the relationship type is used to limit the association relationship represented by the edge to be queried; the time condition is used to limit the creation time, update time, event occurrence time, or other time range related to the query corresponding to the required adjacency edge; the target node label is used to limit the category to which the associated node connected to the target node belongs; the edge attribute filtering condition is used to further filter the adjacency edge according to the status, weight, score, or other attributes corresponding to the adjacency edge; and the access scenario is used to characterize the business access type to which this query belongs.

[0123] Furthermore, the sorting field is used to determine the arrangement of subsequent query results; the pagination parameter is used to limit the range of data to be returned in this query and the pagination position; the path hop count is used to characterize the adjacency levels that need to be expanded outward from the starting node during the graph query process; and the query consistency requirement is used to characterize the requirements of this query on the version and synchronization status of adjacent segment data.

[0124] For example, when a query request is used to find the associated nodes of a target node within a specified time range and corresponding to a specific relationship type, the target node, query direction, relationship type, and time condition can be parsed from the query request; if the query also requires only returning associated nodes of a specific category, the target node label can also be obtained; if it is necessary to further limit the query scope based on the weight or state of adjacent edges, edge attribute filtering conditions can also be obtained.

[0125] Furthermore, when the query request is used for a specific business scenario, the corresponding access scenario can be parsed; when the query request requires sorting according to preset fields and returning results in batches, the sorting fields and pagination parameters can be parsed; when the query request is a multi-hop path query, the number of path hops can also be parsed; when the query has specific requirements for data real-time performance or data version, the corresponding query consistency requirements can also be obtained.

[0126] The above method can be used to extract the target node information, adjacency data filtering information, and query execution control information contained in the query request in a structured manner, so that the subsequent query process can locate the corresponding virtual adjacency table directory based on the target node, and filter the target adjacency fragments and generate the corresponding access plan based on the query conditions.

[0127] (2) Query the virtual adjacency directory based on the query conditions and generate an access plan.

[0128] In one implementation, the virtual adjacency table directory corresponding to the target node is queried based on the target node and query conditions to determine the target adjacency segment that meets the query conditions, and an access plan is generated based on the target adjacency segment.

[0129] Specifically, based on the target node obtained in the preceding steps, the virtual adjacency table corresponding to the target node is determined, and the corresponding virtual adjacency table directory is obtained. The virtual adjacency table directory records the mapping relationship between the virtual adjacency table and multiple adjacency segments, as well as information such as the relationship type, time window, access scenario, physical storage location, segment priority, and consistency status of each adjacency segment. Therefore, query conditions can be matched with the directory entries in the virtual adjacency table directory to filter out the target adjacency segment that matches the current query request from the multiple adjacency segments corresponding to the virtual adjacency table.

[0130] In one implementation, when the query condition specifies a relationship type, adjacent segments that match the relationship type can be filtered in the virtual adjacency table directory; when the query condition also specifies a time range, adjacent segments whose time windows cover the query time range or have an intersection with the query time range can be further selected from the already filtered candidate adjacent segments; when the query condition also corresponds to a specific access scenario, scenario adjacent segments that match the access scenario can be further selected or given priority.

[0131] It should be noted that the above filtering process filters adjacency fragments based on multiple directory fields within the same virtual adjacency table directory corresponding to the target node, rather than creating and querying multiple independent virtual adjacency tables for different relationship types. This allows the target node to logically still correspond to a complete adjacency relationship, while during query execution, only the necessary adjacency fragments are selected based on the query conditions.

[0132] Furthermore, different target adjacency segment selection methods can be adopted according to different query requests. For example, when the query request specifies a relationship type, only the adjacency segments corresponding to the corresponding relationship type are selected; when the query request specifies a time range, only the time window segments covering the time range are selected; when the query request corresponds to a preset access scenario, the scenario-specific adjacency segments corresponding to that access scenario are selected first; when the query request needs to obtain the complete adjacency relationship of the target node, all relevant adjacency segments corresponding to the target node are selected.

[0133] When a query request has pagination restrictions, adjacent segments with higher priority can be selected first; when a query request is a multi-hop path query, adjacent segments with a high hit rate in the historical path query process can be selected first; when a query request has high consistency requirements, adjacent segments that have been synchronized or primary adjacent segments used to store valid and complete edge data can be selected first; when the segment information in the virtual adjacency table directory is incomplete or the target adjacent segment cannot be located normally, a rollback query can be performed through the original edge index or a full adjacency scan.

[0134] Furthermore, after identifying one or more target adjacent segments, an access plan is generated based on the target adjacent segments and the execution query conditions corresponding to the query request.

[0135] The access plan is a set of control information used to manage the actual execution of this graph data query. It transforms the query process, which would normally require scanning the complete adjacency list of the target node, into a targeted access process for one or more target adjacency segments. The access plan determines which target adjacency segments need to be accessed, in what order, how the query results for each segment are processed, under what conditions access will stop, and what fallback method is used when the normal access process fails. Therefore, the access plan is the intermediate control structure connecting the virtual adjacency list directory and the actual adjacency segment access process.

[0136] For example, the access plan may include a unique identifier for the virtual adjacency table, the set of target adjacency segments actually selected in this query, the access order of the target adjacency segments, the merging strategy for segment query results, the deduplication strategy, the sorting strategy, the pagination strategy, the query consistency level, the maximum number of adjacency edges allowed to be scanned in this query, and the rollback strategy.

[0137] The access order of target adjacent segments can be determined by one or more of the following factors: query condition matching degree, segment priority, segment popularity, expected scan cost, pagination requirements, sorting requirements, and consistency status. Specifically, target adjacent segments with a high degree of matching with the current query conditions, a high probability of data hit, or a low expected access cost can be prioritized to reduce invalid access to low-relevance adjacent segments.

[0138] For example, when multiple target adjacent segments correspond to different time windows, and the query request requires returning some adjacent query results in order of time from nearest to farthest, the target adjacent segments corresponding to the more recent time window can be accessed first; when the adjacent data that meets the pagination requirements or the number of results requirements has been obtained after accessing one or more high-priority adjacent segments, the access to the remaining low-priority adjacent segments can be stopped.

[0139] Furthermore, the access plan can also set the maximum number of adjacent edges allowed to be scanned in this query, thereby limiting the data read scale generated by a single query. The maximum number of adjacent edges can be determined based on one or more of the following: query type, user permissions, path hop count, current system load, and historical query costs.

[0140] For example, for single-hop adjacency queries with clearly defined query conditions, a relatively small scan limit can be set; for offline analysis queries that allow for greater processing resource overhead, a relatively large scan limit can be set; for multi-hop path queries, the scan limit for each subsequent hop can be gradually tightened as the number of path hops increases to suppress the continuous increase in the number of candidate adjacency edges during path expansion. When the system load is high or the access priority corresponding to the current query is low, the maximum number of adjacency edges allowed to be scanned for this query can also be reduced accordingly.

[0141] Furthermore, the query consistency level in the access plan can be determined jointly based on the consistency requirements in the query request, the business access scenario, and the current state of the target adjacent segment. For example, for business scenarios with high requirements for real-time data consistency, such as transaction risk assessment, access permission assessment, or immediate reading after data is written, access methods with higher consistency requirements can be prioritized, selecting adjacent segments corresponding to completed data synchronization or the current valid version; for business scenarios that allow for a certain synchronization delay, such as recommendation data retrieval, statistical analysis, or hotspot trend analysis, access methods that allow for a certain delay in data can be adopted.

[0142] When the target adjacent segment is being split, migrated, or reorganized, version verification and incremental data compensation can be added to the access plan. Specifically, the currently valid data version can be determined based on the segment version status recorded in the virtual adjacency table directory. When multiple adjacent segments before and after migration need to be accessed simultaneously, deduplication or validity judgment can be performed based on the adjacent edge identifier or version information, and incremental data generated during the migration process can be used to supplement the data, thereby reducing the impact of changes in the physical organization of adjacent segments on query completeness.

[0143] For example, when a query request is used to retrieve associated nodes of a target node with a specified relationship type within a preset time range, and requests the query results to be returned in a paginated manner according to a preset order, the system first determines the corresponding virtual adjacency table directory based on the target node, then filters the corresponding adjacency segments according to the relationship type, and then determines the target adjacency segments covering the corresponding time range based on the time condition; if the query also corresponds to a specific access scenario, then the adjacency segments corresponding to that scenario are further prioritized.

[0144] Subsequently, the access order of target adjacent segments is determined based on the matching degree between each target adjacent segment and the query conditions, segment priority, access frequency, expected scan cost, and pagination and sorting requirements. The maximum number of adjacent edges allowed to be scanned is determined based on query type, path hop count, and system load. When the query has consistency requirements, the query consistency level is also determined based on the current version status of the adjacent segments. Finally, the target adjacent segment set, access order, merging strategy, deduplication strategy, sorting strategy, pagination strategy, query consistency level, scan limit, and rollback strategy are organized into an access plan, which is used by subsequent query execution processes to access the corresponding target adjacent segments according to the access plan.

[0145] By using the above method, the semantic information in the virtual adjacency table directory can be used to first determine the target adjacency segment that matches the query conditions. Then, an executable access plan can be formed based on the target adjacency segment and the query execution requirements. This transforms the adjacency query for ultra-large degree nodes from scanning the entire adjacency table to performing ordered and controlled targeted access to the target adjacency segment, thereby reducing the reading of invalid adjacency data and lowering the query execution overhead.

[0146] Example 4

[0147] Furthermore, after generating the access plan, the target adjacency segments are accessed according to the target adjacency segments recorded in the access plan and the access order corresponding to each target adjacency segment. Here, a target adjacency segment is a portion of the adjacency data in the complete adjacency relationship of the target node. Multiple target adjacency segments logically belong to the virtual adjacency list corresponding to the target node, but in the physical storage layer, they can be stored in different storage areas, storage blocks, or edge data storage units. Therefore, when executing the access plan, the actual data area where the target adjacency segment is stored can be located based on the physical storage location corresponding to the target adjacency segment in the virtual adjacency list directory, and the corresponding adjacency edge data can be read, without needing to read the complete set of adjacency edges corresponding to the target node.

[0148] In one implementation, the query execution process first reads the identifier of the first target adjacency segment from the access plan, obtains the corresponding physical storage location based on the identifier, and reads the adjacent edges in the first target adjacency segment from the physical storage location. For the read adjacent edges, necessary data validation can be performed according to the edge attribute conditions, target node label conditions, or other query constraints in the query request to obtain the query result for the first segment. If the query result for the first segment has not yet met the result requirements corresponding to the query request, the next target adjacency segment is accessed according to the access plan until all target adjacency segments in the access plan have been accessed, or the currently obtained adjacency data has met the stopping conditions set in the access plan.

[0149] For example, the adjacent edges corresponding to the target node are stored as multiple adjacent segments, each corresponding to a different relationship type, time window, or access scenario. When a query request is used to obtain a certain type of adjacency relationship of the target node within a specified time range, the previous steps have already determined multiple target adjacent segments covering that time range based on the virtual adjacency table directory, and generated an access plan based on the time window corresponding to each target adjacent segment. This step first accesses the target adjacent segments that have a high degree of matching with the current query time range according to the access plan, and reads the adjacent edges in the target adjacent segments; if the target adjacent segment only covers a part of the requested time range, then the access plan continues to access the target adjacent segments covering the remaining time ranges. For other relationship type segments, other time window segments, or other access scenario segments that are not determined to be target adjacent segments, no reading operation is performed. Thus, the traditional "reading all adjacent edges of the target node and then filtering" can be transformed into a query method of "first determining the target adjacent segments according to the query conditions, and then reading the target adjacent segments in a targeted manner".

[0150] Furthermore, when the access plan includes multiple target adjacent segments, the segment query results corresponding to each target adjacent segment can be obtained separately. If the query results of each segment can directly satisfy the query request, the adjacency query result of the target node can be obtained based on the query results of each segment; if the query request has requirements for unified sorting, pagination, deduplication, or consistency, the query results of each segment can be further processed according to preset result processing rules. It should be noted that merging multiple segment query results can be used as a further implementation method of this embodiment, but is not a necessary prerequisite for accessing the target adjacent segment; when only one target adjacent segment is determined according to the query conditions, the query result corresponding to that target adjacent segment can be directly used as the adjacency query result of the target node.

[0151] In another implementation, when a target adjacent segment is detected to be in a split, migration, or reorganization state during the access process, the currently valid physical storage location can be determined based on the segment status recorded in the virtual adjacency table directory. For example, when the original adjacent segment is being split into multiple new adjacent segments, the version status recorded in the directory can be used to determine whether to access the original adjacent segment, the new adjacent segment, or both. Furthermore, the edge identification information can be used to avoid repeatedly returning the same adjacent edge, thus ensuring that query results conforming to the logical adjacency relationship of the target node can still be obtained even when the physical organization of the adjacent segments changes.

[0152] Furthermore, when complete target adjacency fragment information cannot be obtained from the virtual adjacency table directory, or when the target adjacency fragment in the access plan is in an inaccessible state, a preset fallback strategy can be triggered to perform a supplementary query using the original edge index or original adjacency data structure corresponding to the target node. Through this method, semantic adjacency fragments are used to reduce the adjacency edge scanning range under normal circumstances, while maintaining the executability of the query process even when directory information is abnormal or the adjacency fragment state changes, thus balancing the efficiency and reliability of adjacency queries for extremely large nodes.

[0153] Therefore, by accessing the target adjacent fragments according to the access plan, the query execution process can read only the part of the adjacent data related to the current query conditions, without scanning all the adjacent edges corresponding to the target node. This reduces the amount of invalid data read and storage access overhead in the adjacency query of ultra-large degree nodes, and improves the execution efficiency of graph data query.

[0154] Example 5

[0155] After accessing multiple target adjacent segments according to the access plan and obtaining the corresponding segment query results, the query results of multiple segments can be strategically merged according to the result organization requirements corresponding to the query request, so as to obtain the adjacency query results that conform to the logical adjacency relationship of the target nodes.

[0156] Strategic merging is not simply concatenating multiple query fragments; rather, it selects, filters, sorts, deduplicates, compensates, or terminates multiple query fragments based on one or more of the query scope, sorting requirements, pagination requirements, deduplication requirements, consistency requirements, and result quantity requirements. Since the complete adjacency relationship of a target node can be split into multiple adjacency fragments at the physical storage layer, when a query hits multiple target adjacency fragments simultaneously, strategic merging is needed to reorganize the data returned by each target adjacency fragment into query results that conform to the adjacency semantics of the original graph data.

[0157] It should be noted that strategic merging is not required for every adjacency query. When only one target adjacency segment is determined based on the query conditions, and the segment query results corresponding to that target adjacency segment can already satisfy the query request, the segment query results can be directly used as the adjacency query results for the target node. When the query request hits multiple target adjacency segments, or when the query request includes requirements such as full query, unified sorting, pagination, deduplication, consistency control, or result quantity limits, strategic merging can be performed.

[0158] In one implementation, when a query request is used to obtain the complete adjacency relationship of a target node, a full merging approach can be adopted. Specifically, all relevant adjacency fragments corresponding to the target node are read according to the access plan, and the fragment query results corresponding to each adjacency fragment are combined to form the logical complete adjacency relationship of the target node. Full merging can be applied to business scenarios that require obtaining complete adjacency data, such as exporting all friend relationships, expanding complete device topology, performing complete association risk analysis, or offline graph calculation tasks. In such scenarios, the query objective is not to reduce the number of returned results, but to ensure the recoverability of the complete adjacency relationship of the target node; therefore, it is necessary to access and integrate all relevant adjacency fragments corresponding to the target node.

[0159] In another implementation, when the query request includes filtering conditions such as relation type, time range, edge attributes, or target node labels, a condition merging method can be used. Specifically, only the query results corresponding to the target adjacent segments that meet the query conditions, as determined in the aforementioned directory filtering step, are merged, while other adjacent segments that do not meet the query conditions are not processed. For example, when the query request is used to obtain associated nodes of a target node with a specified relation type within a preset time range, multiple target adjacent segments that match the specified relation type and whose time windows cover the preset time range can be determined first based on the virtual adjacent table directory. Then, only the results returned by the aforementioned target adjacent segments are combined. This avoids reading relation type segments or historical time segments that are irrelevant to the current query in order to form query results.

[0160] Furthermore, when the number of query results is large, and the query request only requires a subset of results determined according to a preset sorting rule, a priority merging method can be used. Specifically, based on the access priority of adjacent segments recorded in the access plan, query results of different target adjacent segments are read and merged sequentially. For example, high-frequency access segments can be read first, followed by adjacent segments corresponding to more recent time windows, then ordinary adjacent segments, and cold adjacent segments can be read as needed. When the query results that meet the preset number of results or pagination requirements have been obtained, the remaining low-priority adjacent segments in the access order can be stopped from being read. In this way, unnecessary segment access and data reading can be reduced while ensuring that the query results meet the sorting or number requirements.

[0161] When a query request requires paginated return of adjacent data, a pagination merging approach can be used. Specifically, the access position of each target adjacent segment can be maintained separately, and the data to be returned for this pagination is determined based on a unified sorting rule among the query results of multiple segments. After completing a pagination query, the currently accessed position of each target adjacent segment and the pagination status corresponding to the global result are recorded. When a request for the next page is received, data continues to be read from the current position corresponding to each target adjacent segment, and the next page result is formed according to the unified sorting rule. This avoids the problem of discontinuous pagination order across segments caused by multiple physical adjacent segments maintaining their own data positions.

[0162] In one implementation, when duplicate records or references to the same adjacent edge may exist in different types of adjacent segments, a deduplication and merging method can be used. For example, if an adjacent edge exists in both an adjacent segment formed according to relation type and is referenced by a scene adjacent segment corresponding to a specific access scenario, then when merging multiple segment query results, it can be determined whether different result items correspond to the same adjacent relationship based on one or more of the adjacent edge identifier, associated target node identifier, relation type, and time information. Duplicate results can be deleted or only a valid result can be retained, thereby preventing the same adjacent relationship from being returned repeatedly in the final query results.

[0163] Furthermore, when a single query involves a large number of target adjacent segments or is expected to scan a large number of adjacent edges, a rate-limiting merging method can be adopted. Specifically, based on the maximum scanning range, result quantity conditions, or resource usage limits set in the access plan, the scale of data that can continue to be accessed and merged in this query is controlled. When the number of scanned adjacent edges reaches a preset limit, or when the current query results meet a preset termination condition, access to subsequent adjacent segments can be stopped. This method is applicable to query scenarios that may generate a large number of candidate adjacent edges, such as multi-hop path expansion, risk relationship diffusion, social propagation link analysis, or device anomaly diffusion analysis, to limit the computing resources and storage access overhead generated during the adjacency relationship expansion process.

[0164] Furthermore, when the target adjacent segment is undergoing migration, splitting, or reorganization, a consistent merging approach can be adopted. Specifically, the data version to be read can be determined based on the segment version status and consistency status recorded in the virtual adjacency table directory. For example, only valid versions of adjacent segments that have been committed can be read; when an adjacent segment is in the process of migration and both the old and new segments exist, valid data before and after the migration can be read separately, and deduplication can be performed based on the adjacent edge identifier; when there is incremental data that has not yet been fully merged into the new segment during the migration or splitting process, the corresponding incremental record can be read and compensation processing can be performed; for queries with high consistency requirements, the primary adjacent segment that stores the currently valid data can be accessed first; for queries that allow a certain data synchronization delay, the currently available data replica segment can be accessed.

[0165] For example, when a query request is used to retrieve adjacency data of a target node within a preset time range and requests paginated return of query results in chronological order, multiple target adjacency segments covering the preset time range can be determined first based on the virtual adjacency table directory. Then, the segment query results of each target adjacency segment are read according to the access plan. After multiple segment query results are formed, data that does not meet the query requirements can be excluded first according to the relationship type and time condition. Then, deduplication can be performed based on the adjacency edge identifier. Subsequently, the remaining results are sorted according to a unified time order, and the data to be returned this time is truncated according to the pagination parameters. When the required results for this pagination have been obtained, reading of low-priority adjacency segments that have not yet been accessed can be stopped, and the current access position of each target adjacency segment can be recorded for subsequent pagination queries.

[0166] By employing the aforementioned strategic merging method, even when the underlying adjacency edges are split and stored across multiple physical adjacency fragments, the query results still maintain the integrity, sequential consistency, pagination continuity, result uniqueness, and data consistency required by the original logical adjacency relationship of the target node. This allows the virtual adjacency table to form a unified query view of multiple physical adjacency fragments at the logical level.

[0167] Example 6

[0168] In one implementation, once the target node has established a virtual adjacency table and a corresponding virtual adjacency table directory, for data changes such as adding, deleting, or updating adjacency edge attributes, the changed adjacency edges are no longer uniformly written into the single adjacency table corresponding to the target node. Instead, based on the semantic attributes, routing rules, and physical storage location of the adjacency segments recorded in the virtual adjacency table directory, the changed adjacency edges are targeted to ensure that the organization state of the adjacency segments in the virtual adjacency table directory corresponds to the actual stored adjacency edge data.

[0169] (1) Write new adjacent edges.

[0170] In one implementation, when a write request for a new adjacent edge is received, the adjacent edge is first parsed to obtain edge feature information such as the starting node, ending node, relationship type, edge direction, time information, and business access scenario.

[0171] Among them, the start node and the end node are used to determine the two graph nodes connected by the adjacent edge; the relationship type is used to characterize the association relationship between the two nodes; the edge direction is used to characterize the incoming edge relationship or outgoing edge relationship corresponding to the adjacent edge; the time information may include the edge creation time, the event occurrence time, or other time attributes that participate in the time window division; the business access scenario is used to determine whether the adjacent edge needs to be organized into the adjacent segment corresponding to the specific scenario.

[0172] After obtaining the aforementioned edge feature information, it is determined whether the starting or ending node of the adjacent edge belongs to a super-large degree node with the virtual adjacency list mechanism enabled. When the corresponding node does not have the virtual adjacency list mechanism enabled, the adjacent edge can be written into the corresponding ordinary adjacency list according to the ordinary adjacency data storage method; when the corresponding node belongs to a super-large degree node, the corresponding virtual adjacency list is determined based on the node, and the virtual adjacency list directory corresponding to the virtual adjacency list is read.

[0173] Subsequently, based on attributes such as the relationship type, time information, and access scenario of adjacent edges, the data is matched with directory entries in the virtual adjacency table directory to determine the target adjacent segment for storing that edge. After determining the target adjacent segment, the new adjacent edge is written to the physical storage location corresponding to the target adjacent segment, and the segment status information of that adjacent segment is updated accordingly. The segment status information may include one or more of the following: the number of adjacent edges, the time range covered by the segment, the segment version information, and the segment access popularity information.

[0174] For example, if a new adjacency edge corresponds to a certain relationship type and its event occurrence time falls within the first time window, and the adjacency edge also corresponds to a preset business access scenario, then the candidate adjacency fragments corresponding to the relationship type can be determined first in the virtual adjacency table directory. Then, the candidate adjacency fragments whose time windows cover the event occurrence time can be determined. Finally, the target adjacency fragments that need to be written can be determined by combining the business access scenario. This avoids adding the new adjacency edge to the complete adjacency data set of the target node indiscriminately.

[0175] If no target adjacency segment matching the newly added adjacency edge is found in the virtual adjacency table directory, a new adjacency segment can be created, and its semantic attributes, physical storage location, and mapping relationship with the virtual adjacency table can be written to the virtual adjacency table directory. Furthermore, if the data size or access load of the target adjacency segment exceeds a preset condition after the newly added adjacency edge is written, a secondary split of the target adjacency segment can be triggered.

[0176] After writing new adjacent edges or adjusting the directory, the change information of the virtual adjacency table directory can also be written to the metadata change log to record changes such as the addition, modification, redirection or invalidation of directory entries, so as to facilitate subsequent version verification, fault recovery or migration status tracking.

[0177] (2) Delete adjacent edges.

[0178] In one implementation, when an adjacent edge deletion request is received, the deletion request is parsed to obtain the edge identifier, starting node, ending node, relationship type, edge direction, and time information of the adjacent edge to be deleted.

[0179] When the starting or ending node of an adjacent edge is a super-large node with the virtual adjacency list mechanism enabled, the corresponding virtual adjacency list is determined based on the node, and the virtual adjacency list directory is read to determine the main adjacency segment where the adjacent edge to be deleted is located and the scene reference segment that may reference the adjacent edge.

[0180] The primary adjacency segment stores the complete edge record of adjacent edges, while the scene reference segment stores only the reference information used for quick lookup in a specific access scenario. Therefore, when the same adjacent edge exists in both the primary adjacency segment and one or more scene reference segments, the deletion process needs to process these multiple data locations simultaneously.

[0181] Specifically, a deletion flag, deletion version information, and deletion time information can be written into the main adjacency segment for the adjacent edge to be deleted, so as to indicate that the adjacent edge has become invalid through logical deletion. For physical storage structures that support direct deletion of edge records, the corresponding edge record can also be directly deleted from the main adjacency segment. For reference segments in scenarios that reference adjacent edges, the edge identifiers can be deleted accordingly, or the corresponding reference record can be marked as invalid.

[0182] After completing the above deletion process, update the number of adjacent edges, version status, time range, and popularity information of the affected adjacent segments, and record the corresponding virtual adjacency table directory change information.

[0183] Furthermore, when the adjacent segment containing the edge to be deleted is currently undergoing migration, splitting, or reorganization, the deletion operation can be simultaneously recorded in the incremental change log. During the query process, when it is necessary to access both the old and new adjacent segments before and after the migration, the corresponding adjacent edge is determined based on the deletion version information to ensure that the deleted adjacent edge does not reappear in the query results because the old adjacent segment still exists.

[0184] (3) Update the properties of adjacent edges.

[0185] In one implementation, when an adjacent edge attribute update request is received, the adjacent segment where the adjacent edge is currently located is first located based on the adjacent edge identifier and the original edge feature information, and it is determined whether the attribute being updated this time participates in the adjacent segment routing.

[0186] Among them, the attributes that participate in the routing of adjacent segments refer to the attributes that affect which adjacent edge should be stored in which adjacent segment, such as relationship type, edge direction, time window, access scenario, target node label, or other attributes used for semantic partitioning; the attributes that do not participate in the routing of adjacent segments can include weight, state, score, and other edge attributes used only for sorting, filtering, or business calculation.

[0187] When the update only involves attributes that are not involved in routing, the adjacent fragment containing the adjacent edge can be kept unchanged. The corresponding edge record can be updated directly in the original adjacent fragment, and the version information corresponding to that edge can be incremented or updated simultaneously. If the updated attribute is also involved in query sorting or filtering, the corresponding sort key or filter field can also be updated synchronously.

[0188] When the update involves attributes related to adjacent segment routing, the target adjacent segment corresponding to the adjacent edge needs to be redefined. For example, if the relationship type, time attribute, or access scenario of an adjacent edge changes, the original adjacent segment may no longer satisfy the updated edge feature information. In this case, the old edge records in the original adjacent segment can be marked as moved out or deleted, and the virtual adjacency table directory can be queried again based on the updated relationship type, edge direction, time window, access scenario, or target node label to determine the new target adjacent segment. The updated edge records are then written to the new target adjacent segment.

[0189] When an adjacent edge is also referenced by one or more scene reference fragments, the edge reference information in the corresponding scene reference fragment and the relevant metadata in the virtual adjacency table directory can be updated synchronously.

[0190] Furthermore, to reduce the possibility of multiple valid versions of the same adjacent edge due to concurrent updates, edge attribute update operations can carry version information before the update, and version verification can be performed during update execution. The update is executed when the current valid version matches the original version carried in the update request; if they do not match, the current valid version is reread according to preset concurrency control rules, or the update operation is terminated. Alternatively, an atomic conditional update method can be used, ensuring that only one update result meeting the version condition is valid at any given time.

[0191] (4) Target adjacent segment location.

[0192] In one implementation, when performing a new edge write or a route attribute update, the target virtual adjacency table can be determined first based on the start node or end node corresponding to the adjacent edge, and the virtual adjacency table directory corresponding to the virtual adjacency table can be read.

[0193] Subsequently, the directory items are filtered based on the directory fields such as edge direction, relationship type, time window, access scenario, and target node label corresponding to the adjacent edges, thereby obtaining one or more candidate adjacent segments.

[0194] For example, we can first exclude directory items whose direction is inconsistent with the current adjacent edge based on the edge direction, and then filter the corresponding relation type adjacent segments based on the relation type; then, we can determine the time window adjacent segments that can cover the time information based on the edge time information; when the adjacent edge corresponds to a specific access scenario, we can further determine the candidate adjacent segments that match the access scenario.

[0195] It should be noted that the filtering order of the above semantic attributes can be adjusted according to the actual implementation method, and a fixed order is not required. The ultimate goal is to determine the target adjacent segment that matches the semantic attributes of the adjacent edge to be written through the virtual adjacency table directory.

[0196] When an adjacent edge needs to be recorded in both the main adjacency segment and the scene reference segment, the complete edge record can be written to the main adjacency segment, while the scene reference segment only records the edge identifier, sorting information, time information, and the necessary filtering information required to execute the corresponding scene query. This avoids repeatedly saving the complete data of the same adjacent edge in multiple segments to support multiple access scenarios.

[0197] When the same semantic range further corresponds to multiple capacity-balanced sub-adjacent segments, the final physical sub-adjacent segment to be written can be determined according to the corresponding routing rules.

[0198] When there are multiple candidate adjacent segments that meet the semantic conditions, the final target adjacent segment can be determined by using preset selection rules.

[0199] For example, for write operations with high consistency requirements, the primary adjacency segment with a normal consistency status and a newer version can be selected first; for multiple candidate adjacency segments with an inclusion relationship in the time window, the adjacency segment that can cover the time information of the adjacency edge and has a smaller time range can be selected first, so that the time segment in which the adjacency edge is written has a more accurate semantic range; for access scenarios, the scenario adjacency segment that completely matches the business scenario of the adjacency edge can be selected first, and when there is no completely matching scenario, a preset default scenario segment can be selected.

[0200] When the same semantic adjacency segment is further divided into multiple capacity-balanced sub-adjacency segments, hash selection or range selection can be performed based on a preset routing key to determine the final physical sub-adjacency segment. When multiple candidate adjacency segments meet the semantic requirements but have different current loads, the adjacency segment with lower current storage load or access load can be prioritized among the candidate adjacency segments that meet semantic consistency.

[0201] (5) Creation of new adjacent segments and directory initialization.

[0202] In one implementation, when a target adjacency segment matching the adjacency edge to be written cannot be determined based on the virtual adjacency table directory, a new adjacency segment can be created based on the semantic attributes corresponding to the adjacency edge to be written.

[0203] Specifically, a fragment identifier corresponding to a new adjacent fragment can be generated based on one or more of the following: virtual adjacency list identifier, edge direction, relationship type, time window, access scenario, splitting level, and fragment sequence number. By associating the above information with the fragment identifier, a clear correspondence can be maintained between the new adjacent fragment and its topping virtual adjacency list and its corresponding semantic scope.

[0204] After generating a new adjacent segment, its physical storage location is determined based on the access frequency, expected capacity, access scenario attributes, and current physical partition load of the new adjacent segment.

[0205] For example, for new adjacent segments expected to be accessed frequently or corresponding to a relatively recent time range, a high-performance physical storage area can be prioritized; for regularly accessed segments, a general physical storage area can be selected; for historical or low-frequency adjacent segments, a cold data storage area can be selected; for new adjacent segments with a clear business scenario, the corresponding scenario storage configuration can also be determined according to the scenario coverage strategy. When a new adjacent segment belongs to a sub-adjacent segment formed by capacity balancing, the corresponding physical partition can be selected within a preset partition group, and when multiple available physical partitions exist, the physical partition with the lower load is prioritized.

[0206] After determining the physical storage location of the new adjacent segment, a corresponding directory entry is created in the virtual adjacency list directory. The directory entry can record information such as the virtual adjacency list identifier, adjacent segment identifier, semantic attributes, physical storage location, version status, and consistency status.

[0207] The virtual adjacency list directory itself can be stored in an independent metadata management structure or in a data table used to manage graph data metadata. It can also be cached in query execution nodes or storage nodes to reduce the query overhead caused by frequent access to directory data.

[0208] The physical storage location in the directory entry can point to the actual storage object such as the edge data bucket, key-value storage prefix, columnar data set, file data segment, or physical partition number that actually stores adjacent segments, so that the subsequent query process can locate and access adjacent segments based on the directory entry.

[0209] (7) Secondary splitting of adjacent segments.

[0210] In one implementation, when the data size or access overhead of an adjacent segment continues to increase, a secondary split can be performed on that adjacent segment.

[0211] Specifically, the pre-defined secondary splitting conditions can be determined based on the number of adjacent edges, storage capacity, query latency, or other indicators that reflect the load on the adjacent segments. When the pre-defined secondary splitting conditions are met, the number of sub-adjacent segments to be formed is determined based on the data size of the current adjacent segments and the target data size of a single sub-adjacent segment, and the number of sub-adjacent segments is limited to not exceeding the pre-defined maximum splitting number.

[0212] For example, a secondary split can be performed using a target node identifier hash. Specifically, each adjacent edge in the adjacent segment to be split can be read, and the target node identifier corresponding to each adjacent edge can be obtained; then, a hash operation can be performed on the target node identifier, and different adjacent edges can be divided into corresponding sub-adjacent segments according to the hash result.

[0213] For example, based on a predetermined number of sub-adjacent segments, the hash result corresponding to the target node identifier can be moduloed according to the number of sub-adjacent segments to obtain the corresponding sub-adjacent segment number, and the adjacent edge can be written into the new adjacent segment corresponding to the sub-adjacent segment number. In this way, adjacent edges that were originally concentrated in the same adjacent segment can be distributed to multiple sub-adjacent segments.

[0214] At the physical storage layer, corresponding edge data buckets, data segments, or key-value storage areas can be established for each new sub-adjacent segment. Based on the mapping relationship between each sub-adjacent segment and the preset physical partition group, different sub-adjacent segments are allocated to different physical partitions. When there is a partition with high load in the preset physical partition, other physical partitions with lower load in the same partition group can be selected as the actual storage location for the corresponding sub-adjacent segment.

[0215] After creating multiple new sub-adjacent segments, update the virtual adjacency table directory. Specifically, set the directory entries corresponding to the original adjacency segments to a redirected state, and create new directory entries for each new sub-adjacent segment. The new directory entries can record the parent segment identifier of the original adjacency segment, the routing rules corresponding to this split, the specific route values, the physical storage location of the new sub-adjacent segment, version information, and consistency status.

[0216] Furthermore, during the transition period when the adjacency segment splitting and data migration are not yet fully completed, for newly written adjacency edges, the corresponding new sub-adjacency segments can be directly written according to the hash result corresponding to the target node identifier, so as to avoid continuing to write new data to the original adjacency segments that are about to expire.

[0217] For query requests, before all historical adjacent edges in the original adjacent segment have been migrated to the new sub-adjacent segment, the original adjacent segment and the new sub-adjacent segment that has completed data migration can be accessed simultaneously according to the migration status in the virtual adjacent table directory. The data read from different segments can be deduplicated according to the adjacent edge identifier to avoid the same adjacent edge being returned repeatedly due to its dual existence during the migration process.

[0218] Once the historical data migration from the original adjacent segments is complete, and the incremental data generated during the migration process has been compensated, the directory status corresponding to the original adjacent segments can be updated to unavailable or archived. Afterward, subsequent queries will only access the newly formed sub-adjacent segments through the virtual adjacency table directory.

[0219] Through the aforementioned mechanisms of writing, deleting, updating attributes, locating target adjacent segments, creating new adjacent segments, and secondary splitting, the correspondence between the virtual adjacency table and each adjacent segment can be dynamically maintained even when the adjacency data of the target node continues to change. This ensures that the semantic information, physical storage location, and segment status recorded in the directory remain consistent with the actual adjacency data organization, thereby enabling the aforementioned adjacency query mechanism based on the virtual adjacency table directory to operate continuously for a long time.

[0220] Example 7

[0221] During the processes of writing, deleting, updating attributes, and secondary splitting of adjacent edges, the virtual adjacency table directory and adjacent segments can be maintained according to each data change. However, as the scale of graph data continues to grow and query access patterns change, the granularity of the original adjacent segment partitioning, physical storage location, or access priority may gradually become mismatched with current query requirements. For example, adjacent segments that were originally accessed infrequently may become accessed infrequently, time windows that were originally suitable for merging and storage may need to be further subdivided due to concentrated access, or continuing to maintain multiple small, long-term infrequent adjacent segments independently will increase the overhead of directory management and query merging. Therefore, in one implementation, during the continuous operation of the virtual adjacency table, the operating status of each adjacent segment can be evaluated periodically or when preset event triggering conditions are met, and the partitioning granularity, physical storage location, or query processing strategy of the adjacent segments can be dynamically adjusted based on the evaluation results.

[0222] In one implementation, after the virtual adjacency table directory is established and adjacency data queries are performed based on the virtual adjacency table directory, the actual operating status of the virtual adjacency table can be continuously evaluated, and the splitting granularity, physical storage location, or query processing strategy of adjacency segments can be dynamically adjusted according to the evaluation results.

[0223] Dynamic adjustments can be made using either periodic triggering or event triggering. Periodic triggering involves collecting the operational status information of each adjacent segment at a preset time period and determining whether the current organization of adjacent segments is still suitable for the current access mode based on the operational status information. Event triggering involves immediately performing adjustments on the corresponding adjacent segment when a certain operational indicator meets a preset trigger condition.

[0224] Specifically, one or more operational metrics can be obtained, including the number of edges corresponding to adjacent segments, query latency, access frequency, access ratio corresponding to different time windows, access scenario hit rate, duration of low frequency of segments, query result merging cost, and physical sharding load status. Based on the above operational status information, it can be determined whether the preset adjustment conditions are met.

[0225] For example, when the number of adjacent edges in a certain adjacent segment exceeds a preset threshold, the data granularity of the current adjacent segment can be considered too large; when the response latency of a query for a certain adjacent segment exceeds a preset query latency threshold, the read overhead of the current segment can be considered high; when the access frequency of a certain adjacent segment continues to increase, the adjacent segment can be considered to be gradually becoming a high-frequency access segment; when the query hit ratio corresponding to different time windows changes significantly, the matching degree between the original time window division method and the current access mode can be considered to have decreased; when the segment hit frequency corresponding to a certain access scenario continues to increase, the access scenario can be considered to have a strong independent access demand.

[0226] Furthermore, when a certain adjacent segment remains in a low-frequency access state for an extended period, it can be determined that there is a possibility of further merging between this adjacent segment and other low-frequency segments. When a query requires simultaneous access to multiple adjacent segments, and the computational or storage read overhead generated by the corresponding result merging processing exceeds a preset threshold, it can be considered that the current segmentation is too granular. When there are sustained and significant differences in access load, storage capacity, or processing resource usage between different physical shards, it can be considered that the current physical layout exhibits load skew. Meeting any of the above preset conditions can trigger the dynamic adjustment process corresponding to the virtual adjacency table.

[0227] In one implementation, dynamic adjustment may include one or more of the following: refined splitting, fragment merging, fragment migration, and query processing strategy adjustment.

[0228] When the current adjacent data range is too large or the access characteristics of different sub-data sets differ significantly, a more refined split can be performed. Specifically, the original large time window can be further divided into multiple smaller time windows, so that high-frequency time ranges and low-frequency time ranges form independent adjacent segments respectively; the adjacent segments corresponding to a certain relation type can also be further divided according to the group to which the target node belongs, the attribute range of the target node, or other features that can reflect the locality of query access; when the adjacent edge corresponding to a certain access scenario is frequently accessed, the data corresponding to that access scenario can also be further organized from the original adjacent segments to form independent scenario adjacent segments.

[0229] For example, when the data in the most recent period is frequently accessed in the adjacent segments corresponding to a certain time window, while the data in the earlier time range is accessed much less frequently, the adjacent segment can be re-divided into multiple sub-adjacent segments according to time, and the recent data and historical data can be assigned to different adjacent segments respectively, so that subsequent queries for recent data can only access the corresponding high-frequency sub-adjacent segments.

[0230] In another implementation, when the data size of multiple adjacent segments is small and the long-term access frequency is low, segment merging can be performed. Specifically, multiple small adjacent segments with low long-term frequency can be merged into a larger adjacent segment; for cold adjacent segments corresponding to adjacent historical time ranges, adjacent time windows can be re-merged; for multiple scenario adjacent segments corresponding to different access scenarios but with similar actual access patterns, they can also be reorganized into shared adjacent segments according to their access characteristics.

[0231] For example, when multiple consecutive historical time windows are in a low-frequency access state for a long time, the cold adjacency segments corresponding to the above multiple time windows can be merged into a historical adjacency segment with a larger time range. This reduces the number of adjacency segments that need to be maintained and the number of directory entries in the virtual adjacency table directory, thereby reducing the overhead of directory maintenance and query merging.

[0232] Furthermore, when the semantic division of adjacent segments remains reasonable, but their current physical storage location is no longer suitable for the actual access state, segment migration can be performed. For example, when the access frequency of the original ordinary segment continues to increase and it becomes a high-frequency segment, it can be migrated to a physical storage area with higher access performance; when the original high-frequency segment is no longer frequently accessed for a long time, it can be migrated to a storage area suitable for low-frequency data; when a physical partition is overloaded, one or more adjacent segments can be migrated to other physical partitions with lower load.

[0233] After splitting, merging, or migrating adjacency segments, the corresponding virtual adjacency table directory is updated. For example, when an original adjacency segment is split into multiple new adjacency segments, a new directory entry can be created for each new adjacency segment, and the directory status of the original adjacency segment can be updated; when multiple adjacency segments are merged, a new directory entry can be created for the merged adjacency segment, and the original directory entry can be set to an invalid or redirected state; when an adjacency segment is migrated, the physical storage location and version status in the corresponding directory entry can be updated to ensure that subsequent queries can still locate the currently valid data location through the virtual adjacency table directory.

[0234] In another implementation, the query result processing strategy can be adjusted based on the running status without changing the data content of adjacent segments. For example, the segment access priority can be adjusted based on the recent query hit rate of different adjacent segments, so that adjacent segments that are more likely to be hit by the query are accessed first; when multiple segment switching occurs frequently during cross-segment pagination, the maintenance method of pagination status can be adjusted; when only a limited number of query results need to be returned, the early termination condition can be adjusted based on the historical query hit rate, so that access to low-priority segments stops after the query reaches the required number of results; when multi-hop path queries or relationship diffusion queries generate a large amount of adjacent edge scanning, the scanning constraints during the query process can also be adjusted accordingly to control the data expansion scale of a single query.

[0235] For example, when a high-frequency adjacent segment can consistently provide most of the top-ranked query results, the access priority of that adjacent segment can be increased, and the query termination condition can be appropriately advanced. Conversely, when some low-frequency adjacent segments rarely contribute to the final query results over a long period, their access priority can be reduced, so that they are only accessed when high-priority segments cannot meet the query requirements.

[0236] Through the above dynamic adjustment mechanism, the adjacency segment partitioning method, physical storage layout, and query processing strategy of the virtual adjacency table can be adaptively adjusted as the actual graph data scale and access mode change. This avoids the fixed partitioning strategy from gradually becoming ineffective due to changes in data scale, query hotspots, or business scenarios during long-term operation, thereby enabling adjacency queries for ultra-large degree nodes to maintain low data read overhead and relatively stable query performance over a long period of time.

[0237] After completing the above dynamic adjustments, the semantic attributes, physical storage location, version status, and access control information corresponding to the adjusted adjacency segments in the virtual adjacency table directory are updated synchronously, so that subsequent query requests can still determine the target adjacency segment and generate the corresponding access plan based on the updated virtual adjacency table directory.

[0238] Example 8

[0239] In one implementation, this application is illustrated using a query for hyperscale nodes in a social graph database as an example.

[0240] A target node U001 exists in the social graph database, corresponding to a celebrity user, which has a large number of adjacency edges. The adjacency edges are shown in the table below: Table 1 Adjacency Relationship Table for U001

[0241] As shown in the table above, the number of adjacent edges and actual access characteristics differ for different relationship types. For example, the number of adjacent edges corresponding to the "follow" relationship is relatively large, and it is mainly used for querying the fan list and fan profile; the adjacent edges corresponding to the "interaction" relationship are mainly used for querying recent interactions and recommendation recall; the adjacent edges corresponding to the "mention" relationship are mainly used for querying the spread of trending events; and the adjacent edges corresponding to the "share" relationship are mainly used for querying the content dissemination path.

[0242] When using the traditional adjacency list storage method, all adjacent edges corresponding to the target node U001 are usually organized in the same adjacency list or the same logical adjacency relationship set. When the query request is only used to obtain the interaction relationships of the target node U001 within the most recent preset time range, it may still be necessary to read or scan a large number of follow relationships, mention relationships, sharing relationships, and interaction relationships from earlier time ranges that are not related to the query, thus generating a large amount of invalid data reads.

[0243] To reduce the aforementioned invalid data reading, in this embodiment, a corresponding virtual adjacency table is established for the target node U001. Logically, the virtual adjacency table represents the complete adjacency relationships corresponding to the target node U001, and semantically divides the adjacency edges corresponding to the target node U001 according to one or more of the following: relationship type, time window, and access scenario, to form multiple adjacency segments.

[0244] For example, as shown below, target node U001 corresponds to a virtual adjacency table VAD_U001, and the virtual adjacency table directory registers directory entries corresponding to multiple adjacency segments. Each directory entry records at least the virtual adjacency table identifier, adjacency segment identifier, relationship type, time window, access scenario, and physical storage location. Therefore, when a query request specifies the interaction relationship and the time condition of the last 7 days, the directory entries corresponding to F002, F003, and F004 can be filtered according to the virtual adjacency table directory, and the corresponding adjacency segments can be accessed according to the physical storage location recorded in each directory entry, without having to access the other adjacency segments corresponding to F001, F005 to F008.

[0245] Table 2. Directory of Virtual Adjacency Lists for Node U001 in the Social Database

[0246] For example, when a query request is used to obtain the interactive users of the target node U001 within the most recent preset time range and return a preset number of query results in descending order of interaction scores, the system first queries the corresponding virtual adjacency table directory based on the target node U001, and then filters out the adjacency segments corresponding to this query based on the interaction relationship type and time conditions, such as F002, F003 and F004.

[0247] Subsequently, an access plan is generated based on the target adjacent segments, and the access order of each target adjacent segment is determined in conjunction with sorting requirements. During the query process, the target adjacent segments are read sequentially according to the access plan, and the query results of each segment are merged according to priority. When the number of query results obtained reaches a preset number, access to subsequent low-priority adjacent segments is stopped, and interactive users who meet the query conditions are returned.

[0248] This avoids accessing follow relationships, mention relationships, share relationships, and historical interaction adjacency fragments outside the query time range, thereby reducing the scanning of invalid adjacency edges.

[0249] Example 9

[0250] The following table illustrates how to query gateway nodes in the device graph database: Table 3 Device Network Relationship Table

[0251] As shown in the table above, target node G001 is a regional gateway node and has established connections with a large number of device nodes. The adjacent edges corresponding to target node G001 can include device connection relationships, data reporting relationships, alarm association relationships, and regional affiliation relationships. Different relationship types correspond to different device operation and maintenance query needs, such as querying currently online devices, abnormal communication devices within the most recent preset time range, devices connected to a specified region, and alarm association paths.

[0252] For the target node G001, a corresponding virtual adjacency table is established for it, and the adjacency edges of the target node G001 are semantically divided according to one or more of the relationship type, time window, device status and access scenario to form multiple adjacency segments.

[0253] For example, the following adjacency segments can be formed: the first adjacency segment F101 corresponds to device connection relationships, online device status, and device topology query scenarios; the second adjacency segment F102 corresponds to device connection relationships, offline device status, and device topology query scenarios; the third adjacency segment F103 corresponds to data reporting relationships, the first time window, and data reporting scenarios; the fourth adjacency segment F104 corresponds to alarm association relationships, the second time window, and alarm tracking scenarios; and the fifth adjacency segment F105 corresponds to region affiliation relationships and region query scenarios.

[0254] After forming the above-mentioned adjacency segments, establish the mapping relationship between the virtual adjacency table corresponding to the target node G001 and each adjacency segment, and register the relationship type, time window, device status, access scenario and physical storage location corresponding to each adjacency segment into the virtual adjacency table directory.

[0255] For example, the virtual adjacency list directory corresponding to the target node G001 can be shown in the following table: Table 4. Virtual adjacency table directory for gateway node G001 in the backup database.

[0256] For example, when a query request is used to obtain devices that have abnormal communication with the target node G001 in the past hour, the corresponding virtual adjacency table directory can be determined based on the target node G001, and the directory items can be filtered according to the relationship type, time conditions and access scenarios to determine the target adjacency segments corresponding to the data reporting relationship and alarm association relationship, such as F103 and F104.

[0257] Subsequently, an access plan is generated based on the target adjacency fragment, and the physical storage locations corresponding to F103 and F104 are accessed according to the access plan to obtain the device query results that meet the query conditions, without having to scan all device connection edges and region belonging edges corresponding to the target node G001, thereby reducing the reading of invalid adjacency data.

[0258] This application also provides a graph data query system for ultra-large degree nodes, such as... Figure 2 As shown, it includes: Parsing module 201 is used to parse the query request and obtain the target node and the corresponding query conditions; the target node is a super-large degree node. The query module 202 is used to query the virtual adjacency table directory corresponding to the target node based on the target node and the query conditions, so as to determine the target adjacency segment that meets the query conditions; wherein, the virtual adjacency table directory is used to indicate the mapping relationship between the target adjacency segment and the corresponding physical storage location; The acquisition module 203 is used to obtain the adjacency query results of the target node based on the target adjacency fragment.

[0259] As the system implementation is basically similar to the method implementation, it is described in a relatively simple way. For relevant details, please refer to the description of the method implementation.

[0260] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0261] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0262] This application is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0263] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0264] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0265] It should be noted that: The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of this application. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0266] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0267] Furthermore, it should be noted that the shapes and names of the parts and components described in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this application are included within the scope of protection of this application. Those skilled in the art to which this application pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, as long as they do not deviate from the structure of this application or exceed the scope defined by the claims, all of which should fall within the scope of protection of this application.

Claims

1. A graph data query method for ultra-large degree nodes, characterized in that, include: Parse the query request to obtain the target node and the corresponding query conditions; The target node is an ultra-large node; wherein, the ultra-large node is a node whose node feature information meets the preset activation conditions, and the node feature information is used to characterize at least one of the target node's adjacent data scale, adjacent access load, data growth status and query performance. Based on the target node and the query conditions, the virtual adjacency table directory corresponding to the target node is queried to determine the target adjacency segment that meets the query conditions; wherein, the virtual adjacency table directory is used to indicate the mapping relationship between the target adjacency segment and the corresponding physical storage location; The adjacency query result of the target node is obtained based on the target adjacency segment.

2. The graph data query method for ultra-large degree nodes according to claim 1, characterized in that, The establishment of the virtual adjacency table directory includes: The adjacent edges of the target node are semantically segmented to form multiple adjacent segments; Determine the physical storage locations corresponding to the plurality of adjacent segments; Establish a mapping relationship between the segment identifiers of each adjacent segment and their corresponding physical storage locations; The virtual adjacency table directory is established based on the mapping relationship and the physical storage locations corresponding to the multiple adjacent segments.

3. The graph data query method for ultra-large degree nodes according to claim 2, characterized in that, The semantic partitioning includes dividing the adjacent edges according to at least one of relation type, time window, and access scenario to form multiple adjacent segments, each with corresponding semantic attributes.

4. A graph data query method for ultra-large degree nodes according to claim 2, characterized in that, Determining the physical storage locations corresponding to multiple adjacent segments includes: The underlying storage layer corresponding to adjacent segments is determined based on the access frequency and time freshness of adjacent segments; The scene overlay layer corresponding to the adjacent segment is determined based on the adjacent segment access scenario.

5. A graph data query method for ultra-large degree nodes according to claim 1, characterized in that, The process of determining the target adjacent segments that meet the query conditions includes: The query conditions are matched with the semantic attributes corresponding to each adjacency segment in the virtual adjacency table directory to obtain the matching results; Based on the matching results, the target adjacent segment is determined from multiple adjacent segments corresponding to the virtual adjacent table.

6. The graph data query method for ultra-large degree nodes according to claim 1, characterized in that, An access plan is generated based on the target adjacency segment, and the adjacency query result of the target node is obtained by accessing the target adjacency segment based on the access plan. The process of generating an access plan includes the following steps: Obtain the fragment attributes corresponding to the target adjacent fragment and the execution query conditions corresponding to the query request; Based on the fragment attributes and the execution query conditions, the access order and access control information of the target adjacent fragments are determined to generate the access plan; The fragment attributes include query condition matching degree, fragment priority, fragment popularity, expected scan cost, and consistency status. The access control information includes sorting strategy, pagination strategy, maximum number of adjacent edges allowed for scanning, early termination conditions, and rollback strategy.

7. A graph data query method for ultra-large degree nodes according to claim 1, characterized in that, When the target adjacent segment includes multiple adjacent segments, the segment query results corresponding to the multiple target adjacent segments are strategically merged to obtain the adjacent query result; The strategic merging includes full merging, conditional merging, priority merging, pagination merging, deduplication merging, rate limiting merging, and consistency merging.

8. A graph data query method for ultra-large degree nodes according to claim 2, characterized in that, Before establishing the virtual adjacency list directory, the following is also included: Obtain the node feature information corresponding to the node, wherein the node feature information includes node degree, adjacency query frequency, edge growth rate and query latency; Determine whether the node meets the preset activation conditions based on the node feature information; When the node meets the preset activation conditions, the node is identified as an ultra-large node, and a corresponding virtual adjacency table directory is established for the ultra-large node.

9. A graph data query method for ultra-large degree nodes according to claim 2, characterized in that, Also includes: Obtain the running status information corresponding to the adjacent segments; When the running status information meets the preset adjustment conditions, the adjacent segments are subjected to refined splitting, segment merging, segment migration, and query processing strategy adjustment. The virtual adjacency table directory is updated based on the adjusted adjacency segments.

10. A graph data query system for ultra-large degree nodes, characterized in that, include: The parsing module is used to parse query requests and obtain the target node and the corresponding query conditions; The target node is an ultra-large node; wherein, the ultra-large node is a node whose node feature information meets the preset activation conditions, and the node feature information is used to characterize at least one of the target node's adjacent data scale, adjacent access load, data growth status and query performance. The query module is used to query the virtual adjacency table directory corresponding to the target node based on the target node and the query conditions, so as to determine the target adjacency segment that meets the query conditions; wherein, the virtual adjacency table directory is used to indicate the mapping relationship between the target adjacency segment and the corresponding physical storage location; The acquisition module is used to obtain the adjacency query result of the target node based on the target adjacency fragment.

Images