Edge relationship temperature-aware hierarchical storage method and query method

Abstract

The application discloses an edge relationship temperature-aware layered storage method and a query method, wherein the storage method is based on the current temperature level of each edge shard and performs differentiated storage according to a preset storage rule, and comprises the following steps: based on edge relationship access behavior data, temperature evaluation is performed on each edge shard to obtain a shard overall temperature score corresponding to each edge shard, and a target temperature level corresponding to the edge shard is determined based on the shard overall temperature score; based on the change trend of the shard overall temperature of the edge shard and the target temperature level, the edge shard is migrated; and for each edge shard, temperature evaluation is also performed on edges marked as candidate key edges in the edge shard to obtain an edge temperature score of each candidate key edge, and a key edge and an accurate level of the key edge are determined based on the edge temperature score. Through the design of the shard-level rough score and the edge-level accurate score, the temperature metadata size is controlled, and the ability to identify key edges is retained.

Description

Technical Field

[0001] This invention relates to the field of graph database storage management technology, and in particular to a method for temperature-sensing hierarchical storage and querying of edge relationships for mixed hot and cold graph data. Background Technology

[0002] Graph databases typically employ a "vertex-edge-attribute" data model to represent complex relationship networks, and are widely used in knowledge graphs, financial risk control, social networks, recommendation systems, IoT relationship analysis, permission relationship analysis, supply chain tracking, and other scenarios. In these scenarios, access to graph data is usually not uniformly distributed, but rather exhibits a clear mix of hot and cold edges. For example, some high-value relationship edges frequently appear in real-time queries, path expansion, adjacency traversal, and multi-hop analysis; while some edges are only accessed during periodic statistics, offline analysis, or historical backtracking.

[0003] Existing graph database storage solutions typically focus on optimizations related to node indexes, edge indexes, adjacency list structures, attribute compression, and cache eviction. Some general-purpose databases or big data systems have adopted a hot / cold data tiering approach, classifying data based on access time, access frequency, or data lifecycle, storing hot data on high-speed media and cold data on low-cost media. However, these solutions often manage data at the file, table, partition, time series, or data block level, without fully considering the structural value of edge relationships in the query path within the graph database. Publicly available technologies include time-series data hot / cold tiered storage, hierarchical storage for hot / cold data, and HDFS automatic hot / cold tiering solutions, the core of which is generally based on hot / cold data determination and migration. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies where the granularity of hot and cold data partitioning is too coarse, by providing a method for edge relationship temperature-sensing hierarchical storage and querying of mixed hot and cold graph data.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, this invention proposes a layered storage method for edge-relationship temperature sensing, which performs differentiated storage based on the current temperature level of each edge segment according to preset storage rules, including the following steps: Collect edge relationship access behavior data corresponding to the graph database; Based on edge relationship access behavior data, temperature assessment is performed on each edge shard to obtain the overall temperature score of the shard corresponding to each edge shard, and the target temperature level corresponding to the edge shard is determined based on the overall temperature score of the shard. Based on the overall temperature change trend of the edge segment, it is determined whether to perform segment temperature increase. When it is determined that segment temperature increase is to be performed, the edge segment is migrated according to the target temperature level, and the target temperature level is used as the current temperature level of the corresponding edge segment. After obtaining the overall temperature score of each edge segment, the temperature of the edges marked as candidate key edges within each edge segment is evaluated to obtain the edge temperature score of each candidate key edge. Based on the edge temperature score, the key edges and their precision level are determined. The candidate key edges include edges whose edge relationship type is a preset key relationship.

[0006] As one possible implementation method: Based on edge relationship access behavior data, calculate the shard-level access frequency score, shard relationship composition score, shard-level path hit rate score, and shard-level structure contribution score corresponding to the edge shard, and calculate the overall shard temperature score of the edge shard. in: The corresponding sharding relationship composition score is calculated based on the relation type weight corresponding to the edge relation type, and the proportion of the number of edges of various relation types or the weighted proportion of path hits within the edge shard. The corresponding shard-level structural contribution score is calculated based on the shard cross-partition contribution, shard bridging contribution, and shard adjacency extension contribution corresponding to the edge shard. The shard cross-partition contribution represents the proportion of cross-partition edges in the edge shard. The shard bridging contribution represents the proportion of the number of bridging edges within the edge shard or the proportion of the path hit weight value corresponding to the bridging edge. The shard adjacency extension contribution represents the adjacency extension contribution rate of the edge shard.

[0007] As one possible implementation method: Within the statistics window, the average contribution score corresponding to each edge relationship type is calculated and statistically analyzed based on the query result paths returned by the path-type query. The relation type weights corresponding to each edge relation type are updated based on the obtained average contribution score.

[0008] As one possible implementation method: The total number of times the corresponding edge within the edge segment is scanned as a candidate edge, and the total number of times it is actually retained and enters the next hop candidate path, are used to calculate the corresponding segment adjacency expansion contribution.

[0009] As one possible implementation method: Calculate the sharding cache benefit score for edge sharding; After normalizing the shard-level access frequency score, shard relationship composition score, shard-level path hit rate score, shard-level structure contribution score, and shard cache benefit score corresponding to the edge shard, the corresponding overall shard temperature score is calculated.

[0010] As one possible implementation method: According to the preset drill-down rules, the edge is marked with key identifiers based on the path hit, bridging and adjacency expansion of the edge piece, and the corresponding edge is marked with key identifiers based on the preset key relationship. All edges with key identifiers are selected as candidate key edges. The candidate key edges also include: Bridging edges connect different communities, different data shards, or different business entity clusters. Edges that are hit in a concentrated area are those whose path hit counts exceed the concentration threshold as a percentage of the total path hit counts in the corresponding edge partition. Key extension edges are those edges that have entered the next hop candidate path a preset threshold number of times, indicating that their extension contribution is significant. Edges with abnormal costs: Edges where the cost of reading and decompressing the cold layer exceeds a preset cost threshold.

[0011] As one possible implementation method: The specific steps for performing temperature evaluation on candidate critical edges to obtain the edge temperature score of the candidate critical edges are as follows: Based on edge relationship access behavior data, edge-level access frequency score, edge-level path hit rate score, and edge-level structure contribution score of candidate key edges are calculated. Among them, the edge-level structure contribution score is calculated based on the cross-partition situation, bridging situation, and adjacency expansion contribution rate of the candidate key edges. Based on the edge-level access frequency score, the edge-level path hit rate score, the edge-level structure contribution score, and the relation type weight corresponding to the candidate key edge, the corresponding edge temperature score is calculated.

[0012] As one possible implementation method: It also includes the identification and recycling steps: When the edge temperature score is less than the preset retention threshold and reaches the preset maximum number of times, check whether the corresponding candidate key edge meets the preset key judgment conditions. If the requirements are not met, the corresponding key identifier will be reclaimed. If any key determination condition is met, retain the key identifier, treat it as a key edge, and construct or update the edge temperature profile.

[0013] As one possible implementation, a local migration step is also included; When the precision level corresponding to a critical edge is inconsistent with the current temperature level of the edge segment it belongs to, the candidate critical edge is added to the migration set corresponding to the precision level. Based on preset local migration rules, the corresponding fragments to be migrated are obtained by recombining each key edge in the set to be migrated, and the fragments to be migrated are migrated according to their corresponding precision levels.

[0014] Secondly, this invention also proposes a layered query method for edge relationship temperature perception, comprising the following steps: Determine candidate expansion directions based on the target query statement; Determine the expansion cost corresponding to each candidate expansion direction, and determine the target expansion direction based on the expansion cost; Based on the target expansion direction, differentiated reading is performed according to the current temperature level of the corresponding edge piece to obtain the corresponding query results; The method for calculating the expansion cost corresponding to the current candidate expansion direction is as follows: Determine the number of candidate edges corresponding to the current candidate expansion direction; Based on the current temperature level of the edge fragment and the precision level corresponding to the key edge, determine the hot edge ratio, warm edge ratio and cold edge ratio corresponding to the current candidate direction, and calculate the average hierarchical access cost corresponding to the current candidate direction based on the obtained hot edge ratio, warm edge ratio and cold edge ratio. The expansion cost of the current expansion direction is determined based on the number of candidate edges and the average hierarchical access cost.

[0015] This invention, by adopting the above technical solutions, has significant technical effects: This invention controls the size of metadata by performing an overall evaluation of edge sharding, while accurately evaluating and recording key edges to compensate for the dilution of high-value edges by the average value of sharding. Combined with a differentiated storage strategy, it enables hot and cold data to adaptively match the storage level, taking into account system overhead and management performance. It is suitable for graph database scenarios where there are significant differences between hot and cold access and the main query form is relation edge traversal. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a layered storage method for edge-relationship temperature sensing according to the present invention. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0019] Existing hot and cold data stratification schemes typically manage files, tables, partitions, data blocks, or general cache items. This choice of larger granularity is not accidental, but rather constrained by the scale of metadata, migration costs, and consistency maintenance overhead. In graph databases, the number of edges is usually far greater than the number of nodes. Directly adopting a general data temperature model and creating separate hot and cold labels, location mappings, and migration records for each edge can easily lead to an overabundance of temperature metadata, thus negating the benefits of hot and cold stratification.

[0020] Edges in graph databases are typically organized continuously in the form of adjacency lists, edge blocks, or compressed edge segments. Changes in the temperature of a single edge can affect the starting adjacency offset, relation type indexes, reverse indexes, compressed block boundaries, and cache directories. Existing general-purpose layering techniques, even if they can calculate data temperature, lack an execution mechanism for fine-grained layering of edge relationships without disrupting the continuity of adjacency lists and without frequently rewriting compressed blocks.

[0021] Graph querying is path-dependent; the value of an edge is determined not only by the number of visits but also by whether it is located in a preceding hop of a multi-hop path, a pruned position, a bridging community position, or a business-critical path. Existing technologies for data temperature, data value, or cache weight primarily focus on the data itself or the cached item itself, making it difficult to express the structural contribution of edges in path expansion, and thus failing to support precise hot / cold edge identification at the edge granularity.

[0022] In concurrent graph query scenarios, edge migration cannot simply be a physical relocation; it must also ensure consistent results when the query executor reads from the old location, reads from the new location, and when the cache is hit. Existing hot and cold tiering techniques typically only handle general data block migrations and do not address the consistency and coordination issues between edge location mapping, edge versions, adjacency indexes, and path fragment caches.

[0023] To address the shortcomings of existing technologies, this application proposes a temperature-aware hierarchical storage method for edge relationships in mixed hot and cold graph data. This method uses edge relationships in the graph database as the basic object for temperature identification and hierarchical storage, maintaining an edge temperature profile for each edge or edge fragment. The edge temperature profile includes at least the following indicators: edge access frequency, relationship type weight, path hit rate, recent access decay factor, neighborhood expansion contribution, and business priority. Based on the edge temperature score, the system dynamically classifies edges into hot edges, warm edges, and cold edges, and employs low decoding overhead storage, high compression ratio storage, or balanced storage respectively.

[0024] like Figure 1 As shown, it includes the following steps: S100, Collect edge relationship access behavior data corresponding to the graph database; Based on the edge relationship access behavior during the operation of the graph database, obtain the corresponding edge relationship access behavior data. In this embodiment, the graph database collects corresponding edge relationship access behavior data when performing tasks such as point query, adjacency query, path query, pattern matching query, subgraph query, batch import, and offline analysis. The edge relationship access behavior data includes: edge identifier edgeId, edge start identifier srcVertexId, edge end identifier dstVertexId, edge relationship type edgeType, edge attribute set edgeProperties, edge last access time lastAccessTime, edge access count in the statistics window accessCount, edge hit count in path query pathHitCount, edge query type, edge adjacency table or edge fragment identifier fragmentId, edge fragment access density and candidate scan count within the fragment, whether the edge belongs to the key relationship type marked by the business, read time, decompression time, cache hit status and query response time when accessing the edge or edge fragment; The query types involved by the edges include: single-hop adjacency, multi-hop path, pattern matching, shortest path, recommendation expansion, etc.

[0025] In practical applications, to reduce data collection overhead, those skilled in the art can use sampling and statistical methods to record access behavior based on actual needs. For example, precise counting can be used for high-frequency edges, while approximate counting can be used for low-frequency edges; path hit rate can be collected only in path queries, pattern matching queries, and multi-hop traversal queries, where high-frequency and low-frequency edges can be automatically distinguished based on the frequency judgment threshold configured by the user.

[0026] S200. Based on edge relationship access behavior data, perform temperature evaluation on each edge shard to obtain the overall temperature score of the shard corresponding to each edge shard, and determine the target temperature level corresponding to the edge shard based on the overall temperature score of the shard. Specifically, the following steps are included: S210. Calculate the shard-level access frequency score, shard relationship composition score, shard-level path hit rate score, and shard-level structure contribution score corresponding to each shard. Specifically, the following steps are included: S211. Based on the edge relationship access behavior data, calculate the shard-level access frequency score, i.e., the access frequency score mentioned above. For edge shard B, the number of visits A(B,W) of all edges in edge shard B is counted within the statistics window W, and the corresponding shard-level access frequency score F(B) is calculated by combining the most recent access time of edge shard B. The specific calculation method is as follows: F(B)=Norm(A(B,W))×D B (t); Where Norm represents normalizing the count to a preset interval, D B (t) represents the recent access time decay score of the edge partition; The most recent visit time decay score, i.e., D B The specific calculation method for (t) is as follows: D B (t)=exp(-λ×Δt B ); Where exp represents the natural exponential function, Δt B λ is the time difference between the current time and the most recent access time of edge partition B, and λ is the time decay coefficient.

[0027] S212. Based on the edge relationship access behavior data, calculate the sharding relationship composition score corresponding to each edge shard; For edge shard B, the system calculates the sharding relation composition score R(B) based on the proportion of edges of various relation types within the edge shard or the weighted proportion of path hits. The specific calculation method is as follows: ; Where, p B (type) represents the proportion of edge relation type type in shard B, and R(type) represents the relation type weight corresponding to edge relation type type.

[0028] As one possible implementation, the relation type weight is a pre-configured static parameter, that is, the relation type weight R(type) is pre-set for different edge relation types involved in the graph database; The relation type weight R(type) can be manually set by business rules and query logs or generated by an adaptive learning mechanism. This specification does not impose detailed limitations on it.

[0029] In this embodiment, the initial value is used to express business priors. For example, in financial risk control, relationships such as transfers, guarantees, and actual controllers are naturally more likely to affect the risk path than relationships such as browsing and registration addresses. The adaptive adjustment is used to correct the differences in query preferences under different business stages or different tenants. For example, in a financial risk control graph database: transfer relationship 1.00, guarantee relationship 0.95, actual controller relationship 0.90, employment relationship 0.75, registered address relationship 0.45, browsing relationship 0.20; in a knowledge graph database: entity ownership relationship 0.95, dependency relationship 0.90, hierarchical relationship 0.85, alias relationship 0.60, description relationship 0.30.

[0030] As another possible implementation, the pre-configured relation type weights are set as initial values. In actual applications, the relation type weights corresponding to each relation type are dynamically updated based on the edge relation access behavior data. In this embodiment, the average contribution score corresponding to each edge relationship type is calculated, and the relationship type weight corresponding to each edge relationship type is updated based on the obtained average contribution score. The specific calculation method is as follows: R new (type)=μR old (type)+(1-μ)Q(type); Among them, R old (type) represents the relation type weight before the edge relation type update, R new (type) represents the updated relation type weight of the corresponding edge relation type, Q(type) represents the average contribution score of this edge relation type in recent queries, and μ is the smoothing coefficient.

[0031] As one possible implementation method, within the statistics window W, the average contribution score Q(type) corresponding to each edge relationship type is calculated and statistically analyzed based on the query result path returned by the path type query. As an example, within the statistics window W, the proportion of the target edge relation type in the query result path of each query is calculated and averaged to obtain the average contribution score Q(type) corresponding to the target edge relation type. Wherein, for a single query q, the proportion Q of the target edge relation type in the corresponding query result path is... q The calculation method for (type) is as follows: ; Where ResultPathEdges(q,type) is the number of edges of type type 'type' in all the result paths returned by query q; AllResultEdges(q) represents the total number of edges in all the result paths returned by query q.

[0032] As another example, within the statistics window W, the proportion of the target relation type in all query result paths is summarized and calculated, generating the average contribution score Q(type) corresponding to the target relation type. The specific calculation method is as follows: ; in: Q W This represents the set of path-type queries within the statistics window W; q represents the path-class query set Q. W The qth query; ResultPathEdges(q,type) represents the number of edges of type type 'type' in all the result paths returned by query q; AllResultEdges(q) represents the total number of edges in all the result paths returned by query q.

[0033] As another possible implementation method, after each query is executed, based on the preset contribution determination rules, the actual contribution of each relation type in this query is extracted from the query execution log and averaged in the corresponding statistics window; Those skilled in the art can set contribution judgment rules according to actual needs; in this embodiment, contribution judgment rules are used to indicate contribution judgment conditions and contribution score calculation rules under each contribution judgment condition. In practical applications, the contribution judgment conditions of each edge relationship type in the corresponding query are determined respectively, and the actual contribution of the corresponding edge relationship type under the current query is calculated and statistically obtained based on the corresponding contribution score calculation rules. In this embodiment, the contribution judgment conditions include: participating in the query result path, significantly reducing the number of candidate paths (the reduction in the number of candidate paths is greater than the preset first judgment threshold), hitting high-priority business rules (manually configured), or reducing the query response time.

[0034] S213. Calculate the fragment-level path hit rate score based on edge relationship access behavior data; As one possible implementation, for edge shard B, the total number of times each edge within the shard is actually hit by path queries is counted within the statistics window W, H(B,W), and the shard-level path hit rate score P(B) is calculated. The specific calculation method is as follows: P(B) = H(B,W) / (Q) path (W)+ε); Where H(B,W) represents the total number of path hits for fragment B within the statistical window W, P(B) represents the fragment path hit rate score, and Q... path (W) represents the total number of path-type queries within the statistics window, and ε is a very small constant to prevent the denominator from being zero; As another possible implementation, the fragment-level path hit rate score P(B) is also obtained based on the normalized calculation of the number of times the edge fragment is scanned, and in this case, it is used to represent the degree to which the entire fragment participates in the effective path.

[0035] In this embodiment, while calculating the path hit rate score at the shard level, the path hit concentration is also calculated. If the proportion of the path hits contributed by the top K edges in a shard to the total path hits in the shard exceeds the concentration threshold, critical edge drill-down is triggered even if P(B) is not high overall, in order to avoid a small number of critical edges being diluted by a large number of ordinary cold edges.

[0036] S214. Calculate the contribution score of the shard-level structure based on edge relationship access behavior data; To avoid misclassifying low-frequency but structurally important bridging edges as cold edges, this application also calculates the contribution score of the segmented structure. The specific calculation method for the contribution score S(B) of the fragmented structure is as follows: S(B)=η1·CrossPartition(B)+η2·BridgeScore(B)+η3·ExpandContribution(B); Where CrossPartition(B) represents the cross-partition contribution; BridgeScore(B) represents the bridging contribution; ExpandContribution(B) represents the adjacent expansion contribution; η1, η2, and η3 are weight parameters.

[0037] CrossPartition(B) represents the proportion of cross-partition edges in shard B; That is, for edge partition B, count the number N of edges whose starting and ending points are located in different graph partitions. cross (B), and calculate its ratio to the total number of edges |B| contained in partition B. The specific calculation method is as follows: CrossPartition(B) = N cross (B) / |B|; In this embodiment, when query logs are available, the path hit count of cross-partition edges can be further weighted, so that cross-partition edges that frequently participate in valid paths can make a higher contribution.

[0038] The BridgeScore(B) represents the percentage of bridged edges within a shard or the percentage of the path hit weights corresponding to the bridged edges. Bridge edges are edges that connect different communities, different data shards, or different business entity clusters, with the starting point and the ending point being an edge. The specific calculation method is as follows: ; Where BridgeScore(e) represents the bridging score of a single side e. When a single side e connects different communities, different data shards, or different business entity clusters, the bridging score is 1; otherwise, it is 0. path (e) represents the path hit weight for edge e; In this embodiment, w path (e) The number of times the edge path is hit can be taken; when accurate logs are missing, it degenerates into the percentage of the number of bridging edges.

[0039] ExpandContribution(B) represents the adjacency expansion contribution rate of edge fragment B; In this embodiment, the corresponding fragment adjacency expansion contribution ExpandContribution(B) is calculated based on the total number of times the edges within the corresponding edge fragment B are scanned as candidate edges by the query executor according to the fragmentId, and the total number of times they are actually retained and enter the next-hop candidate path. The specific calculation method is as follows: ExpandContribution(B)=ExpandHitCount(B,W) / (ExpandCandidateCount(B,W)+ε); Where ExpandCandidateCount(B,W) represents the total number of edge scans of fragment B within the statistical window W; ExpandHitCount(B,W) represents the number of times it actually enters the next hop candidate path; and ε is a minimal constant to prevent the denominator from being zero.

[0040] S220. Calculate the overall temperature score of the corresponding edge shard based on the shard-level access frequency score, shard-level relationship composition score, shard-level path hit rate score, and shard-level structure contribution score, and determine the target temperature level of the corresponding edge shard based on the overall temperature score of the shard. S221, Overall temperature rating of each segment; As one possible implementation, after normalizing the shard-level access frequency score F(B), shard relationship composition score R(B), shard-level path hit rate score P(B), and shard-level structure contribution score S(B) corresponding to edge shard B, the overall temperature score of the shard is calculated and used for continuous management of cold edge shards and judgment of overall temperature rise and fall. As another possible implementation method, a sharding cache benefit score C(B) is introduced. After normalizing the sharding-level access frequency score F(B), sharding relationship composition score R(B), sharding-level path hit rate score P(B), sharding-level structure contribution score S(B) and sharding cache benefit score C(B) corresponding to the edge shard B, the corresponding overall sharding temperature score T(B) is calculated, which is used for continuous management of cold edge shards and overall temperature rise and fall judgment. The fragment caching benefit score C(B) can be calculated based on at least one of the following parameters: fragment access count, read and decompression time when not cached, read time after caching, and fragment memory usage. Those skilled in the art can calculate the fragment caching benefit using existing methods after knowing the above parameters, and this specification does not limit it in detail. The calculation method for the overall temperature score T(B) of each section is as follows: T(B)=α×F(B)+β×R(B)+γ×P(B)+δ×S(B)+θ×C(B); Wherein, α is the access frequency weight coefficient, β is the relation type composition weight coefficient, γ is the path hit weight coefficient, δ is the structure contribution weight coefficient, and θ is the cache benefit weight coefficient. Those skilled in the art can set these values ​​according to actual needs, as long as α+β+γ+δ+θ=1 is satisfied.

[0041] S222, Target Temperature Rating; Temperature ratings include cold, warm, and hot; The specific steps for determining the target temperature level of the corresponding edge segment B based on the overall segment temperature score T(B) are as follows: If T(B)≥TH hot The target level is hot edge fragmentation, TH hot This indicates the temperature threshold for classifying heat levels; If TH warm ≤T(B) <TH hot The target level is warm edge fragmentation, TH warm This represents the temperature threshold used to classify temperature levels. If T(B) <TH warm The target level is then cold edge fragmentation.

[0042] As one possible implementation method, to prevent migration jitter, a separate thermal level temperature rise threshold TH is set. hotup TH (Heat rating) hotdown Temperature rating and temperature rise threshold TH warmup Temperature rating and cooling threshold TH warmdown And the heating threshold of the corresponding temperature level is higher than the corresponding cooling threshold; In practical applications, the target level is adjusted to the higher level only when the overall temperature score of the segment reaches the corresponding higher temperature level and the corresponding temperature rise threshold, and the target level is adjusted to the lower level only when it falls below the corresponding lower temperature level and the corresponding temperature drop threshold. For example, if T(B) rises to TH... hotup If so, the target temperature level is upgraded to hot-edge segmentation. However, T(B) needs to be reduced to TH. hotdown Only then was the target temperature level lowered to the edge temperature segment.

[0043] S223. Update the fragment temperature profile of each fragment; In this embodiment, the fragment temperature profile includes: fragmentId, number of internal edges in the fragment (edgeCount), fragment-level access frequency score, recent access time decay score, fragment relationship composition score, fragment-level path hit score, fragment cross-partition contribution, fragment bridging contribution, fragment adjacency expansion contribution, compression / decompression and read cost, cache benefit score, overall fragment temperature score, current temperature level, target temperature level, temperature rise observation flag, recent migration time, and stable period counter; After calculating the overall temperature score of the segment, the segment temperature profile is updated based on the calculated indicators. This profile represents the temperature of the entire segment and is not the same as the temperature of any single side of the segment.

[0044] S224. Based on the overall temperature change trend of the edge segment, determine whether to perform segment temperature increase. When it is determined that segment temperature increase is to be performed, migrate the edge segment according to the target temperature level and use the target temperature level as the current temperature level of the corresponding edge segment. When the target temperature level remains unchanged for several consecutive times, and the overall temperature of the segment increases for several consecutive times (the target temperature level is higher than the current temperature level) or decreases for several consecutive times (the target temperature level is lower than the current temperature level), the temperature of all edges in the segment is evaluated to determine the precise level corresponding to each edge. Determine whether the proportion of the number of edges or the amount of data in the edge shard that reach the target accuracy level exceeds the overall migration threshold. If it does, migrate the edge shard as a whole. If it does not exceed the threshold, perform a local migration judgment.

[0045] After the overall migration is complete, update the current temperature level of the edge shards.

[0046] S300. For each edge segment, perform temperature evaluation on the edges marked as candidate key edges within the edge segment to obtain the edge temperature score of each candidate key edge, and determine the key edge and the precision level of the key edge based on the edge temperature score. In this embodiment, the temperature level corresponding to the edge segment is the temperature level of the ordinary edge in the edge segment, and the critical edge is based on its precise level.

[0047] S310. According to the preset drill-down rules, the edge is marked with a key identifier based on the path hit, bridging and adjacency expansion of the edge piece, and the corresponding edge is marked with a key identifier based on the preset key relationship. All edges with key identifiers are selected as candidate key edges. That is, candidate key edges include edges that have been marked with key identifiers before and edges that have been marked with key identifiers in this temperature assessment; The candidate key edges mentioned in this embodiment include: Key relationship edges, that is, edges with key relationships manually marked and whose relationship type is a preset key relationship; Bridging edges, such as those bridging different communities, different data shards, or different business entity clusters; Hitting concentrated edges: Edges whose path hit counts exceed the concentration threshold as a percentage of the total path hit counts of the corresponding edge fragment. In this embodiment, when the proportion of path hit counts contributed by the top K edges in a fragment exceeds the first concentration threshold, edges whose proportions exceed the second concentration threshold are selected from the top K edges as candidate key edges. Key extension edges, such as those that enter the next hop candidate path a preset threshold number of times, indicate that their extension contribution is significant. For edge with abnormal cost, such as edge whose corresponding cold layer reading and decompression cost exceeds the preset cost threshold, those skilled in the art can characterize the cost based on time consumption and memory usage, but this specification does not limit it in detail; Those skilled in the art can also set their own key determination conditions for candidate key edges based on actual needs, and this specification does not limit them in detail.

[0048] The drilling rules include triggering rules and judgment rules; The triggering rule is used to indicate the conditions for triggering drilling, and the judgment rule is used to indicate the key judgment conditions for candidate key edges; Those skilled in the art can set one or more triggering conditions for each temperature level based on design needs, and set key judgment conditions for each triggering condition to identify candidate key edge types; For example: For all edge fragments, the edge corresponding to the key relationship discovered during the read, write, and temperature evaluation process is marked with a key identifier; For the cold layer: The overall temperature score of the segment reaches the preset temperature rise threshold, and bridging edges, hit-concentrated edges, critical expansion edges, and cost-abnormal edges are identified. For the thermosphere: When the shard-level path hit rate score, shard-level structure contribution score, or shard cache benefit score is abnormal (e.g., the difference between the average score corresponding to the shard and the temperature layer edge exceeds a threshold), the corresponding candidate key edges are identified. If the shard-level path hit rate score is abnormal, path concentration is judged to identify whether there are concentrated hit edges. If the shard-level structure contribution score is abnormal, bridging edges and / or key extension edges are identified based on the specific contribution score. If the shard cache benefit score is abnormal, edges with abnormal costs are identified. When the overall temperature score of a fragment continuously increases or decreases (each of the three consecutive calculations shows either an increase or decrease), identify and label bridging edges, hit-concentrated edges, critical expansion edges, and cost-abnormal edges. For the thermal layer: When migrating to the hot layer, mark all edges as candidate key edges; During the calculation of the overall temperature score of the fragment, the corresponding bridging edges, hit concentration edges, critical expansion edges, and cost anomaly edges are automatically identified and marked.

[0049] S320, Calculate the edge temperature score of candidate critical edges; S321. Based on edge relationship access behavior data, calculate the edge-level access frequency score, edge-level path hit rate score, and edge-level structure contribution score of candidate key edges. Edge-level access frequency score: For each candidate key edge e, the number of visits A(e,W) of the candidate key edge e is counted separately within the statistics window W, and the visit frequency score of the candidate key edge is calculated. The specific calculation method is as follows: F(e) = Norm(A(e,W)) × D e (t); Where A(e,W) represents the number of times candidate key edge e is accessed within the statistics window W, and D e (t) represents the most recent visit time decay factor of candidate key edge e at time t.

[0050] In this embodiment, the access event of a normal cold edge only updates A(B,W), and there is no need to maintain F(e) for each edge. The higher the number of accesses and the closer the most recent access, the higher the edge access frequency score F(e). When there is no access for a long time, even if the historical number of accesses is high, the recent access time decay score D(t) will reduce its current temperature contribution.

[0051] As one possible implementation, when a shard or candidate key edge is continuously accessed in multiple statistical windows, a sliding window weighting method can be used: the weight of recent windows is higher than that of earlier windows; if only the current window has access, the score of the current window is used; if two or more windows have access or a continuous access trend is detected, multi-window weighting is enabled. Edge path hit rate score P(e): For a candidate key edge e, its path hit rate score P(e) is calculated as follows: P(e) = H(e, W) / (Q) path (W)+ε); Where: H(e,W) represents the number of times edge e is hit in the path query; Q path (W) represents the total number of path-type queries within the statistics window; ε is a very small constant to prevent the denominator from being zero.

[0052] As one possible implementation, it also includes a step of correcting the path hit rate score based on the path hit rate score, combined with the hop count position of edge e in the result path and the frequency of occurrence of its corresponding path pattern, to obtain the corresponding path hit rate correction score P′(e). That is, when the edge is located in the first few hops of a high-frequency query path, the path position weight can be increased. The specific calculation method is as follows: P′(e) = P(e) / (1 + hop(e)); Here, hop(e) represents the hop position of the edge in the path query expansion process. The closer the edge is to the query start point, the greater its impact on query pruning and path expansion, and the higher its contribution to path hit.

[0053] Side-level structure contribution score: The specific calculation method for the edge hierarchy contribution score S(e) of candidate key edge e is as follows: ; S(e)=η1·CrossPartition(e)+η2·BridgeScore(e)+η3·ExpandContribution(e); Where CrossPartition(e) indicates whether the candidate key edge e is a cross-partition edge; BridgeScore(e) represents the bridging score for one side e; ExpandContribution(e) represents the adjacency expansion contribution rate of edge e; CrossPartition(e) is set to 1 when the start and end points are in different partitions, and 0 otherwise; BridgeScore(e) is set to 1 when the edge connects different communities, different shards or different business entity clusters, and 0 otherwise. The specific calculation method for the adjacency expansion contribution rate ExpandContribution(e) of edge e is as follows: ExpandContribution(e)=ExpandHitCount(e,W) / (ExpandCandidateCount(e,W)+ε); Where ExpandHitCount(e,W) represents the number of times candidate key edge e enters the next hop candidate path within the statistics window W; ExpandCandidateCount(e,W) represents the number of times edge e is scanned as a candidate key edge.

[0054] As one possible implementation, an edge-level caching benefit score is also introduced. Those skilled in the art can determine this score based on the number of one-sided accesses, the reading and decompression time when not caching, the reading time after caching, and / or memory usage. This specification does not limit this score in detail.

[0055] S322, Calculate edge temperature score Edge temperature scoring is used to identify the precision level of an edge, and also to determine whether the candidate key edge needs to maintain its edge temperature profile, enter the hot edge or warm edge candidate set, and participate in small piece recombination. The score is obtained by weighted summation of edge-level access frequency score, edge-level path hit rate correction score, edge-level structure contribution score, and relation type weight corresponding to the candidate key edge. When introducing edge-level caching revenue score, the specific calculation method is as follows: T(e)=α×F(e)+β×R(type(e))+γ×P'(e)+δ×S(e)+θ×C(e); Where F(e) is the edge-level access frequency score; R(type(e)) is the corresponding relation type weight; P'(e) is the edge-level path hit rate correction score; S(e) is the edge-level structure contribution score; C(e) is the edge-level cache benefit score; α is the access frequency weight coefficient, β is the relation type composition weight coefficient, γ is the path hit weight coefficient, δ is the structure contribution weight coefficient, and θ is the cache benefit weight coefficient, satisfying: α+β+γ+δ+θ=1.

[0056] Those skilled in the art can set the aforementioned weighting coefficients themselves. For example, in scenarios that emphasize query performance, such as real-time transactions, risk control, and recommendations, the weights of γ and θ can be increased; in archival analysis scenarios, θ can be decreased and the weights related to compression benefits can be increased. The weights can be obtained by replaying historical queries to minimize the combined objective of query latency and storage costs after stratification.

[0057] In this embodiment, the segment-level S(B) is used for the overall temperature of the segment, and the edge-level S(e) is used for the candidate critical edge temperature. The two have different uses but the data sources are continuous.

[0058] S330: Edge temperature scoring determines the temperature level of candidate critical edges, forming the precise level corresponding to the candidate critical edges. The precision level will cover the current temperature level of the corresponding edge segment; As one possible implementation method; If T(e)≥TH ehot Then the precision level of this edge is hot edge, TH. ehot This represents the temperature threshold that defines a hot edge; If TH ewarm ≤T(e) <TH ehotThen the precision level of this edge is warm edge, TH. ewarm This represents the temperature threshold used to divide the area into warm edges; If T(e) <TH ewarm If the edge is cold, then its precision level is cold edge.

[0059] S340. Determine critical edges based on edge temperature scores; When the edge temperature score is less than the preset retention threshold and reaches the preset maximum number of times, this application reviews whether the corresponding candidate key edge meets the preset key judgment conditions. If the requirements are not met, the corresponding key identifier will be reclaimed. If any key decision condition is met, the key identifier is retained and used as a key edge; S350. After identifying the key edges, update / or create the edge temperature profile KeyEdgeTemperatureProfile. The edge relationship temperature profile data structure includes: edge identifier edgeId, fragment identifier fragmentId, edge relationship type edgeType, edge access frequency score, relationship type weight, path hit rate score, recent access time decay score, edge expansion contribution in the graph structure, cost when the edge is decompressed / re-encoded or read, estimated benefit of reducing query latency after the edge enters the cache, edge temperature score, current temperature level (consistent with the current temperature level of the edge fragment), accuracy level, last migration time l, and temperature level stability counter.

[0060] The contribution of the edge to the graph structure is calculated based on the degree of bridging and the contribution of adjacent expansion. This is intended to prevent low-frequency but structurally critical bridging edges from being misjudged as cold edges.

[0061] The cost of decompressing, recoding, or reading an edge is calculated based on the reading time, decompression time, recoding time, and block size, and is used to determine the additional cost when an edge is accessed from a cold layer or a compressed layer.

[0062] The estimated benefit of reducing query latency after an edge enters the cache is calculated based on the difference in expected latency before and after caching, the number of accesses, and memory costs. Its purpose is to determine the benefit of an edge entering the cache.

[0063] The temperature level stability counter is used for hysteresis judgment to prevent instantaneous hot spots or occasional cooling from triggering migration. Its counting basis is the statistical period rather than a single access. For example, with a statistical cycle of 5 minutes or 1 hour, the system calculates the target temperature level at the end of the cycle: If the accuracy level of the current cycle is the same as that of the previous cycle and meets the corresponding heating or cooling threshold, then the temperature level stability counter stabilityCounter is incremented by 1. If the threshold is not met or the target level changes in the opposite direction, the temperature level stabilization counter will be reset to zero or decayed by a preset ratio to prevent instantaneous hotspots from triggering migration.

[0064] Note that in this embodiment, to reduce data acquisition overhead, precise counting is used for hot edges and critical edges, while for ordinary cold edges, access events are only added to the approximate counter and summary counter of the edge shard to which they belong; path hit, cross-partition, bridging, and extension contribution signals can be aggregated by edge shard after the query execution is completed; precise events of the corresponding edge are only retained when drill-down is triggered.

[0065] S360. After constructing / updating edge temperature profiles for each key edge, associate the resulting edge temperature profiles with the corresponding piece temperature profiles. In this embodiment, the segmented temperature profile also records the number of key edges and the corresponding identifier for each key edge; the edge temperature profile KeyEdgeTemperatureProfile corresponding to the key edge also records the source segment identifier fragmentId; When the critical edge returns to a low-frequency state and fails to meet the drilling conditions for multiple consecutive cycles, release its critical identifier precise profile.

[0066] S370, after obtaining the edge temperature score or accuracy level, also includes migration management steps; In this embodiment, when the precision level corresponding to the critical edge is inconsistent with the current temperature level of the edge segment, the candidate critical edge is added to the migration set corresponding to the precision level; based on the preset local migration rules, the corresponding migration segment is obtained by recombining each critical edge in the migration set, and the migration segment is migrated according to the corresponding precision level.

[0067] The combinations to be migrated include hot-edge recombination combinations, warm-edge recombination combinations, and cold-edge recombination combinations; As an example, in the edge partition corresponding to a cold edge: If T(e)≥TH ehot If so, then the edge enters the hot edge recombination set; If TH warm ≤T(e) <TH ehot Then it enters the warm edge recombination set; If T(e) <TH ewarm Then, the original cold edge segmentation management will continue.

[0068] When the number of edges or the amount of data that have reached the migration level exceeds the proportion of the shards to the overall migration threshold, the entire edge shard B will be migrated. When the overall migration threshold is not reached, the edges in each reorganized set are reorganized into smaller fragments with the necessary starting point adjacent boundaries, relation type directories and version information, and then migrated. The remaining edges of the original fragments are retained.

[0069] The local migration rule is that the actual migration task is generated only when the temperature score of the candidate critical edge meets the conditions for multiple consecutive statistical periods, and the migration cooling time and system load allow it. Those skilled in the art can set the local migration rule according to actual needs.

[0070] S400: Based on the current temperature level of each edge segment, perform differentiated storage according to preset storage rules; S410 employs a low-decoding-overhead storage and caching strategy for hot-edge fragments (including small fragments that are being migrated); S411, hot-level matching storage and caching core strategy; For objects currently classified as "hot," the system employs a storage strategy prioritizing query performance, as detailed below: Stored in memory adjacency cache, NVMe SSD, or high-performance storage layer; Use light compression or no compression; Create a starting adjacency index, an ending reverse index, and a relation type local index; The hot sub-plots are arranged continuously according to their starting point and relationship type; Cache frequently occurring hot edge path segments; The query executor prioritizes batch prefetching; Cache eviction has a lower priority than warm edge and cold edge caches; Maintain fast version numbers.

[0071] S412, Conditions for determining thermal grade migration; In this embodiment, if the target temperature level of the edge segment is thermal, and the overall temperature score T(B) of the segment continuously meets the corresponding thermal level temperature rise threshold TH... hotup Alternatively, the precision level can be achieved by ensuring that the edge temperature score T(e) continuously meets the thermal level heating threshold TH. hotup If the critical edge is added to the hot edge recombination set, and the stabilization period, migration cooling time, and system load meet the conditions, then the entire partition B or the subpart B formed based on the hot edge recombination set will be reassigned. sub Add a thermal layer migration task.

[0072] S413, Thermal rating migration execution steps; Read the edge position mapping table and determine the migration unit boundary; If a small number of critical edges are heated, the critical edges in the hot edge reorganization set are reorganized into hot mini-pieces according to the starting point, relation type, and adjacency order. Generate new data blocks with light compression or no compression; Create or update adjacency indexes, inverted indexes, relation-type local indexes, and path hint bitmaps; Write high-frequency adjacency lists or path fragments to the cache; Update edge or shard position mappings atomically with the new version number; Old copies are recycled after no activity queries reference them.

[0073] S420 employs a balanced compression and local caching strategy for warm-edge fragmentation; S421, temperature-level matching storage and caching core strategy; For warm-edge shards or warm-edge mini-shards formed by key edges in a warm-edge recombination set, the system adopts a strategy that balances storage cost and query performance. The specific strategy is as follows: Stored in the regular SSD layer; Use block-level compression, such as LZ4, ZSTD low compression levels, or dictionary compression; Grouped and stored by edge type and starting point; Create a sparse index; During the query, partial decompression is performed based on edge fragmentation; When the frequency of short-term access increases, it enters the hot edge reorganization set.

[0074] S422, Temperature Level Migration Judgment Criteria; The machine performs the determination. If the target temperature level of the edge segment is warm, or if the cold edge segment consists of a set of key edges T(e) that continuously meet the warm edge heating threshold, or the hot edge segment consists of a set of key edges that continuously meet the hot edge cooling threshold but not the warm edge cooling threshold, then the small segments obtained by the entire segmentation will be adjusted to the warm edge strategy.

[0075] S423, Temperature Level Migration Execution Steps; Regroup edge data by edge type, starting range, or segment number; Warm edge blocks are generated using block-level compression, and the minimum starting point, maximum starting point, relation type set, and block offset are recorded for each block; Establish sparse indexes and local cache directories; Place warm edge blocks that may be accessed consecutively in the query into the local cache instead of the hot edge core adjacency cache; Update the edge position mapping table and compression format identifier.

[0076] S430 employs a high compression ratio storage and on-demand decompression strategy for cold edge fragmentation; In this embodiment, the normal management object of the cold layer is the cold edge segment B, rather than each edge within the segment.

[0077] The system forms cold edge fragments according to relation type, time range, starting vertex interval, or archive batch. It uses high compression ratio encoding and writes to low-cost disk, object storage, or archive storage, retaining only the Bloom filter, block-level vertex range, relation type summary, archive block offset, and fragment temperature profile (FragmentTemperatureProfile).

[0078] When a query hits a cold edge shard, the query executor first performs block-level filtering based on relation type, vertex range, time range, attribute summary, and Bloom filter.

[0079] After determining that a hit is likely, perform merge reading and batch decompression on one or more cold edge fragments.

[0080] Occasional query results can be cached for a short period of time, but this does not change the shard temperature level.

[0081] If the overall temperature score T(B) of the corresponding segment is continuously lower than the temperature level cooling threshold TH warmdown If the cooling time is exceeded, it will remain or migrate to the cold edge layer; If cold edge fragments show consecutive hits, concentrated path hits, or abnormal decompression costs within a statistical window, candidate key edges are identified and marked with key identifiers to increase subsequent sampling intensity. These key identifiers only trigger observation or key edge drill-down and are not equivalent to migration commands.

[0082] The formal migration process begins only when a segment T(B) continuously meets the warm edge heating condition, or when a candidate key edge T(e) within a segment continuously meets the heating condition and enters the warm edge recombination set.

[0083] When the overall migration conditions are not met, the recombined small segments are migrated, and the remaining parts of the original cold edge segments remain in the cold layer.

[0084] Furthermore, the edge versions before and after the migration are verified. The specific version verification steps are as follows: Record the old version number before migration (oldVersion), the old storage location pointer before migration (oldPointer), the number of edges (edgeCount), the edgeKeyHash summary of the edge identifier set, the endpointHash summary of the edge start and end points, and the propertyHash summary of the edge properties. After writing to the target layer, generate the new version number after the migration of newVersion and the new storage location pointer after the migration of newPointer, and recalculate the digest; Verify the number of edges, edge identifiers, start and end points, and attribute summaries; Perform a trial reading at the new location; After successful verification, the mapping is switched by comparison and swapping.

[0085] Furthermore, queries that have already started using the old version number (oldVersion) will continue to use the old version number (oldVersion) before the migration, while new queries will use the new version number (newVersion) after the migration. The old version reference count will be reset to zero and the old version will be recycled. If the verification fails, the mapping will be rolled back and the invalid new copy will be deleted.

[0086] This invention uses edge relationships as the granularity for precise temperature identification, effectively overcoming the shortcomings of traditional hot and cold tiering schemes, such as coarse management granularity and difficulty in adapting to the expansion characteristics of graph query paths. Furthermore, it integrates multi-dimensional indicators such as edge access frequency, relationship type weight, path hit rate, structural contribution, and caching benefits to construct a temperature scoring model. This ensures that frequently accessed hot edges located in critical query paths are accurately identified and stored in a high-performance storage layer. Lightweight compression, dense indexing, and path prefetching strategies are employed to significantly reduce random I / O and decoding latency in multi-hop traversal and complex path queries. Simultaneously, cold edges with low access frequency and relatively limited value are encoded with high compression ratios and archived for storage, effectively reducing storage resource consumption.

[0087] This invention addresses this issue by setting temperature rise / fall hysteresis thresholds, stabilization cycle counts, and migration cooling times. This ensures that edge temperature only triggers hierarchical switching when migration conditions are consistently met, effectively preventing frequent migrations and system jitter caused by short-term access fluctuations. Furthermore, it employs a hybrid metadata management approach combining edge shard temperature summaries with critical edge temperature profiles, supporting precise edge-level temperature identification while avoiding excessive metadata expansion. During migration, this invention utilizes mechanisms such as edge version numbers, atomic updates to the location mapping table, dual-version read transitions, and old replica reference count reclamation to ensure consistent results for concurrent queries before and after migration. The query executor calculates the proportion of hot edges, cold edges, filtering selection rate, and historical hit rate for each candidate expansion direction during the planning phase. It prioritizes directions with high hot edge proportions and good filtering effects, while performing pre-filtering operations such as relation type, attribute range, or Bloom filters on directions with high cold edge proportions. This reduces unnecessary cold edge block reads and high-compression block decompression overhead, maintaining stable query performance in graph database scenarios with mixed hot and cold storage. It balances storage costs and query efficiency, demonstrating good engineering compatibility and deployment feasibility.

[0088] This application also proposes a temperature-aware hierarchical query method for edge relationships, which is used when a graph database executes a query, the query executor reads edge temperature information and adjusts the query strategy according to the temperature level. The specific steps are as follows: S500: Determine candidate expansion directions based on the target query statement; The specific steps are as follows: S510, Parse the query statement; Parse the user query to obtain the start point, end point, relationship type, path length, filter conditions, and return fields, and obtain the relationship schema corresponding to the query statement.

[0089] S520, Generate candidate expansion directions; Based on the relational schema in the query statement, determine the candidate expansion direction.

[0090] This is existing technology. The innovation of this application lies in the subsequent calculation of the expansion cost, the determination of the priority of candidate expansion directions based on the expansion cost, and the step of differential reading based on temperature stratification. Therefore, the steps of determining candidate expansion directions will not be described in detail.

[0091] S600. Determine the expansion cost corresponding to each candidate expansion direction, and determine the target expansion direction based on the expansion cost; The specific steps are as follows: S610. Determine the number of candidate edges corresponding to the current candidate expansion direction; The system queries the FragmentTemperatureProfile, KeyEdgeTemperatureProfile, or edge location mapping table to obtain the current temperature level or precision level (prioritizing precision level) of the edge fragment corresponding to the candidate edge as the temperature level of the candidate edge. It also queries the storage level, compression format, and cache directory corresponding to each candidate edge.

[0092] S620. Based on the current temperature level of the edge segment and the precision level corresponding to the key edge, determine the hot edge ratio, warm edge ratio and cold edge ratio corresponding to the current candidate direction, and calculate the average hierarchical access cost corresponding to the current candidate direction based on the obtained hot edge ratio, warm edge ratio and cold edge ratio. The average hierarchical access cost AvgLayerCost(d) for candidate expansion direction d is calculated as follows: AvgLayerCost(d)=HotRatio(d)·C hot +WarmRatio(d)·C warm +ColdRatio(d)·C cold; Where HotRatio(d) represents the proportion of the hot side in the expansion direction; WarmRatio(d) represents the proportion of the warm side in the expansion direction; ColdRatio(d) represents the proportion of the cold side in the expansion direction; C hot C represents the unit cost of accessing a hot edge. warm C represents the unit cost of accessing the warm edge. cold This represents the unit cost of accessing a cold edge. Those skilled in the art can set the unit cost according to the actual situation to satisfy C.hot <C warm <C cold That's all.

[0093] S630. Determine the expansion cost based on the number of candidate edges, the average hierarchical access cost, and the current expansion direction.

[0094] For each candidate expansion direction d, the query executor calculates the expansion cost, specifically as follows: ; Where N(d) represents the number of candidate edges under candidate expansion direction d; AvgLayerCost(d) represents the average hierarchical access cost of candidate expansion direction d; FilterPassRatio(d) represents the proportion of candidate edges that still need to be read after being judged by relation type filtering, attribute filtering, vertex range filtering or Bloom filter; P hit (d) represents the historical path hit rate corresponding to candidate expansion direction d.

[0095] According to the above formula, the more candidate edges there are, the higher the proportion of cold edges, and the higher the proportion of edges that still need to be read after filtering, the greater the expansion cost; the higher the historical path hit rate, the lower the expansion cost. The query executor selects the expansion direction in ascending order of expansion cost ExpandCost(d), thus prioritizing the expansion of path directions with a high proportion of hot edges, good filtering effect, and high historical hit rate.

[0096] In practical applications, each candidate expansion direction can be selected as the target expansion direction in ascending order of the expansion cost ExpandCost(d).

[0097] S700. Based on the target expansion direction, perform differentiated reading according to the current temperature level of the corresponding edge segment to obtain the corresponding query results; This embodiment performs differentiated reading based on the target expansion direction and a preset temperature sensing query plan. The temperature sensing query plan includes the following steps: Perform cache reads or batch prefetches on hot edges in the target expansion direction; Perform batch reading and partial decompression on the warm edge segment by edge; For cold edges, first perform block-level filtering. If the filtering may hit, read and decompress according to the cold edge merging rules. The expanded results of hot edges, warm edges, and cold edges are merged according to path variables, edge directions, and filtering conditions.

[0098] In the query plan generation stage, this invention reads the edge temperature summary and calculates the hot edge ratio, cold edge ratio, path hit rate, and filter selection rate of candidate expansion directions (determined by the relation schema in the query statement) (obtained by the system from the temperature summary table and the edge position mapping table), thereby calculating the corresponding expansion cost. When there are multiple candidate expansion directions, the direction with a high proportion of hot edges and a high filtering selection rate is selected first based on the obtained expansion cost. For directions with a high proportion of cold edges, when the query plan detects that the proportion of cold edges in a certain candidate expansion direction is higher than a preset threshold, the query executor does not immediately read the cold edge data block, but first performs pre-filtering based on the query conditions.

[0099] In this embodiment, if the query request specifies the relationship type, only cold edge blocks that match the relationship type are retained; if the query request includes edge attribute conditions, it is determined whether a match is possible based on the attribute summary or minimum / maximum range of the cold edge block; if the query request specifies the start point, end point, community, shard, or time range, irrelevant blocks are first filtered based on the cold edge block metadata; if the cold edge layer maintains a Bloom filter, it is first determined whether the target edgeId, vertex ID, or relationship type may exist in the cold edge block. The system only performs cold edge block reading and decompression when the above filtering conditions determine that the cold edge block may be hit; if the filtering result indicates that it is impossible to hit, the cold edge block is skipped, thereby reducing the number of cold edge random I / O and high compression block decompression times, and reducing the tail latency of path queries with a high proportion of cold edges.

[0100] During the query execution phase, path fragment caching and batch prefetching are used for continuous hot edge fragments; local decompression is performed on warm edge fragments; and delayed reading, merged reading, and batch decompression are performed on cold edge fragments. For queries whose return results do not depend on a complete cold edge scan, candidate results generated from hot and warm layers are returned first, and the supplementary results from cold edge fragments are processed as asynchronous or two-stage query results.

[0101] Specific judgment criteria include: If a candidate edge is a hot edge and its adjacency list exists in the cache directory, then the memory adjacency cache is read directly; otherwise, hot edge prefetching is triggered. If the candidate edge is a warm edge and the number of hits within the same edge fragment reaches the preset batch read threshold (batchReadThreshold), then the edge fragments are read in batches and partially decompressed. If a candidate edge is a cold edge (in the query planning phase, candidate key edges are usually candidate key edge patterns, candidate adjacency ranges, or candidate edge fragments, and do not indicate that a specific edgeId has been confirmed to exist), then the relation type, vertex range, time range, and Bloom filter are executed first, and the archive block is read only if the filtering result indicates that it may exist.

[0102] As one possible implementation method: Based on a preset temperature sensing query plan, prefetching, batch reading, filtering, decompression, and result merging are performed. During the query execution phase, path fragment caching and batch prefetching are used for continuous hot edge segments; warm edges are partially decompressed by edge fragmentation; and cold edges are read with delay, merged read, and batch decompressed. For queries whose results do not depend on a complete cold edge scan, candidate results generated by hot and warm edges are returned first, and cold edge supplementary results are processed as asynchronous or two-stage query results.

[0103] Furthermore, consecutive hot edge segments do not mean that there are no other temperature edges in the entire query path, but rather that among several adjacent edge patterns in the query plan, there exist sub-paths that can be continuously read from the hot layer or path segment cache. The criteria for determining consecutive hot edge segments are: Edge patterns are adjacent in the query path, for example, the i-th hop and the (i+1)-th hop are connected; The temperature rating of the corresponding edge or edge segment is HOT. Adjacent edge segments have the same or compatible path segment cache key, which can consist of the starting range, relation type sequence, direction, attribute filtering conditions, and version number; The length of the continuous hot edge reaches a preset threshold or the segment has been hit in the path segment cache directory; The required version for this segment is the same as the version used to query the snapshot. When the above conditions are met, the query executor can directly read the path segment cache or perform batch prefetching of the hot edge adjacency list.

[0104] Furthermore, the cold edge access merging method is as follows: The query executor groups cold edge requests that are hit in the same query or the same batch of concurrent queries according to the unique identifier of the storage block, storageBlockId, the relation type of the edge, the vertex range, and the archive offset (if all are consistent, they are grouped into the same smallest execution group, and only one block read and one decompression are initiated). For multiple edge unique identifiers edgeId or vertex ranges within the same archive block, only one block read and one decompression are initiated. Then, the results are split in memory according to edgeId, start point, end point, and filtering conditions, thereby merging multiple small I / O operations into a small number of sequential block reads.

[0105] Furthermore, cold edges can also be cached, but their cached objects are short-term query results or decompressed archive block fragments. The cache duration is short, the priority is low, and they are not included in the hot edge adjacency cache. The above-mentioned delayed read cold edge means that filtering is prioritized to reduce the physical reads of cold edges; if a read is necessary and a result is generated, it can be written to the short-term result cache of the cold edge so that it can be reused for repeated queries in a short period of time.

[0106] In this embodiment, when the query executor finds that a cold edge fragment is hit continuously in a short period of time and the decompression cost exceeds the threshold, it does not migrate immediately, but adds the fragment to the warming observation queue. Only when the path hit rate, access frequency and stability period meet the conditions will the migration from the cold edge layer to the warm edge layer or the hot edge layer be triggered, thereby avoiding cache pollution and migration jitter caused by occasional queries.

[0107] S800: Update the query execution log based on the obtained query results; The number of accesses, path hits, changes in the number of candidates, execution time, and cache hit status of this query are written back to the query execution log for subsequent temperature profile updates.

[0108] This invention is applicable to graph database scenarios where there are significant differences between hot and cold access and where querying primarily involves traversing relational edges, including financial risk control graphs, knowledge graphs, social relationship graphs, recommendation relationship graphs, permission relationship graphs, supply chain tracking graphs, and IoT relationship analysis graphs. Its applicability stems from the fact that the query costs in these scenarios are mainly generated by edge expansion, path matching, and adjacency traversal. Within the same graph, there are both high-frequency, real-time accessed key relational edges and ordinary relational edges used only for historical backtracking, offline statistics, or low-frequency completion.

[0109] The following uses a financial risk control chart database as an example to illustrate the specific implementation process of this invention: A financial institution has constructed a corporate relationship graph database, which includes nodes such as enterprises, individuals, accounts, devices, addresses, and mobile phone numbers, as well as edge relationships such as transfers, guarantees, appointments, shareholdings, logins, shared devices, and registered addresses.

[0110] During system operation, it was found that: relationships such as transfers, guarantees, actual controllers, and shareholdings are frequently involved in risk path queries; relationships such as registered addresses, contact numbers, and browsing history are numerous, but most have low access frequency; some edges, although accessed infrequently, are often located in high-risk group identification paths; mixed storage of hot and cold edges leads to a large amount of random I / O and decompression overhead during multi-hop queries.

[0111] After adopting the hierarchical storage method disclosed above, the following steps are performed.

[0112] Step 1: Form edge fragments and collect two levels of statistical signals; Graph databases can be used to query enterprise relationship paths, account transfer hops, shared devices, and historical registration addresses.

[0113] The system shards low-frequency registration addresses, browsing activity, and historical relationships into cold edge fragments B1 and B2 based on relationship type and starting point range. It then aggregates access counts, path hits, candidate scans, cross-partition activity, and decompression time by fragmentId to maintain the corresponding fragment temperature profile. Simultaneously, it collects known high-frequency or critical relationships such as transfers, guarantees, and controlling shareholders to further maintain the edge temperature profiles corresponding to critical edges.

[0114] For example, cold edge shard B1 contains 1000 registered addresses, shared devices, and regular browsing relationships; B2 contains 800 historical archive relationships. B1 includes the critical edge E3 "Enterprise C -> Address D, Registered Address" and the critical edge E6 "Enterprise C -> Device Z, Shared Device". Within the statistics window, B1 has a total of 160 visits and 30 path hits; B2 has a total of 20 visits and 1 path hit.

[0115] Step 2: Calculate the overall temperature of the cold edge segment; The system calculates the following for B1: CrossPartition(B1) = 0.08, BridgeScore(B1) = 0.12. ExpandContribution(B1)=0.10.

[0116] Combining relation type composition, recent access, decompression cost, and caching benefit, we obtain T(B1) = 0.31; and calculate T(B2) = 0.09.

[0117] Let TH warm =0.35, therefore B1 and B2 as a whole maintain cold edge segmentation, and the complete temperature is not calculated for each of the 1800 edges.

[0118] Step 3: Drill down into candidate critical edges; In this case, although T(B1) is below the warm edge threshold, the query log shows that a small number of edgeIds in B1 contributed more than 70% of the path hits, triggering drill-down to identify the concentrated hit edges; Identify candidate key edges (included in the previously labeled E3 and E6), and perform temperature identification and build or update edge temperature profiles for the identified candidate key edges.

[0119] In this case, T(E6) was calculated to be 0.84, while the verification temperature of E3 was 0.22, and E3 did not meet other key judgment conditions.

[0120] E6 then enters the hot edge recombination set, and E3 recovers the edge temperature profile and key identifiers, and continues to be managed by the fragmentation summary of B1 along with the remaining ordinary edges.

[0121] Step 4: Generate physical migration units and perform tiered storage; Since the proportion of edges in B1 that meet the heating conditions is low, the system does not migrate B1 as a whole, nor does it directly and arbitrarily move a single E6. Instead, it reorganizes E6 and several edges with the same starting point, the same relationship type, and the need to jointly maintain the adjacency continuity into a thermal mini-segment B1-H.

[0122] B1-H is written to the NVMe SSD and enters the memory adjacency cache, using lightweight encoding; the remaining B1 portion continues to be stored in the cold layer as highly compressed archive blocks.

[0123] The system updates the location mapping of B1-H and related edge identifier edgeId, and reclaims old copies after the old version references are reduced to zero.

[0124] Step 5: Query the actuator's two-stage temperature information; When executing the query "Does Enterprise C share equipment or registered address with the blacklisted entity?", the query executor first reads the cold fragment summary of B1 and performs relation type, vertex range and Bloom filter filtering; after finding that E6 in the candidate range has an edge temperature profile, it prioritizes reading the relevant adjacency data of E6 from B1-H.

[0125] The registered address relationship is still processed by performing merge reading and batch decompression according to B1, thus utilizing the low-latency access of the critical edge while avoiding the overall overheating of the cold shard.

[0126] Step Six: Dynamic Adjustment and Image Retrieval; If the overall access and path hits of B1 continue to increase and cause T(B1) to continuously reach the warm edge threshold, the entire system will migrate B1 to the warm layer; if only a small number of edges in B1-H remain hot, the small fragment layout will be maintained.

[0127] If E6 no longer meets the key relationship and structural contribution conditions for multiple consecutive periods, and its T(E6) is lower than the preset retention threshold, the system releases the edge temperature profile KeyEdgeTemperatureProfile of E6 and re-incorporates the subsequent statistics into the corresponding shard summary.

[0128] This application also provides a temperature-sensing hierarchical storage system for edge relationships of hot-cold mixed graph data; The system includes the following modules: Edge access behavior collection module: Used to collect the access behavior of edge relationships during the operation of graph database, including edge access count, access time, query type, path hit count, cache hit status, decompression time and other information.

[0129] Two-level temperature scoring module: used to calculate the overall temperature T(B) of the fragment, the critical edge temperature T(e), the path hit concentration and KeySignal(B), and output the fragment target level and the candidate critical edge target level.

[0130] Two-level temperature profile generation module: used to generate piecewise temperature profiles by edge segmentation, and generate edge temperature profiles for key edges, and maintain the association, recycling and version status of the two.

[0131] Tiered storage management module: Used to store hot edge shards, warm edge shards, cold edge shards, and key edge recombined small shards to different levels, and maintain the position mapping between edge level and shard level.

[0132] Compression strategy selection module: used to select different compression methods based on the edge temperature level, including lightweight compression, block compression and high compression ratio compression.

[0133] Cache strategy control module: used to execute different caching strategies based on edge temperature level, including hot edge adjacency caching, path segment caching, warm edge local caching, and cold edge short-term result caching.

[0134] The fragmented migration scheduling module is used to select between overall fragment migration or critical edge reorganization and small fragment migration based on fragment temperature, critical edge temperature, and the proportion of heated edges, and controls the migration through hysteresis threshold, stabilization period, and load awareness mechanism.

[0135] The query execution collaboration module is used to prioritize reading the shard temperature summary and override the shard default level when there is a critical edge temperature profile. It adjusts the path expansion, prefetching, cold edge block filtering, and batch decompression strategies based on the two-level temperature information.

[0136] Consistency Maintenance Module: Used to maintain query consistency during edge migration, compression format conversion, and cache refresh, including edge version number, location mapping update, old replica recycling, and concurrent access verification.

[0137] The monitoring and visualization module outputs the statistical results of edge temperature layering, including the number of hot edge fragments, warm edge fragments, and cold edge fragments; the number of edges and storage usage at each layer; the number of current critical edge temperature profiles and their recycling; the total number of fragment migrations and the number of small fragment migrations; the query hit rate and compression rate at each layer; the hot layer cache hit rate; the cold edge fragment drill-down trigger rate; the critical edge missed detection rate or false temperature rise rate; the average query latency and tail latency changes; and the temperature metadata usage. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple; relevant details can be found in the description of the method embodiment.

[0138] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0139] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] This invention is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0141] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0143] It should be noted that: The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0144] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments.

[0145] Furthermore, it should be noted that the specific embodiments described in this specification may differ in the shape and name of their components. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the scope of protection of this patent.

Claims

1. A method for edge-relational temperature-sensing hierarchical storage, characterized in that, Based on the current temperature level of each shard, differentiated storage is performed according to preset storage rules, including the following steps: Collect edge relationship access behavior data corresponding to the graph database; Based on edge relationship access behavior data, temperature assessment is performed on each edge shard to obtain the overall temperature score of the shard corresponding to each edge shard, and the target temperature level corresponding to the edge shard is determined based on the overall temperature score of the shard. Based on the overall temperature change trend of the edge segment, it is determined whether to perform segment temperature increase. When it is determined that segment temperature increase is to be performed, the edge segment is migrated according to the target temperature level, and the target temperature level is used as the current temperature level of the corresponding edge segment. After obtaining the overall temperature score of each edge segment, the temperature of the edges marked as candidate key edges within each edge segment is evaluated to obtain the edge temperature score of each candidate key edge. Based on the edge temperature score, the key edges and their precision level are determined. The candidate key edges include edges whose edge relationship type is a preset key relationship.

2. The edge-relationship temperature-sensing hierarchical storage method according to claim 1, characterized in that: Based on edge relationship access behavior data, calculate the shard-level access frequency score, shard relationship composition score, shard-level path hit rate score, and shard-level structure contribution score corresponding to the edge shard, and calculate the overall shard temperature score of the edge shard. in: Based on the relation type weight corresponding to the edge relation type, and the proportion of the number of edges of various relation types or the weighted proportion of path hits within the edge partition, the corresponding partition relation composition score is calculated. The corresponding shard-level structural contribution score is calculated based on the shard cross-partition contribution, shard bridging contribution, and shard adjacency extension contribution corresponding to the edge shard. The shard cross-partition contribution represents the proportion of cross-partition edges in the edge shard. The shard bridging contribution represents the proportion of the number of bridging edges within the edge shard or the proportion of the path hit weight value corresponding to the bridging edge. The shard adjacency extension contribution represents the adjacency extension contribution rate of the edge shard.

3. The edge-relationship temperature sensing hierarchical storage method according to claim 2, characterized in that: Within the statistics window, the average contribution score corresponding to each edge relationship type is calculated and statistically analyzed based on the query result paths returned by the path-type query. The relation type weights corresponding to each edge relation type are updated based on the obtained average contribution score.

4. The edge-relationship temperature sensing hierarchical storage method according to claim 2, characterized in that: The total number of times the corresponding edge within the edge segment is scanned as a candidate edge, and the total number of times it is actually retained and enters the next hop candidate path, are used to calculate the corresponding segment adjacency expansion contribution.

5. The edge-relationship temperature sensing hierarchical storage method according to claim 2, characterized in that: Calculate the sharding cache benefit score for edge sharding; After normalizing the shard-level access frequency score, shard relationship composition score, shard-level path hit rate score, shard-level structure contribution score, and shard cache benefit score corresponding to the edge shard, the corresponding overall shard temperature score is calculated.

6. The edge-relationship temperature sensing hierarchical storage method according to claim 1, characterized in that: According to the preset drill-down rules, the edge is marked with key identifiers based on the path hit, bridging and adjacency expansion of the edge piece, and the corresponding edge is marked with key identifiers based on the preset key relationship. All edges with key identifiers are selected as candidate key edges. The candidate key edges also include: Bridging edges connect different communities, different data shards, or different business entity clusters. Edges that are hit in a concentrated area are those whose path hit counts exceed the concentration threshold as a percentage of the total path hit counts in the corresponding edge partition. Key extension edges are those edges that have entered the next hop candidate path a preset threshold number of times, indicating that their extension contribution is significant. Edges with abnormal costs: Edges where the cost of reading and decompressing the cold layer exceeds a preset cost threshold.

7. The edge-relationship temperature-sensing hierarchical storage method according to claim 1, characterized in that, The specific steps for evaluating the temperature of candidate critical edges and obtaining their edge temperature scores are as follows: Based on edge relationship access behavior data, edge-level access frequency score, edge-level path hit rate score, and edge-level structure contribution score of candidate key edges are calculated. Among them, the edge-level structure contribution score is calculated based on the cross-partition situation, bridging situation, and adjacency expansion contribution rate of the candidate key edges. Based on the edge-level access frequency score, the edge-level path hit rate score, the edge-level structure contribution score, and the relation type weight corresponding to the candidate key edge, the corresponding edge temperature score is calculated.

8. The edge-relationship temperature-sensing hierarchical storage method according to claim 6, characterized in that, It also includes the identification and recycling steps: When the edge temperature score is less than the preset retention threshold and reaches the preset maximum number of times, check whether the corresponding candidate key edge meets the preset key judgment conditions. If the requirements are not met, the corresponding key identifier will be reclaimed. If any key determination condition is met, retain the key identifier, treat it as a key edge, and construct or update the edge temperature profile.

9. The edge-relationship temperature-sensing hierarchical storage method according to claim 8, characterized in that, It also includes a local migration step; When the precision level corresponding to a critical edge is inconsistent with the current temperature level of the edge segment it belongs to, the candidate critical edge is added to the migration set corresponding to the precision level. Based on preset local migration rules, the corresponding fragments to be migrated are obtained by recombining each key edge in the set to be migrated, and the fragments to be migrated are migrated according to their corresponding precision levels.

10. A layered query method for edge-relation temperature sensing, characterized in that: Determine candidate expansion directions based on the target query statement; Determine the expansion cost corresponding to each candidate expansion direction, and determine the target expansion direction based on the expansion cost; Based on the target expansion direction, differentiated reading is performed according to the current temperature level of the corresponding edge piece to obtain the corresponding query results; The method for calculating the expansion cost corresponding to the current candidate expansion direction is as follows: Determine the number of candidate edges corresponding to the current candidate expansion direction; Based on the current temperature level of the edge fragment and the precision level corresponding to the key edge, determine the hot edge ratio, warm edge ratio and cold edge ratio corresponding to the current candidate direction, and calculate the average hierarchical access cost corresponding to the current candidate direction based on the obtained hot edge ratio, warm edge ratio and cold edge ratio. The expansion cost of the current expansion direction is determined based on the number of candidate edges and the average hierarchical access cost.

Images