Cloud-native property graph data storage system, method, and computer device

Abstract

The application relates to a cloud-native attribute graph data storage system, method and computer device. The system comprises a data storage module for splitting an attribute graph into a topological structure subfile, an attribute subfile and a vertex subgraph file in a cloud object storage service; a query planning module for dividing a query into multiple query segments and rewriting the query segments into subqueries of corresponding subgraph files according to labels and attributes, selecting a data access mode meeting a time delay constraint and having the lowest cost for each subquery, and generating an economic cost-optimal query plan; a cache manager for decomposing a subquery into fine-grained data segments according to cache hit conditions and dynamically generating a hybrid query plan, and performing a cost-priority cache replacement based on a cache comprehensive benefit model; and a query execution module for executing the hybrid query plan segment by segment according to the query segments and merging the query segments to generate a query answer. The system can reduce cloud storage access cost and improve query processing efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a cloud-native attribute graph data storage system, method and computer device. Background Technology

[0002] With the development of graph data processing technology, attribute graphs, as a flexible and highly expressive data model, have gradually gained widespread application. Using vertices and edges as basic building blocks, both vertices and edges can carry attributes and be labeled to distinguish entity and relation types, effectively representing complex relational data in scenarios such as knowledge graphs, social networks, and recommendation systems. Graph pattern query technology based on attribute graphs has also developed accordingly. By identifying specific graph structures and combining vertex or edge attribute filtering conditions to extract target data from the graph, mainstream graph query languages ​​such as Cypher, GQL, and Gremlin have emerged. Meanwhile, with the continuous growth of data volume, cloud environments, with their dynamic elasticity, scalability, and economies of scale, are widely adopted in large-scale data management scenarios. Managing graph data in cloud environments has become the mainstream attribute graph data storage method in the industry.

[0003] However, traditional attribute graph data storage methods face the challenge of balancing economic cost and query performance in a cloud environment. Traditional attribute graph database management systems typically employ a distributed architecture, whose complex design leads to high economic costs for data storage and query processing, and makes it difficult to dynamically scale up or down with increasing data volume. Even when deployed in a cloud environment, they often require external components to enhance scalability, further increasing architectural complexity and economic costs, and incurring additional performance overhead. While some cloud-native attribute graph storage systems have automated access control, scaling, and version upgrades through cloud-native component hosting, their underlying technical architecture remains a storage-compute coupled model. This fails to fully leverage the economic and elastic advantages of decoupling storage and computation in a cloud environment. In querying large-scale attribute graph data, the overall economic cost remains high, making it difficult to effectively reduce operating costs while ensuring query performance. Summary of the Invention

[0004] Therefore, it is necessary to provide a cloud-native attribute graph data storage system, method, and computer device to address the aforementioned technical issues.

[0005] A cloud-native attribute graph data storage system, the system comprising: The data storage module is used to store attribute graph data in the cloud object storage service. The attribute graph data includes edge data and vertex data. Edge data under the same edge label is split into multiple edge sub-graph files according to edge attributes. The edge sub-graph files include topology sub-files and attribute sub-files. Vertex data under the same vertex label is split into multiple vertex sub-graph files according to vertex attributes. The query planning module is used to divide the graph pattern matching clause into multiple query segments. Each query segment contains labeled graph patterns and data operations. The labeled graph patterns in each query segment are rewritten according to vertex labels and edge labels, and the data operations are rewritten according to the attributes involved, into subqueries of the corresponding subgraph files in each cloud object storage service. A pre-set greedy access method is used to select the data access method that meets the query latency constraints and has the lowest cloud storage access cost for each subquery, generating the most cost-effective query plan. The cache manager receives the optimal cost query plan and, based on the cache hit rate and data access method of each subquery in the optimal cost query plan, decomposes each subquery into a combination of multiple query methods for fine-grained data fragments, dynamically generating a hybrid query plan. The cache manager also receives attribute data retrieved from the data storage module by the query execution module and caches it locally. Based on a comprehensive cache benefit model, it determines the cache benefit of each piece of data to be cached and dynamically performs cost-priority cache replacement. The comprehensive cache benefit model normalizes the time and economic costs of network transmission, data scanning, and computational layer operations when retrieving cached data from the data storage module into a benefit value to maximize the economic benefit of caching. The query execution module receives the mixed query plan and executes it segment by segment in sequence. For each subquery in the current query segment, according to the data source and data access method in the mixed query plan, it first retrieves the topology subfile, calls the corresponding operator to perform graph pattern matching evaluation, performs structure and identifier evaluation on the graph pattern evaluation result, and finally retrieves the attribute value in the attribute subfile according to the identifier in the evaluation result, calls the corresponding operator to perform data operation evaluation on the attribute value, and merges the query segments to generate the query answer.

[0006] A cloud-native attribute graph data storage method, the method comprising: Attribute graph data is stored in the cloud object storage service. The attribute graph data includes edge data and vertex data. Edge data under the same edge label is split into multiple edge sub-graph files according to edge attributes. The edge sub-graph files include topology sub-files and attribute sub-files. Vertex data under the same vertex label is split into multiple vertex sub-graph files according to vertex attributes. The graph pattern matching clause is divided into multiple query segments. Each query segment contains labeled graph patterns and data operations. The labeled graph patterns in each query segment are rewritten according to vertex labels and edge labels, and the data operations are rewritten according to the attributes involved, into subqueries of the corresponding subgraph files in each cloud object storage service. A pre-set greedy access method is used to select the data access method that satisfies the query latency constraint and has the lowest cloud storage access cost for each subquery, generating the optimal economic cost query plan. The system receives the optimal economic cost query plan, and based on the cache hit status and data access method of the data required by each subquery in the optimal economic cost query plan, it decomposes each subquery into a combination of multiple query methods for fine-grained data fragments, dynamically generates a hybrid query plan, receives attribute data retrieved from the data storage module and caches it locally, determines the cache benefit of each data to be cached based on the comprehensive cache benefit model, and dynamically executes cost-priority cache replacement; the comprehensive cache benefit model normalizes the time cost and economic cost of network transmission, data scanning and computing layer operation when each cached data is retrieved from the data storage module into a benefit value, so as to maximize the economic benefit of caching; The system receives a mixed query plan and executes it segment by segment in the order of the query segments. For each subquery in the current query segment, according to the data source and data access method in the mixed query plan, it first retrieves the topology structure subfile, calls the corresponding operator to perform graph pattern matching evaluation, performs structure and identifier evaluation on the graph pattern operation result, and finally retrieves the attribute value in the attribute subfile according to the identifier in the operation result, calls the corresponding operator to perform data operation evaluation on the attribute value, and merges the query segments to generate the query answer.

[0007] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps: Attribute graph data is stored in the cloud object storage service. The attribute graph data includes edge data and vertex data. Edge data under the same edge label is split into multiple edge sub-graph files according to edge attributes. The edge sub-graph files include topology sub-files and attribute sub-files. Vertex data under the same vertex label is split into multiple vertex sub-graph files according to vertex attributes. The graph pattern matching clause is divided into multiple query segments. Each query segment contains labeled graph patterns and data operations. The labeled graph patterns in each query segment are rewritten according to vertex labels and edge labels, and the data operations are rewritten according to the attributes involved, into subqueries of the corresponding subgraph files in each cloud object storage service. A pre-set greedy access method is used to select the data access method that satisfies the query latency constraint and has the lowest cloud storage access cost for each subquery, generating the optimal economic cost query plan. The system receives the optimal economic cost query plan, and based on the cache hit status and data access method of the data required by each subquery in the optimal economic cost query plan, it decomposes each subquery into a combination of multiple query methods for fine-grained data fragments, dynamically generates a hybrid query plan, receives attribute data retrieved from the data storage module and caches it locally, determines the cache benefit of each data to be cached based on the comprehensive cache benefit model, and dynamically executes cost-priority cache replacement; the comprehensive cache benefit model normalizes the time cost and economic cost of network transmission, data scanning and computing layer operation when each cached data is retrieved from the data storage module into a benefit value, so as to maximize the economic benefit of caching; The system receives a mixed query plan and executes it segment by segment in the order of the query segments. For each subquery in the current query segment, according to the data source and data access method in the mixed query plan, it first retrieves the topology structure subfile, calls the corresponding operator to perform graph pattern matching evaluation, performs structure and identifier evaluation on the graph pattern operation result, and finally retrieves the attribute value in the attribute subfile according to the identifier in the operation result, calls the corresponding operator to perform data operation evaluation on the attribute value, and merges the query segments to generate the query answer.

[0008] The aforementioned cloud-native attribute graph data storage system, method, and computer equipment, by splitting attribute graph data into topology structure sub-files and attribute sub-files according to edge and vertex attributes, can separate topology calculation from attribute reading, retrieve required data on demand, and avoid invalid data transmission caused by reading the entire subgraph. The query is divided into multiple query segments by graph pattern matching clauses and rewritten into subqueries of corresponding subgraph files according to labels and attributes, enabling the query to be accurately mapped to the target file in cloud object storage. The greedy access method selection approach can select the data access method with the lowest cloud storage access cost while meeting query latency constraints. Subqueries are decomposed into fine-grained data fragments based on cache hits and data access methods, and a hybrid query plan is dynamically generated, enabling parallel execution of local processing of cached data and remote retrieval of uncached data. Based on a comprehensive cache benefit model, cache benefits are determined and cost-priority cache replacement is performed, prioritizing the retention of data fragments with high retrieval costs and reducing the cloud storage overhead of repeated accesses. The topology structure sub-file is retrieved first to complete graph pattern matching and structure identifier evaluation, and then attribute values ​​are retrieved based on the identifiers for data operation evaluation, delaying cross-layer transmission of attribute data and reading only necessary attribute values. The embodiments of the present invention can balance query performance and economic cost in a cloud object storage environment, and are suitable for low-cost query and analysis scenarios of large-scale attribute graphs. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the structure of a cloud-native attribute graph data storage system in a specific embodiment; Figure 2 This is a schematic diagram of an example label partition in one embodiment; Figure 3 This is a schematic diagram illustrating secondary partitioning of an example attribute in one embodiment; Figure 4 This is a schematic diagram of an example vertex data index in one embodiment; Figure 5 This is a schematic diagram of an example edge data index in one embodiment; Figure 6 This is a schematic diagram of a query process example in one embodiment; Figure 7 This provides average performance and cost data for each system under full workload in one embodiment, where... Figure 7 (a) is a diagram comparing the storage costs of each system. Figure 7 (b) is a diagram showing the query performance comparison of each system. Figure 7 (c) is a diagram comparing the query costs of each system; Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0011] In one embodiment, a cloud-native attribute graph data storage system is provided, comprising: The data storage module is used to store attribute graph data in the cloud object storage service. The attribute graph data includes edge data and vertex data. Edge data under the same edge label is split into multiple edge sub-graph files according to edge attributes. The edge sub-graph files include topology sub-files and attribute sub-files. Vertex data under the same vertex label is split into multiple vertex sub-graph files according to vertex attributes. The query planning module is used to divide the graph pattern matching clause into multiple query segments. Each query segment contains labeled graph patterns and data operations. The labeled graph patterns in each query segment are rewritten according to vertex labels and edge labels, and the data operations are rewritten according to the attributes involved, into subqueries of the corresponding subgraph files in each cloud object storage service. A pre-set greedy access method is used to select the data access method that meets the query latency constraints and has the lowest cloud storage access cost for each subquery, generating the most cost-effective query plan. The cache manager receives the optimal cost-performance query plan. Based on the cache hit rate and data access method of each subquery in the optimal cost-performance query plan, it breaks down each subquery into a combination of multiple query methods for fine-grained data fragments, dynamically generating a hybrid query plan. The cache manager also receives attribute data retrieved from the data storage module by the query execution module and caches it locally. Based on the comprehensive cache benefit model, it determines the cache benefit of each data to be cached and dynamically performs cost-priority cache replacement. The comprehensive cache benefit model normalizes the time and economic costs of network transmission, data scanning, and computation layer operations when retrieving cached data from the data storage module into a benefit value to maximize the economic benefit of caching. The query execution module receives the mixed query plan and executes it segment by segment in sequence. For each subquery in the current query segment, according to the data source and data access method in the mixed query plan, it first retrieves the topology subfile, calls the corresponding operator to perform graph pattern matching evaluation, performs structure and identifier evaluation on the graph pattern evaluation result, and finally retrieves the attribute value in the attribute subfile according to the identifier in the evaluation result, calls the corresponding operator to perform data operation evaluation on the attribute value, and merges the query segments to generate the query answer.

[0012] In the aforementioned cloud-native attribute graph data storage system, by splitting attribute graph data into topology structure sub-files and attribute sub-files according to edge and vertex attributes, topology calculation and attribute reading can be separated. Data can be retrieved on demand, avoiding invalid data transmission caused by reading the entire subgraph. The query is divided into multiple query segments by graph pattern matching clauses and rewritten into subqueries in the corresponding subgraph files according to labels and attributes. This allows the query to be accurately mapped to the target file in cloud object storage. A greedy access method is used to select the data access method with the lowest cloud storage access cost while meeting query latency constraints. Subqueries are decomposed into fine-grained data fragments based on cache hits and data access methods, and a hybrid query plan is dynamically generated. This allows local processing of cached data and remote retrieval of uncached data to be executed in parallel. A cache benefit model is used to determine cache benefit and perform cost-priority cache replacement, prioritizing the retention of data fragments with high retrieval costs and reducing the cloud storage overhead of repeated accesses. The topology structure sub-file is retrieved first to complete graph pattern matching and structure identifier evaluation, and then attribute values ​​are retrieved based on the identifiers for data operation evaluation. This postpones cross-layer transmission of attribute data, reading only the necessary attribute values. The embodiments of the present invention can balance query performance and economic cost in a cloud object storage environment, and are suitable for low-cost query and analysis scenarios of large-scale attribute graphs.

[0013] In one specific embodiment, such as Figure 1The diagram illustrates the system architecture of this invention. The system employs a storage-compute separation architecture. The storage layer is responsible for storing attribute graphs and indexes in the cloud, while the compute layer handles user-submitted Cypher queries. The storage layer uses object storage (such as Amazon S3), which offers advantages such as low cost and minimal performance loss under mixed workloads. The compute layer is built on compute servers (such as Amazon EC2), a service model widely adopted by many cloud-native applications. It is worth noting that the system is not limited to a specific cloud platform; it can be deployed on any platform that supports scalable object storage services like Amazon S3 and EC2 compute services.

[0014] At the storage layer, the core is to design an attribute graph storage structure in the object files to ensure the correctness of path reasoning during query processing. Specifically, data is divided into two main categories: edge data and attribute data, stored in two independent folders within the same object storage bucket. Each category of data is further partitioned based on labels defined in the attribute graph, with each label corresponding to an object in the corresponding folder. Using the label as the key, users can efficiently retrieve the path or attribute data corresponding to that label. In addition, a Byte Position Estimating (BPE) index is designed: the label index is used to efficiently construct data request plans during query processing, while the BPE index is used to further optimize data access performance.

[0015] At the compute layer, AWS EC2 services are used to handle query processing logic. When executing a user-submitted Cypher query, the query is first parsed into a labeled graph pattern. Based on the vertices and edges in the pattern, corresponding data is retrieved from the attribute graph edge files and attribute files stored in object storage. Since different access methods of cloud object storage services (such as GET and SELECT operations) and the number of files accessed during query execution can lead to differences in economic costs and performance characteristics, an optimized query structure is adopted to minimize monetary costs while meeting performance requirements. Regarding the caching mechanism, hot input data is stored in the compute layer's local cache (memory or disk) for high-speed I / O. The logical query plan output by the optimizer is received, and the query, which has already been parsed into accesses to vertex and edge object files, is further decomposed into two paths: cached computation and remote query, based on the current cached content. Furthermore, cost-based decisions determine which data is stored in the compute node cache, achieving optimal acceleration at minimal cost.

[0016] In one embodiment, the topology sub-file, attribute sub-file, and vertex sub-graph file are all stored in rows; each row in the topology sub-file is arranged in the order of edge identifier, end vertex identifier, and start vertex identifier; each row in the attribute sub-file is arranged in the order of edge identifier and edge attribute value; and each row in the vertex sub-graph file is arranged in the order of vertex identifier and vertex attribute value.

[0017] In this embodiment, data is stored in object storage as object files. Data retrieval requires either downloading the entire object file via an retrieval operation or performing a full scan of the object file via a selection operation; random reading of specific lines within the file is not possible. Furthermore, retrieval operations are charged based on the amount of data transmitted, while selection operations are charged based on the amount of data scanned. The amount of data transmitted and scanned in a single request directly determines the economic and time costs of the query. Traditional attribute graph storage is typically based on an adjacency list model, where a single file records all path information of the graph. However, in object storage scenarios, this storage method has extremely low query processing efficiency: queries often require multiple accesses to the data file, and the adjacency list format means that each access requires scanning the entire graph, resulting in decreased query performance and increased economic costs.

[0018] This invention performs a two-level partitioning of attribute graph data: the first level partitions vertices and edges by label, aggregating elements with the same label into independent objects for storage. The second level performs a secondary partitioning of the data corresponding to each label by attribute. During query processing, the pattern matching stage only needs to access the topology sub-file to obtain vertex identifiers and connection relationships, without reading any attribute data; the attribute filtering or projection stage only needs to access the attribute sub-files corresponding to the relevant attributes, without downloading the complete file containing all attributes, minimizing invalid data transmission and scanning in the cloud object storage environment, thereby reducing query costs and improving retrieval speed. In addition, the endpoint vertex identifiers in the topology sub-file are arranged before the starting vertex identifiers. This field order is suitable for common incoming edge traversal scenarios in graph queries: when the query retrieves its incoming edges from the target vertex in reverse, the endpoint vertex identifier field at the beginning of each row can be directly read to perform filtering and connection, reducing field parsing overhead.

[0019] In the first-level partitioning, a large attribute graph is split into multiple smaller subgraphs (called partitions), each stored as an independent object. Since labels in the attribute graph represent entity categories and are typically limited in number, the vertices and edges of the graph are partitioned based on their labels. This label partitioning aggregates elements with the same label into a single data file, stored as an object file, and named after the corresponding label; this is called the label file.

[0020] In the edge data file, each edge is stored line by line, in the following format: [Edge ID (EID) - End Vertex ID (INV) - Start Vertex ID (OUTV) - Attribute 1 - Attribute 2 - ... - Attribute n]; In the vertex data file, each vertex is stored line by line, in the following format: [Vertex ID (VID) - Attribute 1 - Attribute 2 - ... - Attribute n]; Tag partitioning breaks down the original adjacency list structure into independent files organized by tags. However, since vertices and edges in an attribute graph are described by multiple attributes, especially vertices which typically contain a large number of attributes, each tag-based file remains relatively large. Furthermore, for cloud object storage, the data transfer volume in a single request has a far greater impact on performance than the number of requests. Also, query processing usually only requires IDs or partial attributes, rather than complete vertex or edge data. Therefore, partitioning the tag files by attribute to reduce the size of individual files can improve data retrieval speed and reduce costs.

[0021] like Figure 2 The example label partition diagram shown is as follows: Figure 3 The example attribute secondary partitioning diagram shown below illustrates that s3object is the object file identifier in the cloud object storage service; knows is the edge label, representing the recognition relationship between vertices; study_At is the edge label, representing the reading relationship between vertices; Person is the vertex label, representing a person; organization is the vertex label, representing an organization. EID is the edge identifier, INV is the endpoint vertex identifier, OUTV is the starting vertex identifier, VID is the vertex identifier; classYear is the year of enrollment, firstName is the first name, lastName is the last name, gender is the gender, creationDate is the creation date, type is the type, and name is the name. Figure 3 Showing the Figure 2 The attribute partitioning results for the edges and vertices of the `knows` and `organization` tags are as follows. The edge file labeled `knows` is split into two files: `knows.ID` and `knows.creationDate`, which contain the edge ID, endpoint ID, and origin ID, as well as the edge ID and creation date, respectively. The vertex file labeled `organization` is split into two files: `organization.type` and `organization.name`, which contain the vertex ID and type, as well as the vertex ID and name, respectively.

[0022] Specifically: 1. For vertex data, each file generated after attribute partitioning contains only an identifier (ID) and a reserved attribute, in the format [Vertex ID (VID) - Attribute 1].

[0023] 2. For edge data, since an edge connects two vertices, the starting vertex and the ending vertex of the edge are considered as a group. Therefore, after attribute partitioning, the edge data file is divided into two types of structures: [edge ID (EID) - ending vertex ID (INV) - starting vertex ID (OUTV)] and [edge ID (EID) - attribute 1].

[0024] In one embodiment, in the data storage module, for sub-graph files whose data selectively meets preset conditions, a corresponding byte position estimation index is constructed. The byte position estimation index records the byte offset range of the data value in the corresponding sub-graph file. The byte position estimation index adds index entries incrementally from high to low according to the data selectivity of the attribute. When the sum of the current index size and the maximum byte offset range covered by the index is less than the product of the threshold parameter and the byte size of the corresponding sub-graph file, the current index entry is retained and addition continues; otherwise, the addition of index entries is stopped.

[0025] In this embodiment, the cloud object storage service provides two basic data access methods: GET and SELECT. This invention designs an index structure called the Byte Position Estimating Index (BPE index), which, combined with the GET and SELECT operators, achieves a balance between cost and performance.

[0026] like Figure 4 The example vertex data index diagram shown illustrates an example vertex index file for the Person.gender attribute. Here, s3object is the object file identifier in the cloud object storage service, VID is the vertex identifier, and gender is the gender attribute. The total size of the Person-gender object file is 80 characters (actual storage is in bytes); the value "attribute value 1", such as male, appears in records 1 and 2, starting at position 0 and ending at position 28; the value "attribute value 2", such as female, appears in records 3 and 4, starting at position 28 and ending at position 60.

[0027] like Figure 5 The example shown is a schematic diagram of edge data indexing, where EID is the edge identifier, INV is the endpoint vertex identifier, and OUTV is the starting vertex identifier. Figure 5 An example edge index file with the `studyAt.ID` tag is provided, and the design concept is similar to that of the vertex index. INV value 17 appears in records 2 and 4, and INV value 18 appears in records 1 and 3; in this case, secondary filtering is required on the computing server to ensure the correctness of the final result. OUTV values ​​each appear only once; creating index files for these values ​​would reduce indexing efficiency, therefore, no record index files are created for these values.

[0028] There are six ways to access data in object storage: direct GET (GG), direct SELECT (DS), index GET (GG), index SELECT (GS), SELECT index GET (SG), and SELECT index SELECT (SS).

[0029] In one embodiment, dividing the graph pattern matching clause into multiple query segments includes: parsing the attribute graph query into labeled graph patterns and data operations; the labeled graph patterns correspond to graph pattern matching clauses, and the data operations correspond to filtering, passing, and output clauses; dividing the query into multiple query segments according to the number of labeled graph pattern matching clauses, with each query segment including at least one labeled graph pattern and at least one data operation.

[0030] In this embodiment, since the attribute graph is decomposed into multiple object files for storage, the purpose of query decomposition is to identify the specific object file that needs to be accessed for a given query.

[0031] The input query is first split into two parts: labeled graph pattern and data operations. The labeled graph pattern corresponds to the MATCH clause in a Cypher statement, while the data operations part consists of clauses such as WHERE, WITH, RETURN, ORDER BY, and LIMIT. This approach abandons the traditional use of WITH as a connector in Cypher queries, treating it as a data operation similar to WHERE.

[0032] Therefore, the entire query is divided into multiple query segments based on the MATCH clause, represented as S={s1,s2,...,st}. Each segment includes several labeled graph patterns L={l1,l2,...,lt} and data operations O={o1,o2,...,ot}.

[0033] In one embodiment, rewriting the labeled graph patterns in each query segment into sub-queries corresponding to sub-graph files in each cloud object storage service according to vertex and edge labels and the attributes involved in data operations includes: mapping vertices and edges with the same label to the same sub-graph file in the data storage module based on the labels of vertices and edges in the labeled graph patterns and the attributes involved in data operations; and generating sub-queries containing sub-graph file identifiers and target attribute values ​​according to the labels and attributes corresponding to each sub-graph file.

[0034] In this embodiment, the labeled graph pattern is further decomposed into a set of object file accesses based on edge and vertex labels. Since the attribute graph uses label-based partitioning, all vertices and edges with the same label are stored in the same partition. Therefore, the pattern portion of the input query is decomposed into object data file accesses for vertices and edges with the same label. Then, utilizing Cypher's linear execution semantics, the decomposed data operations can be applied sequentially to the retrieved data, and the results are passed to the next segment.

[0035] Formally, given a query Q, it is decomposed into a set of fragments S={s1,s2,...,st}, where each fragment si contains multiple labeled graph patterns and data operations, i.e., si={l1,o1,...,lt,ot}; further decomposed into a set of subqueries Di={q1,q2,...,qt}, where each subquery qi corresponds to an attribute data under a specific label.

[0036] like Figure 6 As shown, query Q is divided into two segments S={s1,s2}. The first segment s1={l1,o1,o2} corresponds to D1={q1,q2} and contains two attributes: Person.ID and knows.ID. The second segment s2={l1,l2,o1,o2,o3} corresponds to D2={q1,q2,q3,q4} since Person.ID has already been calculated in the first segment and passed to the second segment. It contains four attributes: isLocatedIn.ID, Place.name, studyAt.ID, and University.name, which represent the edge identifier in the relation, the location name, the edge identifier in the relation, and the university name, respectively.

[0037] In one embodiment, a pre-set greedy access method is used to select the data access method that satisfies the query latency constraint and has the lowest cloud storage access cost for each subquery, generating an economically optimal query plan. This includes: selecting the optimal data access method for each subquery from multiple preset data access methods, including: directly obtaining the subgraph file, directly executing the selection query on the subgraph file, obtaining the byte offset range of the target data value by estimating the index through byte position and then obtaining the data by the range, and obtaining the byte offset range of the target data value by estimating the index through byte position and then executing the selection query by the range; calculating the execution time and cloud storage access cost of each data access method based on the size of the subgraph file corresponding to the subquery, the amount of returned data, and the index size; iterating through each data access method in descending order of unit time efficiency, checking whether the cumulative execution time still satisfies the query latency constraint after each selection, if it does, retaining the current data access method and marking the corresponding subquery as selected, otherwise skipping the current data access method, until all subqueries have selected a data access method, and outputting the combination of data access methods with the lowest cumulative cloud storage access cost as the economically optimal query plan.

[0038] In this embodiment, after query decomposition, the query is split into multiple labeled graph patterns containing several object file subqueries and data operations. Each subquery can be solved using a single access method. Given a fragment decomposition Di={q1,q2,...,qt}, a suitable access method needs to be determined for each decomposed subquery.

[0039] For each subquery qi in si, one of the aforementioned six access methods is used. To record this decision, six Boolean variables are defined: x^{DG}(qi), x^{DS}(qi), x^{GG}(qi), x^{GS}(qi), x^{SG}(qi), and x^{SS}(qi), each corresponding to one access method, and exactly one of these variables takes the value 1. The formula is expressed as: ; To accurately quantify the overhead of different access methods, this paper constructs a unified time cost model and economic cost model, covering the full-dimensional overhead of six data access methods. For any subquery qi, its total query time consists of three parts: index access time, data reading time, and computation layer processing time. The total economic cost corresponds to the cloud service's read billing and computation billing: Time cost: ; Economic cost: ; Based on cloud service provider billing rules and performance test data, significant differences exist in the cost characteristics of different access methods: GET operations are billed only based on the amount of data transmitted, with no scanning overhead; SELECT operations require a full scan of the target file, resulting in higher billing costs, but can reduce computational logic and local computation time. In contrast, range access based on BPE indexes can precisely narrow the data scanning and transmission range, achieving optimization in both time and cost.

[0040] This paper defines the optimization objective of subqueries as: minimizing the total economic cost of a single query while satisfying the user's preset query latency constraints. This optimization problem can be formally described as: given the set of access methods for all subqueries, the time and cost parameters of each method, and the global latency threshold, find the optimal combination of access methods that minimizes the total cost.

[0041] Since this combinatorial optimization problem is NP-hard, it is impossible to find the global optimum in polynomial time. To adapt to the efficiency requirements of real-time query scenarios, this paper designs a greedy heuristic algorithm that approximates the global optimum through iterative local optima, significantly reducing the overall query cost while ensuring compliance with query latency requirements.

[0042] This article defines the unit time efficiency index This is used to quantify the overall benefits of different data access methods, and its calculation formula is as follows: ; Algorithm 1 presents a greedy heuristic algorithm for solving the access method determination problem. Given a Cypher query... With time constraints The goal of this algorithm is to find a set of access schemes. This reduces the economic cost of the query. Minimize. The solution process mainly includes the following steps: 1. All candidate access methods According to the corresponding Sort the values ​​in descending order and construct a priority queue; 2. Select sequentially starting from the head of the priority queue, while ensuring that each subquery... Select only one The corresponding access method; 3. After each selection is completed, verify whether the total time cost is satisfied. If the validation fails, continue selecting elements from the priority queue until all subqueries have selected the corresponding element. ; 4. Based on the selected and their corresponding access methods To obtain the complete solution .

[0043] Table 1. Pseudocode for Algorithm 1

[0044] This greedy heuristic algorithm abandons the high-complexity solution method of global exhaustive traversal, and the algorithm's time complexity is only [missing information]. (in Number of subqueries It offers six fixed data access methods, has extremely low computational overhead, and can adapt to real-time optimization scenarios for large-scale graph queries.

[0045] Based on the above query decomposition, access method modeling, and optimization algorithms, the system's complete query execution chain forms a closed loop: The first step is query parsing and splitting: receiving the user's Cypher query, parsing the syntax structure, splitting it into two major modules: labeled graph pattern and data operation, and further dividing it into multiple independently executable query fragments and subqueries.

[0046] The second step is access strategy optimization: Based on the BPE index metadata and cost model, a greedy heuristic optimization algorithm is used to match the optimal data access method for each subquery, generating a low-cost, low-latency query execution plan.

[0047] The third step is hierarchical query evaluation: priority is given to performing pattern evaluation, which completes core topology calculations such as vertex connection and path matching using edge file data; then ID evaluation and structure evaluation are performed to complete data filtering, deduplication, and structure reconstruction; finally, attribute evaluation is triggered on demand, and vertex attribute data is loaded through a delayed reading strategy.

[0048] The fourth step is result aggregation and output: merge the intermediate results of all subqueries, perform the final projection, aggregation, sorting and other operations, generate standardized query results and return them to the user.

[0049] In one embodiment, receiving attribute data retrieved from the data storage module by the query execution module and caching it locally, determining the caching benefit of each piece of data to be cached based on a comprehensive caching benefit model, and dynamically performing cost-priority cache replacement includes: storing the attribute data retrieved from the data storage module by the query execution module in key-value pairs in the cloud storage volume of the compute node, where the key is the identifier of the data segment and the value is the corresponding attribute value; determining the network transmission overhead of the data segment based on the data size of the data segment and the network bandwidth from the compute node to the cloud storage service; determining the data scanning overhead of the data segment by estimating the index coverage based on the byte position of the subgraph file corresponding to the data segment, where a higher index coverage results in a lower data scanning overhead; and determining the computational cost of the data segment based on the number of operators applied to the data segment. The cache benefit is calculated by taking the sum of network transmission overhead, data scanning overhead, and computational layer overhead of the data segment as the cache benefit, and constructing a comprehensive cache benefit model with the goal of maximizing the economic benefit of caching. The cache benefit of each data segment to be cached is calculated using the comprehensive cache benefit model and arranged from high to low cache benefit. When a data segment to be cached arrives, if the remaining cache space can accommodate the current data segment to be cached, it is loaded into the local cache. If the remaining cache space is insufficient, the cache benefit of the current data segment to be cached is compared with the cache benefit of the data segment with the lowest benefit in the local cache using the comprehensive cache benefit model. If the cache benefit of the current data segment to be cached is higher, the data segment with the lowest benefit in the local cache is evicted and the current data segment to be cached is loaded; otherwise, it is not loaded.

[0050] In this embodiment, the caching mechanism designed in this invention includes two main modules: hybrid query plan generation and cache management. The hybrid query plan generation receives the logical query plan output by the query optimizer, combines it with the current cached content, further breaks it down, converts it into a cache-integrated query plan, and then pushes it to the query processor for computation. Cache management manages the data cached in the computation layer and its data format based on a cache replacement strategy framework.

[0051] Conventional cached query plans read all data from the compute node's local cache. When a cache miss occurs, the data from the storage layer is typically loaded into the cache or merged directly before executing the query. However, object storage cannot precisely manipulate data; even considering BPE indexes, it can only locate byte positions by narrowing the scan range. Therefore, hybrid query plans split the query into multiple parts for parallel execution. This accelerates queries while simultaneously resolving the cache miss issue.

[0052] The system uses a heuristic approach to generate separable query plans. After obtaining an initial plan from the query optimizer, the execution of operators is split based on the current cached content. Specifically, as long as the accessed data is cached, operators on segments are always processed based on the cached segments. Otherwise, segments are processed using the BPE index pattern whenever possible. If neither approach is feasible, a raw GET or SELECT request pattern is used, loading the missing segments from the storage layer. It is important to note that for a given input object file, only one hybrid query plan can be generated. In general, the hybrid query plan is decomposed based on the following two principles: local processing of cached data is more efficient than reading from the storage layer; and processing based on the BPE index is more efficient than retrieving all accessed segments and then processing them locally.

[0053] Therefore, in this system, access to data fragments in an object file of a storage layer can be extended to include the following three modes: Local processing based on cache: Scan all data fragments in the cache, extract all matching data fragments, and then perform direct computation; Local processing based on cache + BPE index query: Missing data is retrieved and processed in parallel using the BPE index; Local processing based on cache + GET / SELCT direct query: Missing data is accessed and retrieved in parallel using the GET / SELCT method. By further integrating the access methods in the system query optimizer, the original 6 access methods have been expanded to 15. This refined separation approach can fully utilize cache resources under cloud object storage conditions, effectively improving performance.

[0054] The following example query illustrates the workflow of a mixed query. Figure 6 Take fragment 2, which contains tag pattern 2 and data operation 2, as an example. Assume that fragments U1 and U2 of University.name in the example are both cached; fragment P1 of Place.name is cached, P2 is not cached but has a corresponding record in BPE; fragment F1 of Person.FirstName is cached, while F2 is neither cached nor has a corresponding record in BPE.

[0055] Get and calculate University.name: Since both shards U1 and U2 are cached, after retrieving the data fragments from the cache, the U1 and U2 shards are merged locally and then filtered with the University.ID obtained from the MATCH statement to get the queried data; Retrieving and calculating Place.name: Since only the P1 shard is cached, the query is split into two parts. The first part filters the P1 shard directly using Place.ID locally to obtain partial results. Since the content in the P2 shard has corresponding records in BPE, the other part of the query first selects the BPE-based access method according to the optimization algorithm, then matches the closest byte range based on the maximum and minimum values ​​of the remaining Place.ID, and then executes SELECT or GET to perform the query and subsequent calculations. After completion, it is merged with the calculation results of P1 to remove duplicates and obtain the final result.

[0056] Retrieving and calculating Person.FirstName: Since only the F1 shard is cached, the query is split into two parts. The first part is filtered locally using Friend.ID (named Friend in MATCH) in the F1 shard to obtain partial results. Since the content in the F2 shard has no corresponding record in BPE, the query and subsequent calculation are performed based on SELECT or GET according to the optimization algorithm. After completion, the result is merged with the calculation result of F1 to remove duplicates and obtain the final result.

[0057] Cache management is responsible for deciding which table segments should be fetched into the cache, which segments should be evicted, and when cache replacement should occur. In traditional caching systems, cache misses require loading data from storage into the cache, and the default cache segment size is a fixed value, meaning the cost of each cache miss is the same. However, in this system, we can choose to use SELECT pushdown calculation or data retrieval based on BPE. Therefore, some segments may be more suitable for pushdown than others, which affects the benefits of caching. In other words, segments that cannot be queried via BPE or whose attributes result in large scans (which would be costly using SELECT) should be considered for caching with higher weight; while segments that can be retrieved at low cost can be cached with lower weight—the additional benefits of caching compared to pushdown are relatively small.

[0058] Ideally, the cache should contain only the fragments that will benefit most from residing there. The benefit value `s.benefit` represents the gain that fragment `s` can obtain by residing in the cache. The main difference between different caching strategies lies in how the benefit value is determined. For example, for LRU and LFU strategies, the benefit value represents recentity and frequency, respectively. In this system, a greedy algorithm is used to define cache writes and evictions.

[0059] Algorithm 2 demonstrates how the reward-based caching framework works in the system. When a set of fragments is accessed, its reward value is updated (lines 1–2) to reflect the new access. Then, for each missing fragment s, it is cached in descending order of reward, if there is still space in the cache (lines 3–6). If space is insufficient, space is freed up by evicting the fragment with the lowest reward (i.e., fragment t) from the cache until the reward value of t is higher than the reward value of the newly updated fragment (lines 8–12), at which point the caching algorithm stops loading new fragments.

[0060] Table 2 Pseudocode for Algorithm 2

[0061] Since this system is designed to minimize the cost of query usage, caching data at the computation layer will undoubtedly increase the overall operating cost of the system. Although the caching framework and strategy adopt methods to improve the cost-benefit ratio to offset this, it is still necessary to reduce the cost of cached data storage as much as possible.

[0062] Based on the above considerations, and because memory is more expensive, cached data is stored in cloud storage volumes at the computing layer. Following the 2 / 8 access characteristic of real business scenarios, the system only retains 20% of the hot data at the computing layer.

[0063] Furthermore, the data stored in the cache is hashed into 32-bit numeric data and stored in a key-value format to avoid long strings appearing in the attribute data. Specifically, the key is a composite key of the rate of return and the primary key of the data, and the value is the value of the attribute. Taking Person.FirstName as an example, it can be expressed as: ({rate of return, Person.ID}-{FirstName}).

[0064] Traditional LFU (Least Frequently Used) or LRU (Least Recently Used) strategies prioritize caching based on access frequency or order, while other caching strategies calculate based on the time cost of queries. However, in scenarios optimized for usage costs, frequently accessed fragments may incur greater economic costs due to the inability to use BPE (Browser Object Provider) or the use of costly access methods. Furthermore, since BPE data is stored in cloud object storage, directly updating BPE is inefficient. Therefore, caching strategies use query cost as a weight, with the weight determined by the economic cost of querying the fragment. In other words, the higher the query cost, the greater the benefit of caching the fragment, and thus the higher its weight.

[0065] Since different data access methods and whether the data is in the BPE index will affect the query cost, the system adopts an estimation method that takes into account the BPE factor. That is, after the query is completed, and it is known whether the data is in the BPE index, the system uses the time and economic cost model built by the calculation layer to define the upper bound of the economic cost of the SELECT access method with the highest cost, and then estimates the query cost of the data segment to calculate the weight. The specific calculation method is as follows.

[0066] All formulas are calculated for a single data segment s, with the following common parameters: size(s): Total size of the original data fragment; BW: Hardware bandwidth of the corresponding stage (default is a constant value); C: Computational cost per unit data size, default is a constant value under the same hardware configuration; D: Total hardware cost of the computation and storage layers per unit time; Total cache weight The higher the weight, the higher the priority of the fragment cache. ; Network transmission cost : ; Selectivity(s): The percentage of data size in the total file size of fragment s after filtering by the BPE index. Lower selectivity results in better filtering, less network data transfer, shorter processing time, and lower caching benefit for that fragment. If there is no corresponding record in the BPE index, it represents the size of the data itself. size(s): The total size of the original data fragment; Network bandwidth between compute nodes and cloud storage; Data scanning cost : ; : Data scanning cost for SELECT access method, which is 1 if it is a GET request.

[0067] Because of the BPE index, the scanning range of the original data can be limited, so the scanning range is the size after the index is filtered.

[0068] Computational layer computational cost : ; The number of operators that the current query is applying to this segment (e.g., the WHERE clause contains multiple conditions).

[0069] In one embodiment, firstly, the topology sub-file is retrieved, and the corresponding operator is called to perform graph pattern matching evaluation operation, including: obtaining the endpoint vertex identifier and starting vertex identifier of the edge from the topology sub-file; using the set of constant vertex identifiers specified in the subquery as the filtering condition, filtering the endpoint vertex identifier or starting vertex identifier of the edge connecting constant vertices in the topology sub-file to obtain a candidate edge set; performing equi-join on the candidate edge sets corresponding to different edge labels to obtain an identifier set that meets the query path requirements.

[0070] In this embodiment, the evaluation operators serve the query evaluation process and are divided into two main categories: pattern evaluation and data operation evaluation. The purpose of pattern evaluation is to solve for pattern matching results on paths composed of multiple labeled graph patterns. The basic idea of ​​pattern evaluation is to perform a connection operation on the identifiers of the endpoint and starting vertices of adjacent edges in the query clause to obtain a result set that satisfies the specified path requirements. Based on the path structure defined in the query, pattern evaluation can be divided into two categories: basic pattern evaluation and variable-length path evaluation.

[0071] The first step in the basic schema evaluation process is to read the edge file corresponding to a given label from object storage and obtain the INV and OUTV fields. Then, for vertices with constant attribute values, the INV or OUTV fields of the edges connecting these vertices are filtered within the schema. Finally, a join operation is performed on the INV or OUTV fields of edges with different labels to obtain the identifiers (IDs) of the specified vertices in the entire query path.

[0072] Variable-length path evaluation includes: a) Target vertex evaluation This paper extends the Breadth-First Search (BFS) algorithm. It maintains a visited set V, and before each BFS connection expansion, it compares the visited vertices with V to exclude those that have already been connected, thus reducing the computational cost of each connection.

[0073] Specifically, before the iterative connection process begins, the system initializes an empty visited set V. In each iteration, before performing BFS connection expansion, the set of vertex IDs to be connected is matched with the vertex IDs in V, and connection operations are performed only on unmatched IDs. After the connection is completed, the successfully connected vertex IDs are inserted into V, marking the completion of this iteration. Then, based on the query conditions, the iteration terminates after a specified number of steps or when the target vertex is found. Finally, intermediate results are merged to complete the entire process.

[0074] b) Shortest path evaluation The shortest path query requires finding the shortest path between a specified source vertex and a target vertex within a variable number of steps and edges with specified labels. This paper directly adopts existing path evaluation techniques and uses a bidirectional breadth-first search to solve for the shortest path.

[0075] The shortest path evaluation first reads the OUTV and INV fields of the edge with the specified label, constructing a bidirectional adjacency structure, denoted as SG. The source vertex and target vertex specified in the query are used as the starting points of the forward search queue QF and the reverse search queue QB, respectively. The search process traverses SG, iteratively expanding QF and QB, prioritizing the expansion of the shorter queues, until an intersecting vertex is found. Finally, starting from the intersecting vertex, backtracking along QF and QB, and concatenating the two path segments yields the complete path.

[0076] Figure 6 Examples of pattern evaluation are shown. Basic pattern evaluation is used to solve the OPTIONALMATCH clause in labeled pattern 2, and shortest path evaluation is used to solve the MATCH clause in labeled pattern 1.

[0077] The OPTIONAL MATCH clause involves three types of vertices: Person, Place, and University, and two types of edges: studyAt and isLocatedIn. The Person vertices are a set of constants calculated from fragment 1. After reading the INV and OUTV fields of studyAt.ID and isLocatedIn.ID from object storage, the OUTV values ​​in studyAt.ID are filtered using the Person constants to generate new INV and OUTV tables. Finally, the INV values ​​in the new tables and the OUTV values ​​in isLocatedIn.ID are used as join keys to perform a join operation, resulting in a schema evaluation result containing Person.ID, University.ID, and City.ID.

[0078] The MATCH clause requires finding the shortest path between `p` and `friend` using the `knows` edge, where `p` and `friend` are constants in the query. First, the OUTV and INV fields of the `Person` vertex are read from the `knows.ID` edge, constructing a bidirectional adjacency structure `SG(knows)`. Vertex `p` serves as the starting point of the forward search queue `QF(p)`, and `friend` serves as the starting point of the reverse search queue `QB(friend)`. The expansion stops after a maximum of 3 iterations or when an intersecting vertex is encountered. Starting from the intersecting vertex, backtracking along `QF(p)` and `QB(friend)`, the two paths are joined to form a complete path; the number of vertices on the path is the distance required by the query.

[0079] In one embodiment, performing structure and identifier evaluation operations on the graph pattern operation results using corresponding operators, and finally retrieving attribute values ​​from the attribute subfile based on the identifiers in the operation results, and calling corresponding operators to perform data operation evaluation on the attribute values ​​includes: performing graph pattern structure verification locally at the computation layer based on the identifier set, and outputting a subset of identifiers that satisfy the structural constraints; performing intersection, union, and difference set operations on the identifier subsets, and outputting a target identifier set; retrieving corresponding attribute values ​​from the attribute subfile of the data storage module based on the edge identifiers and vertex identifiers in the target identifier set; performing filtering, projection, and aggregation operations on the retrieved attribute values, and outputting the data operation results of the current query segment.

[0080] In this embodiment, during the data operation evaluation stage, the system divides data operators into three categories: structure evaluation, identifier (ID) evaluation, and attribute evaluation.

[0081] Structure evaluation refers to the transformation of intermediate data representations passed to subsequent clauses or used as query output, mainly including operations such as projection, aggregation, and reconstruction of result tuples.

[0082] Since structure evaluation is used to pass intermediate results rather than generate output, the system only transforms the representation structure based on the results of other operators at the computation layer and fills in the calculated results. If it involves attributes that have not yet been calculated by other operators, they are temporarily set to null values, and the corresponding object file is read when subsequent attribute calculations require them. Therefore, when performing structure evaluation, the system does not need to interact with the storage layer, avoiding the most significant factor affecting performance.

[0083] Identifier evaluation involves operations on vertex identifiers (IDs), including sorting, filtering, and deduplication. Therefore, identifier evaluation operators also process the evaluation results of pattern evaluation or other operators. Since the ID is a unique identifier for all data and already exists in any computation, the system ignores other attributes in the intermediate result data when performing this operation and directly performs the corresponding computation on the ID. Therefore, it does not need to interact with the storage layer and can control the amount of data being processed.

[0084] Attribute evaluation encompasses operations that must evaluate or manipulate specific attribute values, such as predicate filtering, arithmetic evaluation, statistical aggregation, or logical evaluation of attribute fields. When executing this operator, the system first retrieves the object file containing the corresponding attribute from the cloud object storage service, then parses the operator in the query statement at the computation layer, and finally calls the corresponding function to perform calculations on the attribute data in the object file.

[0085] It can be observed that when the above query operators are called according to the execution order, the vertex attribute subgraph file is only read in the final stage. In the multi-hop query and variable-length query with the largest amount of computation, only the ID in the edge file is processed. Therefore, they together form a delayed reading mechanism.

[0086] Since most of the data in the attribute graph belongs to the attribute data of the vertices, as the query processing progresses, reading the attribute data later can avoid most of the invalid data. Therefore, the delayed reading mechanism can effectively accelerate the query calculation.

[0087] It can be observed that for all types of path and vertex queries in schema evaluation, the system only needs to operate on vertex identifiers (IDs). Therefore, even when filtering conditions involving edge attributes are involved, the computation can be completed using only the edge data. Based on this strategy, when executing operators included in schema evaluation, the system only needs to access the edge object file from cloud object storage, without needing to access other parts of the graph data.

[0088] Furthermore, the system's structure evaluation and identifier evaluation are entirely based on the results of the preorder pattern evaluation, utilizing vertex identifiers in the edge data. These operations do not require additional reading of vertex attribute data from object storage. For attribute evaluation, the system uses the results of identifier evaluation to locate the relevant vertices, then sends a request to object storage to retrieve the corresponding attribute records, and finally performs the necessary numerical or logical operations on the retrieved data to generate the final query result.

[0089] This mechanism effectively postpones the retrieval of vertex attribute data from object storage, only reading it when explicitly required for attribute evaluation. Since attribute values ​​constitute the largest portion of attribute graph data, this delayed retrieval strategy minimizes unnecessary data access, transmission, and memory computation in the early stages of querying, reducing memory overhead and evaluation time costs, and improving data transmission efficiency. Furthermore, the system utilizes the range functionality of GET and SELECT operations to selectively retrieve vertex attribute data from object storage. By limiting the data transmission range, the amount of data retrieved remotely is significantly reduced, thereby improving the overall efficiency of query evaluation.

[0090] Figure 6 An example of operator evaluation is shown. Data operation 1 includes a WHERE operator, which requires removing elements from the friend variable that are the same as p. This operation can be efficiently achieved by evaluating identifiers (IDs): based on the identifiers contained in p, the corresponding ID values ​​are filtered out from friend.

[0091] Data operation 2 includes WITH and RETURN clauses: the WITH clause is executed through structure evaluation, projecting the uni.name and uniCity.name properties into the intermediate variable unis; while the RETURN clause needs to access vertex property information to perform the final projection and generate specific output values, so it is executed through property evaluation.

[0092] The effectiveness of this invention was verified through experiments. All experiments were deployed on mainstream public cloud platforms, with Amazon S3 object storage for the storage layer and EC2 elastic computing servers for the compute layer, all configured as m.large 2C / 8GB. Comparative experiments selected three mainstream graph databases: the traditional distributed graph database Neo4j, JanusGraph, and the cloud-native graph database Amazon Neptune. Two standard datasets were used: the general public dataset LDBC SNB SF10 (social network benchmark dataset). The workload included mainstream query types such as basic attribute queries, graph pattern matching, variable-length path queries, and multi-condition aggregation queries. To eliminate randomness, the experiments were based on official benchmark queries, including 14 complex queries and 7 simple queries, iterated 100 times. The system performance was quantified from two dimensions: economic cost and query performance. The cost metric was the total cloud service overhead for all queries (including storage resource fees, query scanning fees, network request fees, and compute resource usage fees); the performance metric was the total response time for all queries, comprehensively evaluating the system's cost-effectiveness.

[0093] Figure 7 The data presents the average performance and cost of each system under full workload, among which... Figure 7 (a) is a diagram comparing the storage costs of each system. Figure 7 (b) is a diagram showing the query performance comparison of each system. Figure 7 (c) is a diagram comparing the query costs of different systems. In the diagram, A, B, C, and D represent our system, Neo4j, JanusGraph, and Neptune, respectively. Experimental results show that, under the premise of equivalent query performance, the average economic cost per query of our system is reduced by approximately 40% compared to traditional distributed graph databases, and by 2.2 times compared to currently available commercial cloud-native graph databases, demonstrating a significant cost advantage. In terms of query performance, our system has surpassed some mainstream attribute graph databases and is on par with the best performer among the compared databases.

[0094] In one embodiment, a cloud-native attribute graph data storage method is provided, including: In the cloud object storage service, attribute graph data is stored. Attribute graph data includes edge data and vertex data. Edge data under the same edge label is split into multiple edge sub-graph files according to edge attributes. Edge sub-graph files include topology sub-files and attribute sub-files. Vertex data under the same vertex label is split into multiple vertex sub-graph files according to vertex attributes. The graph pattern matching clause is divided into multiple query segments. Each query segment contains labeled graph patterns and data operations. The labeled graph patterns in each query segment are rewritten according to vertex labels and edge labels, and the data operations are rewritten according to the attributes involved, into subqueries of the corresponding subgraph files in each cloud object storage service. A pre-set greedy access method is used to select the data access method that satisfies the query latency constraint and has the lowest cloud storage access cost for each subquery, generating the optimal economic cost query plan. The system receives the optimal economic cost query plan. Based on the cache hit rate and data access method of each subquery in the optimal economic cost query plan, it decomposes each subquery into a combination of multiple query methods for fine-grained data fragments, dynamically generating a hybrid query plan. It receives attribute data retrieved from the data storage module and caches it locally. Based on the comprehensive cache benefit model, it determines the cache benefit of each data to be cached and dynamically executes cost-priority cache replacement. The comprehensive cache benefit model normalizes the time and economic costs of network transmission, data scanning, and computation layer operations when retrieving cached data from the data storage module into a benefit value to maximize the economic benefit of caching. Receive the mixed query plan and execute it segment by segment in the order of query segments. For each subquery in the current query segment, according to the data source and data access method in the mixed query plan, first retrieve the topology structure subfile, call the corresponding operator to perform graph pattern matching evaluation operation, perform structure and identifier evaluation operation on the graph pattern operation result, finally retrieve the attribute value in the attribute subfile according to the identifier in the operation result, call the corresponding operator to perform data operation evaluation on the attribute value, and merge the query segments to generate the query answer.

[0095] For specific limitations regarding cloud-native attribute graph data storage methods, please refer to the limitations of cloud-native attribute graph data storage systems mentioned above, which will not be repeated here. Each module in the aforementioned cloud-native attribute graph data storage system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0096] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a cloud-native attribute graph data storage method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0097] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0098] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A cloud-native attribute graph data storage system, characterized in that, The system includes: The data storage module is used to store attribute graph data in the cloud object storage service. The attribute graph data includes edge data and vertex data. Edge data under the same edge label is split into multiple edge sub-graph files according to edge attributes. The edge sub-graph files include topology sub-files and attribute sub-files. Vertex data under the same vertex label is split into multiple vertex sub-graph files according to vertex attributes. The query planning module is used to divide the graph pattern matching clause into multiple query segments. Each query segment contains labeled graph patterns and data operations. The labeled graph patterns in each query segment are rewritten according to vertex labels and edge labels, and the data operations are rewritten according to the attributes involved, into subqueries of the corresponding subgraph files in each cloud object storage service. A pre-set greedy access method is used to select the data access method that meets the query latency constraints and has the lowest cloud storage access cost for each subquery, generating the most cost-effective query plan. The cache manager receives the optimal cost query plan and, based on the cache hit rate and data access method of each subquery in the optimal cost query plan, decomposes each subquery into a combination of multiple query methods for fine-grained data fragments, dynamically generating a hybrid query plan. The cache manager also receives attribute data retrieved from the data storage module by the query execution module and caches it locally. Based on a comprehensive cache benefit model, it determines the cache benefit of each piece of data to be cached and dynamically performs cost-priority cache replacement. The comprehensive cache benefit model normalizes the time and economic costs of network transmission, data scanning, and computational layer operations when retrieving cached data from the data storage module into a benefit value to maximize the economic benefit of caching. The query execution module receives the mixed query plan and executes it segment by segment in sequence. For each subquery in the current query segment, according to the data source and data access method in the mixed query plan, it first retrieves the topology subfile, calls the corresponding operator to perform graph pattern matching evaluation, performs structure and identifier evaluation on the graph pattern evaluation result, and finally retrieves the attribute value in the attribute subfile according to the identifier in the evaluation result, calls the corresponding operator to perform data operation evaluation on the attribute value, and merges the query segments to generate the query answer.

2. The system according to claim 1, characterized in that, In the data storage module, for sub-map files where the data selectively meets preset conditions, a corresponding byte position estimation index is constructed. The byte position estimation index records the byte offset range of the data value in the corresponding sub-map file. The byte position estimation index adds index entries incrementally from high to low based on the data of the attribute. When the sum of the current index byte size and the maximum byte offset range covered by the index is less than the product of the threshold parameter and the corresponding subgraph file byte size, the current index entry is retained and addition continues; otherwise, the addition of index entries is stopped.

3. The system according to claim 1, characterized in that, The graph-based pattern matching clause is divided into multiple query segments, including: The attribute graph query is parsed into labeled graph patterns and data operations; the labeled graph patterns correspond to graph pattern matching clauses, and the data operations correspond to filtering, passing, and output clauses. The query is divided into multiple query segments based on the number of labeled graph pattern matching clauses. Each query segment includes at least one labeled graph pattern and at least one data operation.

4. The system according to claim 1, characterized in that, The step of rewriting the labeled graph patterns in each query segment according to vertex and edge labels, and according to the attributes involved in data operations, into sub-queries for the corresponding sub-graph files in each cloud object storage service includes: Based on the labels of vertices and edges in the labeled graph pattern, and the attributes involved in data operations, vertices and edges with the same label are mapped to the same subgraph file in the data storage module. Based on the labels and attributes corresponding to each subgraph file, generate a subquery containing the subgraph file identifier and the target attribute value.

5. The system according to claim 2, characterized in that, The method of selecting a pre-set greedy access method for each subquery, which satisfies the query latency constraint and has the lowest cloud storage access cost, generates an optimal cost-effective query plan, including: For each subquery, the optimal data access method is selected from a variety of preset data access methods, including: directly obtaining the subgraph file, directly executing the selection query on the subgraph file, obtaining the byte offset range of the target data value by estimating the index through byte position and then obtaining the data by the range, and obtaining the byte offset range of the target data value by estimating the index through byte position and then executing the selection query by the range. Based on the size of the subgraph file corresponding to the subquery, the amount of data returned, and the index size, calculate the execution time and cloud storage access cost for each data access method. Iterate through each data access method in descending order of efficiency per unit time. After each selection, check whether the cumulative execution time still meets the query latency constraint. If it does, retain the current data access method and mark the corresponding subquery as selected. Otherwise, skip the current data access method until all subqueries have selected a data access method. Output the combination of data access methods with the lowest cumulative cloud storage access cost as the optimal query plan in terms of economic cost.

6. The system according to claim 2, characterized in that, The receiving query execution module retrieves attribute data from the data storage module and caches it locally. Based on a comprehensive cache benefit model, it determines the cache benefit of each piece of data to be cached and dynamically performs cost-priority cache replacement, including: The attribute data retrieved from the data storage module by the query execution module is stored in the cloud storage volume of the compute node in the form of key-value pairs. The key is the identifier of the data segment, and the value is the corresponding attribute value. The network transmission overhead of the data segment is determined based on the data size of the data segment and the network bandwidth from the computing node to the cloud storage service. The data scan overhead of a data segment is determined by estimating the index coverage based on the byte position of the corresponding subgraph file. The higher the index coverage, the lower the data scan overhead. The computational overhead of a data segment is determined by the number of operators applied to it. The sum of network transmission overhead, data scanning overhead, and computational layer overhead for data fragments is used as the cache benefit, and a comprehensive cache benefit model is constructed with the goal of maximizing the economic benefits of caching. The cache benefit of each data segment to be cached is calculated using the cache comprehensive benefit model, and the segments are arranged from high to low cache benefit. When a data segment to be cached arrives, if the remaining space in the cache can accommodate the current data segment to be cached, then load it into the local cache; If the remaining cache space is insufficient, the cache benefit of the data segment to be cached is compared with that of the data segment with the lowest benefit in the local cache using the cache comprehensive benefit model. If the cache benefit of the data segment to be cached is higher, the data segment with the lowest benefit in the local cache is evicted and the data segment to be cached is loaded; otherwise, it is not loaded.

7. The system according to claim 1 or 2, characterized in that, The step of first retrieving the topology sub-file and then calling the corresponding operator to perform graph pattern matching evaluation includes: Obtain the endpoint vertex identifier and starting vertex identifier of the edge from the topology sub-file. Using the set of constant vertex identifiers specified in the subquery as the filtering condition, filter the endpoint vertex identifier or starting vertex identifier of the edge connecting the constant vertex in the topology sub-file to obtain a candidate edge set. Perform equi-joins on the candidate edge sets corresponding to different edge labels to obtain a set of labels that meet the query path requirements.

8. The system according to claim 1 or 2, characterized in that, The process of performing structure and identifier evaluation operations on the graph pattern operation results using corresponding operators, and finally retrieving attribute values ​​from the attribute subfile based on the identifiers in the operation results, and calling the corresponding operators to perform data operation evaluation on the attribute values ​​includes: Based on the identifier set, the graph pattern structure verification is performed locally at the computation layer, and the identifier subset that satisfies the structural constraints is output. Perform intersection, union, and difference operations on the subset of identifiers and output the target set of identifiers; Based on the edge and vertex identifiers in the target identifier set, retrieve the corresponding attribute values ​​from the attribute subfile of the data storage module; Perform filtering, projection, and aggregation operations on the retrieved attribute values, and output the data operation results for the current query segment.

9. A cloud-native attribute graph data storage method, characterized in that, The method includes: Attribute graph data is stored in the cloud object storage service. The attribute graph data includes edge data and vertex data. Edge data under the same edge label is split into multiple edge sub-graph files according to edge attributes. The edge sub-graph files include topology sub-files and attribute sub-files. Vertex data under the same vertex label is split into multiple vertex sub-graph files according to vertex attributes. The graph pattern matching clause is divided into multiple query segments. Each query segment contains labeled graph patterns and data operations. The labeled graph patterns in each query segment are rewritten according to vertex labels and edge labels, and the data operations are rewritten according to the attributes involved, into subqueries of the corresponding subgraph files in each cloud object storage service. A pre-set greedy access method is used to select the data access method that satisfies the query latency constraint and has the lowest cloud storage access cost for each subquery, generating the optimal economic cost query plan. The system receives the optimal economic cost query plan, and based on the cache hit status and data access method of the data required by each subquery in the optimal economic cost query plan, it decomposes each subquery into a combination of multiple query methods for fine-grained data fragments, dynamically generates a hybrid query plan, receives attribute data retrieved from the data storage module and caches it locally, determines the cache benefit of each data to be cached based on the comprehensive cache benefit model, and dynamically executes cost-priority cache replacement; the comprehensive cache benefit model normalizes the time cost and economic cost of network transmission, data scanning and computing layer operation when each cached data is retrieved from the data storage module into a benefit value, so as to maximize the economic benefit of caching; The system receives a mixed query plan and executes it segment by segment in the order of the query segments. For each subquery in the current query segment, according to the data source and data access method in the mixed query plan, it first retrieves the topology structure subfile, calls the corresponding operator to perform graph pattern matching evaluation, performs structure and identifier evaluation on the graph pattern operation result, and finally retrieves the attribute value in the attribute subfile according to the identifier in the operation result, calls the corresponding operator to perform data operation evaluation on the attribute value, and merges the query segments to generate the query answer.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 9.

Images