Graph data processing method and device, equipment, medium and product

Abstract

The invention provides a graph data processing method and device, equipment, a medium and a product, and relates to the technical field of artificial intelligence and cloud computing, in particular to the technical field of big data and graph data processing. The specific implementation scheme is as follows: responding to a graph processing request through an interface layer; and performing graph data processing according to the graph processing request and the graph processing layer and / or the query processing layer to obtain a graph processing result. Through the technical scheme, the image data processing efficiency can be improved.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and cloud computing technology, and in particular to the fields of big data and graph data processing technology. Background Art

[0002] With the rapid development of the internet, emerging technologies such as big data and cloud computing have emerged, resulting in explosive growth in data volumes and increasingly complex relationships between them. In numerous fields, such as social networks, information dissemination, and biological networks, data mining techniques focus on the relationships between nodes (such as articles) and edges (such as follows and forwarding) within a network, as well as the structure and characteristics of these relationships. Traditional relational databases have limitations when processing such complex network data. They require converting entities and relationships into relational schemas, which increases the complexity of data processing. Graph databases, however, have emerged as specialized database systems for storing and querying network data. They store data in a graph structure, directly representing network entities and relationships. They offer efficient querying and strong scalability, enabling effective analysis and mining of network data using graph algorithms. Therefore, the implementation of graph databases is crucial for improving the efficiency of graph data processing. Summary of the Invention

[0003] The present disclosure provides a graph data processing method, apparatus, device, and medium.

[0004] According to one aspect of the present disclosure, a graph data processing method is provided, which is applied to a graph data storage system, wherein the graph data storage system includes an interface layer, a query processing layer, and a graph processing layer; wherein the graph processing layer includes a graph model layer, a logic storage layer, and a basic storage layer; the basic storage layer is used to store attribute graphs;

[0005] The method comprises:

[0006] Responding to graph processing requests through the interface layer;

[0007] Graph data processing is performed according to the graph processing request, and the graph processing layer and / or the query processing layer to obtain a graph processing result.

[0008] According to another aspect of the present disclosure, a graph data processing device is provided, configured in a graph data storage system, wherein the graph data storage system includes an interface layer, a query processing layer, and a graph processing layer; wherein the graph processing layer includes a graph model layer, a logic storage layer, and a basic storage layer; the basic storage layer is used to store attribute graphs;

[0009] The device comprises:

[0010] A graph processing request response module, configured to respond to graph processing requests through an interface layer;

[0011] The graph processing result determination module is used to perform graph data processing according to the graph processing request and the graph processing layer and / or the query processing layer to obtain a graph processing result.

[0012] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the graph data processing method described in any embodiment of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the graph data processing method described in any embodiment of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the graph data processing method described in any embodiment of the present disclosure when executed by a processor.

[0018] According to the technology disclosed in the present invention, the processing efficiency of graph data can be improved.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a flowchart of a graph data processing method provided according to an embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram of a to-be-stored graph structure provided according to an embodiment of the present disclosure;

[0023] Figure 3 is a flowchart of another graph data processing method provided according to an embodiment of the present disclosure;

[0024] Figure 4 is a flowchart of another graph data processing method provided according to an embodiment of the present disclosure;

[0025] Figure 5 is a structural diagram of a graph data processing device provided according to an embodiment of the present disclosure;

[0026] Figure 6 It is a block diagram of an electronic device used to implement the graph data processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] In addition, it should be noted that the collection, storage, use, processing, transmission, provision and disclosure of cloud phone-related data involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0030] Currently, there are several main approaches to graph database implementation: 1) Native distributed graph databases, represented by products such as TigerGraph and JanusGraph. These products are designed with distributed environments in mind. TigerGraph uses a proprietary storage engine and parallel computing framework to deliver high-performance graph analytics. JanusGraph supports multiple backend storage systems (such as HBase and Cassandra), providing flexibility and scalability. 2) Clustered standalone graph databases, represented by products such as Neo4j Enterprise, achieve clustering through master-slave replication or sharding. These systems generally retain the ease of use of standalone versions but have limitations in horizontal scalability and distributed transaction consistency. 3) Layers based on general-purpose distributed storage, such as GraphX ​​based on Spark and Gelly based on Flink. These systems utilize graph processing as a component of a distributed computing framework, making them suitable for batch analytics but with limited support for real-time transaction processing. 4) Relational mapping of graph data: Graph data is mapped to a relational database for storage and graph operations are implemented through specialized query transformations. This approach leverages mature relational database technology but suffers from poor performance when handling complex graph traversals.

[0031] Existing distributed graph database technologies have the following major shortcomings: 1) It is difficult to balance strong consistency and high performance. Most distributed graph databases either provide weak consistency guarantees to achieve better performance (such as JanusGraph) or sacrifice performance and scalability for strong consistency guarantees (such as some Neo4j deployment modes). 2) Inefficient graph partitioning. Existing graph partitioning schemes typically use simple hashing or vertex-based partitioning strategies, which cannot effectively handle the associative nature of graph data, resulting in a large number of cross-partition operations and network communication overhead. 3) Poor performance for complex graph traversals. In a distributed environment, the performance of multi-hop graph traversals (such as n-hop queries) typically drops sharply, limiting their application in complex graph analysis scenarios. 4) Scalability bottlenecks. Many graph database systems experience performance bottlenecks when scaling to large-scale clusters, making it impossible to achieve near-linear performance improvements. 5) Proprietary storage engines limit flexibility. Some high-performance graph databases use proprietary storage engines. Although they have excellent performance, they lack flexibility and are difficult to adapt to the needs of different application scenarios.

[0032] Therefore, the present disclosure provides a distributed graph database storage system based on FoundationDB, which is suitable for various application scenarios that need to process large-scale graph data, especially social network analysis, knowledge graphs, recommendation systems, fraud detection, network security analysis and other scenarios.

[0033] Figure 1It is a flowchart of a graph data processing method provided according to an embodiment of the present disclosure. The method is applicable to various application scenarios that require processing large-scale graph data, such as social network analysis, knowledge graphs, recommendation systems, fraud detection, network security analysis and other scenarios. The method can be executed by a graph data processing device, which can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the graph data processing function, such as a graph data storage system on a PC. Optionally, the graph data storage system includes an interface layer, a query processing layer, and a graph processing layer; wherein the graph processing layer includes a graph model layer, a logical storage layer and a basic storage layer; the basic storage layer is used to store attribute graphs;

[0034] The interface layer can provide the resource presentation layer state transfer application programming interface (Representational State Transfer Application Programming Interface, REST API) and native client interface to support the creation, query and management of graph data.

[0035] The query processing layer is responsible for query parsing, optimization, and execution, and can convert high-level graph queries into underlying storage operations.

[0036] The graph model layer is used to implement the property graph model and maintain the upper-layer graph data index.

[0037] The logical storage layer, also known as the graph data storage abstraction layer, is used to map graph models into key-value pairs and manage key design, serialization encoding, and index maintenance.

[0038] The base storage layer, implemented by the FoundationDB distributed KV engine, provides primitive key-value storage, transaction processing, fault-tolerant replication, shard routing, and other capabilities, and is completely unaware of the upper-layer graph structure. Understandably, using FoundationDB, which provides strongly consistent ACID transactions, ensures that concurrent operations on property graph data remain consistent and isolated. ACID represents the four major properties of transactions: atomicity, consistency, isolation, and durability.

[0039] like Figure 1 As shown, the graph data processing method of this embodiment may include:

[0040] S101, responding to a graph processing request through the interface layer.

[0041] Among them, the graph processing request refers to a request for processing graph data; optionally, the graph processing request can be a graph storage request, a graph query request and a graph update request; among them, the graph storage request refers to a request for storing graph data; the graph query request refers to a request for querying graph data; the graph update request refers to a request for updating graph data.

[0042] S102: Perform graph data processing according to the graph processing request and the graph processing layer and / or the query processing layer to obtain a graph processing result.

[0043] The graph processing results refer to the results obtained after processing the graph data, including but not limited to storage results, query results, and update results.

[0044] An optional method is that if the graph processing request is a graph storage request, graph data processing is performed according to the graph processing request and the graph processing layer to obtain a graph processing result, including: determining the graph data to be stored according to the graph storage request; performing graph abstraction processing on the graph data to be stored through the graph model layer to obtain a graph structure to be stored; the graph structure to be stored includes vertices and edges; performing semantic mapping on the graph structure to be stored through the logical storage layer to obtain key-value pairs; performing underlying key-value mapping on the key-value pairs through the graph basic storage layer to obtain an attribute graph, and storing the attribute graph as the graph processing result.

[0045] The graph data to be stored refers to the graph data that needs to be stored. The graph structure to be stored refers to the graph model obtained by extracting the graph structure of the graph data to be stored, that is, the graph structure, including vertices and edges. The vertex represents an entity in the graph and has the following characteristics: 1) a unique identifier, which can be a vertex ID; 2) a label (Label), which indicates the vertex type; 3) properties (Properties), which are data in the form of key-value pairs. The edge represents the relationship between two vertices and has the following characteristics: 1) a unique identifier, which can be an edge ID; 2) a label (Label), which indicates the relationship type; 3) directionality, which connects the starting point (outgoing vertex) and the end point (e.g., vertex); 4) properties (Properties), which are data in the form of key-value pairs.

[0046] Specifically, the graph storage request is parsed to obtain the graph data to be stored. The graph model layer extracts vertices and edges from the graph data to be stored, as well as vertex attributes and edge attributes, to obtain the graph structure to be stored, such as Figure 2 The example shown; further, the vertices, edges and attributes in the graph structure to be stored are mapped into key-value pairs through the logical storage layer, and finally the key-value pairs are mapped to the underlying key value through the basic storage layer to obtain the attribute graph, and the attribute graph is stored as the graph processing result.

[0047] It can be understood that the graph model layer extracts semantics from graph data, the logical storage layer extracts semantics from graph structure, and the basic storage layer does not understand graph semantics and only performs distributed KV storage. In this way, the graph processing layers cooperate with each other to realize the storage of graph data, which can improve the storage performance of graph data.

[0048] The property graph includes vertex metadata keys, vertex attribute keys, edge metadata keys, edge attribute keys, outgoing edge index keys, and incoming edge index keys;

[0049] Vertex attribute keys are used to store basic information about vertices. The key format of vertex metadata keys is [vertex prefix][vertex ID]; the value format of vertex metadata keys is {vertex label information}; for example, the key format is: [vertex prefix (0x01)][vertex ID]; the value format is: {label:"label name"}.

[0050] Vertex attribute keys are used to store vertex attribute values; the key format of a vertex attribute key is [vertex attribute prefix][vertex ID][attribute name]; the value format of a vertex attribute key is attribute value; for example, its key format is: [vertex attribute prefix (0x04)][vertex ID][attribute name]; the value format is: attribute value (after serialization).

[0051] The edge metadata key is used to store basic information about the edge. The key format of the edge metadata key is [edge prefix][edge ID]. The value format of the edge metadata key is {edge label information, outgoing vertex ID, incoming vertex ID, and time information}. For example, its key format is [edge prefix (0x02)][edge ID]; the key format is: {label:"label name", out_v:"outgoing vertex ID", in_v:"incoming vertex ID", created_at: timestamp,...}.

[0052] The edge attribute key is used to store the attribute value of the edge; the key format of the edge attribute key is [edge attribute prefix][edge ID][attribute name]; the value format of the edge attribute key is the attribute value; for example, its key format is: [edge attribute prefix (0x05)][edge ID][attribute name]; the value format is: attribute value (after serialization).

[0053] The outgoing edge index key is used to record the outgoing edges of a vertex; the key format of the outgoing edge index key is [outgoing edge index prefix][outgoing vertex ID][edge label][edge ID]; the value format of the outgoing edge index key is the incoming vertex ID; for example, its key format is: [outgoing edge index prefix (0x06)][outgoing vertex ID][edge label][edge ID]; the value format is: incoming vertex ID.

[0054] The incoming edge index key is used to record the incoming edges of a vertex; the key format of the incoming edge index key is [incoming edge index prefix][incoming vertex identifier][edge label][edge identifier]; the value format of the incoming edge index key is the outgoing vertex identifier; for example, the key format: [incoming edge index prefix (0x07)][incoming vertex ID][edge label][edge ID]; the value format: outgoing vertex ID.

[0055] A specific example is as follows:

[0056] Vertex A (id: v1, label: person, properties: {name: "张三", age: 30});

[0057] Vertex B (id: v2, label: person, properties: {name: "李四", age: 25});

[0058] Edge E (id: e1, label: knows, out_v: v1, in_v: v2, properties: {since: 2020});

[0059] The property graph stored in the basic storage layer is:

[0060] 0x01:7631 -> {label: "person", created_at: 1609459200};

[0061] 0x04:7631:6e616d -> "张三";

[0062] 0x04:7631:616765 -> 30;

[0063] 0x01: -> {label: "person", created_at: 1609459200};

[0064] 0x04:7632:6e616d65 -> "李四";

[0065] 0x04:7632:616765 -> 25;

[0066] 0x02:6531 -> {label: "knows", out_v: "v1", in_v: "v2", created_at: 1609459200};

[0067] 0x05:6531:73696e6365 -> 2020;

[0068] 0x06:7631:6b6e6f7773:65 -> "v2";

[0069] 0x07:7632:6b6e6f7773:6531->"v1.

[0070] It can be understood that the above-designed property graph key-value pair mapping method can support efficient vertex neighbor queries and quickly obtain the outgoing or incoming edges of a specific vertex, thereby greatly improving the graph traversal performance.

[0071] This disclosure optimizes data locality and reduces cross-partition communication through an efficient key design strategy.

[0072] The technical solution provided by the embodiments of the present disclosure utilizes an interface layer in a graph data storage system to respond to graph processing requests; based on the graph processing requests and the graph processing layer and / or query processing layer, graph data processing is performed to obtain graph processing results. This technical solution improves graph data processing efficiency by performing graph data processing, such as storage and querying, through the graph data storage system provided by the present disclosure.

[0073] Figure 3 This is a flowchart of another graph data processing method provided according to an embodiment of the present disclosure. Based on the above embodiment, if the graph processing request is a graph query request, this embodiment further optimizes "according to the graph processing request, the graph processing layer and the query processing layer, the graph data processing is performed to obtain the graph processing result", and provides an optional implementation scheme. Figure 3 As shown,

[0074] S201, responding to a graph processing request through the interface layer.

[0075] S202: Determine the graph data to be queried according to the graph query request.

[0076] The graph data to be queried refers to query statements for graph-related data that need to be queried, including but not limited to query statements for vertices, vertex attributes, edges, edge-related attributes, paths between vertices, etc.

[0077] It should be noted that the graph query request can be a Cypher query language.

[0078] Specifically, the graph query request is parsed to obtain the graph data to be queried. For example, the graph data to be queried is:

[0079] MATCH(a:Person{name:"Alice"})-[:FRIEND]->(b:Person);

[0080] RETURN b.name.

[0081] S203: Perform graph operation analysis on the graph data to be queried through the query processing layer to obtain an intermediate graph representation.

[0082] Among them, the graph intermediate representation is a general graph operation representation.

[0083] An optional method is to perform syntax parsing on the graph data to be queried through the query processing layer and split it into character sets; perform syntax analysis on the character sets and build a syntax tree; and perform semantic analysis on the syntax tree to obtain an intermediate representation of the graph.

[0084] Specifically, the query processing layer performs syntax parsing on the graph data to be queried, that is, splitting the graph data into tokens, such as MATCH, a, :Persoon, ->, etc., to obtain a character set. Then, the character set is parsed to build a syntax tree. Finally, the syntax tree is semantically analyzed, such as binding entity types, checking variables, converting to logical graph query operations, etc., to obtain the intermediate representation of the graph, for example:

[0085] "start":{

[0086] "label":"Person",

[0087] "filter":{"name":"Alice"}

[0088] },

[0089] "traverse":{

[0090] "direction":"OUT",

[0091] "edgeLabel":"FRIEND",

[0092] "targetLabel":"Person"

[0093] },

[0094] "return":["b.name"]

[0095] }

[0096] It can be understood that by parsing the graph data to be queried, i.e., the graph query statement, an intermediate representation of the graph is obtained, which facilitates the improvement of the efficiency of subsequent graph queries.

[0097] S204: rewrite and optimize the rules of the graph intermediate representation through the query processing layer to obtain optimized query logic.

[0098] In this embodiment, rule rewriting and optimization refers to the rewriting and optimization of the graph intermediate representation, i.e., the query logic, including but not limited to various types of logic optimization, such as predicate pushdown, edge index selection, restricted projection fields, and filter merging. Predicate pushdown refers to narrowing the search scope, for example, limiting the scan scope based on Label = Person and name = Alice before scanning vertices; edge index selection refers to querying using edge index keys without scanning all edges, for example, using an index based on FRIEND edges without scanning all edges; restricted projection fields refers to selecting only the fields corresponding to the results returned by the intermediate representation layer, for example, only taking b.name to avoid pulling in all attributes; and filter merging refers to merging query conditions, for example, merging WHERE conditions to reduce multiple scans. Optimizing query logic refers to query logic after optimizing the graph intermediate representation.

[0099] Specifically, the query processing layer selects the appropriate logic optimization type to rewrite and optimize the graph intermediate representation rules to obtain the optimized query logic, for example,

[0100] 1. Use the attribute index to find the vertex V1 with label = Person and name = Alice;

[0101] 2. Scan the range key ("tenant","Eo",V1,"FRIEND",*) to get the outgoing edge;

[0102] 3. Scan the ToID of each outgoing edge ("tenant", "N", ToID) to obtain vertex attributes;

[0103] 4. The projected name attribute is returned as the result.

[0104] This can reduce the query cost and thus improve the efficiency of graph data queries.

[0105] S205: The query processing layer obtains the query result of the graph data to be queried from the basic storage layer based on the optimized query logic, and uses the query result as the graph processing result.

[0106] Specifically, the query logic is optimized to compile the KV operation sequence of FoundationDB, the query result of the graph data to be queried is obtained from the basic storage layer, and the query result is returned as the graph processing result.

[0107] In an optional manner, the graph data to be queried includes target attributes and attributes to be queried; accordingly, the query processing layer obtains the query result of the graph data to be queried from the basic storage layer based on the optimized query logic, including: based on the attribute index prefix, according to the target attribute, querying from the attribute graph stored in the basic storage layer to obtain the query starting point; according to the out-edge index key and the query starting point, querying from the attribute graph to obtain at least one associated vertex of the query starting point; according to the attribute to be queried, batch reading the vertex attribute key of at least one associated vertex to obtain the associated attributes of the associated vertex; assembling at least one attribute to obtain the query result.

[0108] The target attribute refers to the vertex attribute given in the graph data to be queried. The query attribute refers to the attribute information related to the vertex corresponding to the target attribute that needs to be returned. The query starting point refers to the vertex corresponding to the target attribute. The associated vertex refers to the vertex related to the query starting point. The associated attribute refers to the attribute information of the associated vertex.

[0109] Specifically, the query starting point is obtained from the attribute graph stored in the basic storage layer based on the attribute index prefix and the target attribute, that is, the point query attribute index. For example:

[0110] / / Use the secondary index prefix (the attribute index prefix is ​​0x08) to query vertices whose attribute is "name" = "Alice".

[0111] db.getRange(

[0112] Tuple.from(0x08,"Person","name","Alice"),

[0113] Tuple.from(0x08,"Person","name","Alice",0xFF) )

[0115] →Returns the vertex ID: "V1".

[0116] Then, using the outbound edge index key and the query starting point as the index, query the outbound edge of the query starting point from the property graph, that is, at least one associated vertex of the query starting point, for example,

[0117] / / Use the outgoing edge index key [0x06][out_v][edge_label][edge_id]→in_v to query the edge starting from a vertex.

[0118] db.getRange(

[0119] Tuple.from(0x06,"v1","FRIEND"),

[0120] Tuple.from(0x06,"v1","FRIEND",0xFF) )

[0122] / / Return: multiple key-value pairs, for example:

[0123] / / 0x06:v1:FRIEND:e123→v2

[0124] / / 0x06:v1:FRIEND:e124→v3.

[0125] Then, according to the attributes to be checked, batch read the vertex attribute key of at least one associated vertex, read the value of the vertex attribute key, and obtain the associated attributes of the associated vertex, for example,

[0126] / / To query the properties of vertices v2 and v3 (such as name), you need to read the vertex property key [0x04][vertex_id][property_key]

[0127] db.getBatch([

[0128] Tuple.from(0x04,"v2","name"),

[0129] Tuple.from(0x04,"v3","name") ])

[0131] / / return:

[0132] / / 0x04:v2:name→"Bob"

[0133] / / 0x04:v3:name→"Carol".

[0134] Finally, at least one attribute is assembled to obtain the query result, that is, [{b.name:"Bob"},{b.name:"Carol"}] is constructed.

[0135] It is understandable that performing point queries using out-edge index keys and attribute index keys can improve the efficiency of graph data queries.

[0136] The technical solution provided by the embodiments of the present disclosure determines the graph data to be queried based on a graph query request; performs graph operation analysis on the graph data to be queried by the query processing layer to obtain an intermediate graph representation; rewrites and optimizes the intermediate graph representation by the query processing layer to obtain optimized query logic; and, based on the optimized query logic, obtains a query result for the graph data to be queried from the basic storage layer by the query processing layer, and uses the query result as the graph processing result. The above technical solution can improve the efficiency of graph data queries by optimizing the query logic for graph data queries.

[0137] Figure 4 This is a flowchart of another graph data processing method provided according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes "obtaining the query result of the graph data to be queried from the basic storage layer based on the optimized query logic through the query processing layer" and provides an optional implementation plan. Figure 4 As shown, the method includes:

[0138] S301, responding to a graph processing request through the interface layer.

[0139] S302: Determine the graph data to be queried according to the graph query request.

[0140] S303: Perform graph operation analysis on the graph data to be queried through the query processing layer to obtain an intermediate graph representation.

[0141] S304: rewrite and optimize the graph intermediate representation through the query processing layer to obtain optimized query logic.

[0142] S305: The query processing layer obtains the query result of the graph data to be queried from the basic storage layer based on the optimized query logic, and uses the query result as the graph processing result.

[0143] In an optional manner, the graph data to be queried includes querying whether there is a path from the target starting point to the target end point; accordingly, the query processing layer obtains the query result of the graph data to be queried from the basic storage layer based on the optimized query logic, including: initializing the forward frontier set with the target starting point, and initializing the reverse frontier set with the target end point; using the vertices in the forward frontier set and the vertices in the reverse frontier set as query starting points respectively, performing a path query from the attribute graph stored in the basic storage layer, and obtaining the query result of the graph data to be queried.

[0144] The forward frontier set is used to store the point set of the forward query starting point, including at least one vertex. The reverse frontier set is used to store the point set of the reverse query starting point, including at least one vertex.

[0145] Specifically, the forward frontier set is initialized with the target starting point, i.e., the target starting point is initially in the forward frontier set, and the reverse frontier set is initialized with the target endpoint, i.e., the target endpoint is initially in the reverse frontier set. Then, using vertices in the forward frontier set and the reverse frontier set as the query starting point, the outgoing edge vertices of the query starting point are searched from the attribute graph stored in the base storage layer. The path is determined based on the outgoing edge vertices to obtain the query result of the graph data to be queried.

[0146] It is understandable that the query speed can be increased through bidirectional query.

[0147] Exemplarily, taking the vertices in the forward frontier set and the vertices in the reverse frontier set as query starting points, respectively, a path query is performed from the property graph stored in the basic storage layer to obtain the query results of the graph data to be queried, including: taking the midpoint of the forward frontier set as the query starting point, forward indexing is performed according to the outbound index key of the query starting point, the first neighbor vertex of the query starting point is determined from the property graph stored in the basic storage layer, and the forward frontier set is updated according to the first neighbor vertex; taking the midpoint of the reverse frontier set as the query end point, reverse indexing is performed according to the inbound index key of the query end point, the second neighbor vertex of the query end point is determined from the property graph stored in the basic storage layer, and the reverse frontier set is updated according to the second neighbor vertex; judging whether the forward frontier set and the reverse frontier set intersect; and determining the query results of the graph data to be queried based on the intersection judgment result.

[0148] Specifically, the midpoint of the forward frontier set is used as the query starting point, and forward indexing is performed according to the outbound index key of the query starting point, and the first neighbor vertex of the query starting point is determined from the attribute graph stored in the basic storage layer, wherein the number of the first neighbor vertices is one or more, and the forward frontier set is updated according to the first neighbor vertex, that is, the vertex in the forward frontier set is deleted and replaced with the first neighbor vertex; the midpoint of the reverse frontier set is used as the query end point, and reverse indexing is performed according to the inbound index key of the query end point, and the second neighbor vertex of the query end point is determined from the attribute graph stored in the basic storage layer, wherein the number of the second neighbor vertices is one or more, and the reverse frontier set is updated according to the second neighbor vertex, that is, the vertex in the reverse frontier set is deleted and replaced with the second neighbor vertex. Determine whether the forward frontier set and the reverse frontier set intersect, that is, determine whether the forward frontier set and the reverse frontier set have an intersection; based on the intersection judgment result, if there is an intersection in the intersection judgment result, then backtrace the path based on the intersection to determine the complete path, and use the complete path as the query result of the graph data to be queried.

[0149] It is understandable that by updating the forward front set and the reverse front set and determining a new query starting point, a bidirectional path search is performed, thereby improving the search speed.

[0150] Furthermore, based on the intersection judgment result, the query result of the map data to be queried is determined, including: if the intersection judgment result includes the intersection point, the path is backtracked based on the intersection point, the target starting point and the target end point to obtain the complete path, and the complete path is used as the query result of the map data to be queried; if the intersection judgment result does not include the intersection point, the index query is re-performed from the attribute graph based on the new forward frontier set and the new reverse frontier set.

[0151] Specifically, if the intersection judgment result includes the intersection, the path is backtracked according to the intersection, the target starting point and the target end point to obtain the complete path. Specifically, the forward path can be determined by querying the edge metadata key and the edge attribute key, and the path mapping is performed according to the intersection and the target starting point; the reverse path is determined by querying the edge metadata key and the edge attribute key, and the path mapping is performed according to the intersection and the target end point; the complete path is determined based on the forward path and the reverse path, and the complete path is used as the query result of the map data to be queried; if the intersection judgment result does not include the intersection, and the search depth does not reach the maximum depth, the index query is re-performed from the attribute graph according to the new forward frontier set and the new reverse frontier set, that is, a circular index query, and the search depth, that is, the number of searches, is determined. If the search depth is greater than the maximum depth, it is returned that the path does not exist; wherein the maximum depth can be set based on actual conditions, for example, 10.

[0152] It can be understood that by performing path backtracking mapping through edge metadata keys and edge attribute keys, the path can be quickly determined and query efficiency can be improved.

[0153] Another optional method is that if there are multiple vertices in the forward front set, and / or there are multiple vertices in the reverse front set, then the vertices in the forward front set and the vertices in the reverse front set are used as query starting points respectively, and a path query is performed from the attribute graph stored in the basic storage layer to obtain the query results of the graph data to be queried, including: determining at least two sub-query tasks based on the number of vertices and the forward front set and the reverse front set, and the sub-query tasks include a sub-forward front set and a sub-reverse front set; in parallel, using the vertices in the sub-forward front set and the vertices in the sub-reverse front set as query starting points respectively, and performing a path query from the attribute graph stored in the basic storage layer to obtain sub-query results; merging at least two sub-query results to obtain the query results of the graph data to be queried.

[0154] Specifically, if there are multiple vertices in the forward front set, and / or there are multiple vertices in the reverse front set, then at least two sub-query tasks are determined based on the number of vertices, the forward front set, and the reverse front set. The sub-query task includes a sub-forward front set and a sub-reverse front set, and each of the sub-forward front set and the reverse front set includes a vertex. Then, each sub-query task is executed in parallel, that is, the vertices in the sub-forward front set and the vertices in the sub-reverse front set are used as query starting points in parallel, and a path query is performed from the attribute graph stored in the basic storage layer to obtain the sub-query result. The specific query process is the same as the specific query process described above, "Using the vertices in the forward front set and the vertices in the reverse front set as query starting points, a path query is performed from the attribute graph stored in the basic storage layer to obtain the query result of the graph data to be queried." Then, the at least two sub-query results are sorted and merged to obtain a complete query path, and the complete query path is used as the query result of the graph data to be queried.

[0155] It is understandable that performing path queries in parallel can further improve query efficiency.

[0156] It should be noted that the basic storage layer of this disclosure uses FoundationDB, which provides strong ACID transaction consistency, ensuring that concurrent operations on property graph data remain consistent and isolated. The optimistic concurrency control mechanism of this system is implemented based on the following process:

[0157] 1. Read phase: The transaction reads data without locking and records the read keys in the read set.

[0158] 2. Local computing: Perform calculations and operations based on the read data.

[0159] 3. Write phase: Prepare to write data and record the modification key in the write set.

[0160] 4. Verification phase: Check whether the keys in the read set have been modified by other transactions before committing.

[0161] 5. Commit / Rollback: Commit if verification passes, otherwise roll back and try again.

[0162] Among them, it should be explained that the meaning of optimistic concurrency control is: optimistic concurrency control assumes that concurrency conflicts rarely occur. Optimistic concurrency control assumes that concurrency conflicts rarely occur, that is, no locking is done during reading and checking is done during writing, avoiding performance bottlenecks caused by locking; that is, no locking is done during transaction execution, but only checking whether there has been any changes at the time of submission.

[0163] FoundationDB implements conflict detection and resolution mechanisms internally, but graph operations often involve multiple keys and require special handling:

[0164] Read-write skew detection: For operations that rely on updating multiple vertices or edges, the system detects potential conflicts through read-write set analysis.

[0165] Automatic retry mechanism: For transactions that fail due to conflicts, the system implements an exponential backoff retry strategy.

[0166] Figure 5 It is a structural diagram of a graph data processing device provided according to an embodiment of the present disclosure. This embodiment is applicable to various application scenarios that require processing large-scale graph data, such as social network analysis, knowledge graphs, recommendation systems, fraud detection, network security analysis and other scenarios. The device can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the graph data processing function, such as a graph data storage system on a PC. Optionally, the graph data storage system includes an interface layer, a query processing layer, and a graph processing layer; wherein the graph processing layer includes a graph model layer, a logical storage layer and a basic storage layer; the basic storage layer is used to store attribute graphs.

[0167] like Figure 5 As shown, the graph data processing device 400 includes:

[0168] A graph processing request response module 401 is configured to respond to a graph processing request via an interface layer;

[0169] The graph processing result determination module 402 is used to perform graph data processing according to the graph processing request and the graph processing layer and / or the query processing layer to obtain a graph processing result.

[0170] The technical solution provided by the embodiments of the present disclosure utilizes an interface layer in a graph data storage system to respond to graph processing requests; based on the graph processing requests and the graph processing layer and / or query processing layer, graph data processing is performed to obtain graph processing results. This technical solution improves graph data processing efficiency by performing graph data processing, such as storage and querying, through the graph data storage system provided by the present disclosure.

[0171] Furthermore, the graph processing result determination module 402 is used to:

[0172] Determine the graph data to be stored according to the graph storage request;

[0173] The graph model layer performs graph abstraction processing on the graph data to be stored to obtain the graph structure to be stored; the graph structure to be stored includes vertices and edges;

[0174] Perform semantic mapping on the graph structure to be stored through the logical storage layer to obtain key-value pairs;

[0175] The underlying key-value mapping of key-value pairs is performed through the graph basic storage layer to obtain the attribute graph, which is then stored as the graph processing result.

[0176] Furthermore, the property graph includes vertex metadata keys, vertex attribute keys, edge metadata keys, edge attribute keys, outbound edge index keys, and inbound edge index keys; wherein,

[0177] Vertex attribute keys are used to store basic information about vertices. The key format of vertex metadata keys is [vertex prefix][vertex identifier]. The value format of vertex metadata keys is {vertex label information}.

[0178] Vertex attribute keys are used to store vertex attribute values; the key format of vertex attribute keys is [vertex attribute prefix][vertex identifier][attribute name]; the value format of vertex attribute keys is attribute value;

[0179] The edge metadata key is used to store basic information about the edge. The key format of the edge metadata key is [edge prefix][edge identifier]. The value format of the edge metadata key is {edge label information, outgoing vertex identifier, incoming vertex identifier, and time information}.

[0180] The edge attribute key is used to store the attribute value of the edge; the key format of the edge attribute key is [edge attribute prefix][edge identifier][attribute name]; the value format of the edge attribute key is attribute value;

[0181] The out-edge index key is used to record the outgoing edges of the vertex; the key format of the out-edge index key is [out-edge index prefix][out-vertex identifier][edge label][edge identifier]; the value format of the out-edge index key is the in-vertex identifier;

[0182] The in-edge index key is used to record the in-edges of a vertex. The key format of the in-edge index key is [in-edge index prefix][in-vertex identifier][edge label][edge identifier]. The value format of the in-edge index key is the out-vertex identifier.

[0183] Furthermore, if the graph processing request is a graph query request, the graph processing result determination module 402 includes:

[0184] a to-be-queried map data determining unit, configured to determine the to-be-queried map data according to the map query request;

[0185] A graph intermediate representation determination unit is used to perform graph operation analysis on the query graph data through the query processing layer to obtain the graph intermediate representation;

[0186] An optimized query logic determination unit is used to rewrite and optimize the graph intermediate representation through the query processing layer to obtain optimized query logic;

[0187] The graph processing result determination unit is used to obtain the query result of the graph data to be queried from the basic storage layer based on the optimized query logic through the query processing layer, and use the query result as the graph processing result.

[0188] Furthermore, the middle portion of the figure shows that the determination unit is specifically used for:

[0189] The query processing layer performs syntax analysis on the image data to be queried and splits it into character sets;

[0190] Perform grammatical analysis on the character set and construct a syntax tree;

[0191] Perform semantic analysis on the syntax tree to obtain the intermediate representation of the graph.

[0192] Furthermore, the graph data to be checked includes target attributes and attributes to be checked; accordingly, the graph processing result determination unit is used to:

[0193] Based on the attribute index prefix, the query starting point is obtained from the attribute graph stored in the basic storage layer according to the target attribute;

[0194] According to the outgoing edge index key and the query starting point, query and obtain at least one associated vertex of the query starting point from the property graph;

[0195] According to the attributes to be checked, batch read the vertex attribute keys of at least one associated vertex to obtain the associated attributes of the associated vertex;

[0196] At least one attribute is assembled to obtain a query result.

[0197] Furthermore, the map data to be checked includes querying whether there is a path from the target starting point to the target end point; accordingly, the map processing result determination unit is used to:

[0198] Initialize the forward frontier set with the target starting point, and initialize the reverse frontier set with the target end point;

[0199] Taking the vertices in the forward front set and the vertices in the reverse front set as query starting points, respectively, a path query is performed from the attribute graph stored in the basic storage layer to obtain the query results of the graph data to be queried.

[0200] Furthermore, the graph processing result determination unit is specifically configured to:

[0201] Take the midpoint of the forward frontier set as the query starting point, perform forward indexing based on the outbound edge index key of the query starting point, determine the first neighbor vertex of the query starting point from the property graph stored in the base storage layer, and update the forward frontier set based on the first neighbor vertex;

[0202] The midpoint of the reverse frontier set is used as the query endpoint. The reverse index is performed based on the incoming edge index key of the query endpoint. The second neighbor vertex of the query endpoint is determined from the property graph stored in the base storage layer. The reverse frontier set is updated based on the second neighbor vertex.

[0203] Determine whether the forward front set and the reverse front set intersect;

[0204] According to the intersection judgment result, the query result of the map data to be queried is determined.

[0205] Furthermore, the graph processing result determination unit is specifically configured to:

[0206] If the intersection judgment result includes an intersection point, the path is backtracked based on the intersection point, the target starting point, and the target end point to obtain a complete path, which is used as the query result of the map data to be queried;

[0207] If the intersection judgment result does not include the intersection point, the index query is re-performed from the attribute graph based on the new forward front set and the new reverse front set.

[0208] Furthermore, the graph processing result determination unit is specifically configured to:

[0209] By querying the edge metadata key and edge attribute key, the path mapping is performed according to the intersection point and the target starting point to determine the forward path;

[0210] By querying the edge metadata key and edge attribute key, the path mapping is performed according to the intersection point and the target end point to determine the reverse path;

[0211] Based on the forward path and the reverse path, the complete path is determined.

[0212] Furthermore, if there are multiple vertices in the forward front set, and / or there are multiple vertices in the reverse front set, then the graph processing result determination unit is specifically configured to:

[0213] Determine at least two sub-query tasks according to the number of vertices and the forward front set and the reverse front set, wherein the sub-query task includes a sub-forward front set and a sub-reverse front set;

[0214] In parallel, the vertices in the sub-forward front set and the vertices in the sub-reverse front set are used as query starting points, and path queries are performed from the attribute graph stored in the basic storage layer to obtain sub-query results;

[0215] Merge at least two sub-query results to obtain the query result of the graph data to be queried.

[0216] Furthermore, the basic storage layer is the FoundationDB distributed KV engine.

[0217] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0218] Figure 6 It is a block diagram of an electronic device used to implement the graph data processing method of the embodiment of the present disclosure. Figure 6 It is a block diagram of an electronic device used to implement the graph data processing method of the embodiment of the present disclosure. Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0219] like Figure 6 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0220] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0221] The computing unit 501 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the graph data processing method. For example, in some embodiments, the graph data processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the graph data processing method described above can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the graph data processing method by any other suitable means (e.g., via firmware).

[0222] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0223] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0224] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0225] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0226] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0227] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0228] Artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0229] Cloud computing refers to a technology system that provides network access to elastically scalable shared pools of physical or virtual resources. These resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on-demand in a self-service manner. Cloud computing technology provides efficient and powerful data processing capabilities for the application of technologies such as artificial intelligence and blockchain, as well as for model training.

[0230] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0231] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A graph data processing method, applied to a graph data storage system, wherein: The graph data storage system includes an interface layer, a query processing layer, and a graph processing layer; wherein the graph processing layer includes a graph model layer, a logic storage layer, and a basic storage layer; the basic storage layer is used to store attribute graphs; The method comprises: Responding to graph processing requests through the interface layer; Graph data processing is performed according to the graph processing request, and the graph processing layer and / or the query processing layer to obtain a graph processing result.

2. The method according to claim 1, wherein If the graph processing request is a graph storage request, graph data processing is performed according to the graph processing request and the graph processing layer to obtain a graph processing result, including: determining the graph data to be stored according to the graph storage request; Performing graph abstraction processing on the graph data to be stored through a graph model layer to obtain a graph structure to be stored; the graph structure to be stored includes vertices and edges; Perform semantic mapping on the graph structure to be stored through the logical storage layer to obtain key-value pairs; The key-value pairs are mapped to underlying key values ​​through a graph base storage layer to obtain a property graph, which is then stored as a graph processing result.

3. The method according to claim 1 or 2, wherein: The property graph includes vertex metadata keys, vertex attribute keys, edge metadata keys, edge attribute keys, outbound edge index keys and inbound edge index keys; wherein, The vertex attribute key is used to store basic information of the vertex; the key format of the vertex metadata key is [vertex prefix][vertex identifier]; the value format of the vertex metadata key is {vertex label information}; The vertex attribute key is used to store the attribute value of the vertex; the key format of the vertex attribute key is [vertex attribute prefix][vertex identifier][attribute name]; the value format of the vertex attribute key is attribute value; The edge metadata key is used to store basic information about the edge; the key format of the edge metadata key is [edge prefix][edge identifier]; the value format of the edge metadata key is {edge label information, outgoing vertex identifier, incoming vertex identifier and time information}; The edge attribute key is used to store the attribute value of the edge; the key format of the edge attribute key is [edge attribute prefix][edge identifier][attribute name]; the value format of the edge attribute key is attribute value; The outbound edge index key is used to record the outbound edge of the vertex; the key format of the outbound edge index key is [outbound edge index prefix][outbound vertex identifier][edge label][edge identifier]; the value format of the outbound edge index key is inbound vertex identifier; The in-edge index key is used to record the in-edge of the vertex; the key format of the in-edge index key is [in-edge index prefix][in-vertex identifier][edge label][edge identifier]; the value format of the in-edge index key is the out-vertex identifier.

4. The method according to claim 1, wherein If the graph processing request is a graph query request, graph data processing is performed according to the graph processing request, the graph processing layer, and the query processing layer to obtain a graph processing result, including: Determining the graph data to be queried according to the graph query request; Performing graph operation analysis on the graph data to be queried by the query processing layer to obtain an intermediate graph representation; The query processing layer rewrites and optimizes the graph intermediate representation rules to obtain optimized query logic; The query processing layer obtains query results of the graph data to be queried from the basic storage layer based on the optimized query logic, and uses the query results as graph processing results.

5. The method according to claim 4, wherein The query processing layer performs graph operation analysis on the graph data to be queried to obtain an intermediate graph representation, including: The query processing layer performs syntax analysis on the image data to be queried and splits it into character sets; Performing grammatical analysis on the character set to construct a grammar tree; Perform semantic analysis on the syntax tree to obtain the graph intermediate representation.

6. The method according to claim 4, wherein: The image data to be queried includes target attributes and attributes to be queried. Accordingly, the query processing layer obtains query results of the image data to be queried from the basic storage layer based on the optimized query logic, including: Based on the attribute index prefix, the query starting point is obtained from the attribute graph stored in the basic storage layer according to the target attribute; According to the outgoing edge index key and the query starting point, query and obtain at least one associated vertex of the query starting point from the property graph; According to the attribute to be checked, batch read the vertex attribute key of the at least one associated vertex to obtain the associated attribute of the associated vertex; At least one attribute is assembled to obtain a query result.

7. The method according to claim 4, wherein: The map data to be queried includes querying whether a path exists from a target starting point to a target end point. Accordingly, the query processing layer obtains a query result of the map data to be queried from the basic storage layer based on the optimized query logic, including: Initializing a forward front set with the target starting point, and initializing a reverse front set with the target end point; Taking the vertices in the forward front set and the vertices in the reverse front set as query starting points respectively, a path query is performed from the attribute graph stored in the basic storage layer to obtain a query result of the graph data to be queried.

8. The method according to claim 7, wherein: Taking the vertices in the forward front set and the vertices in the reverse front set as query starting points, respectively, a path query is performed from the attribute graph stored in the basic storage layer to obtain query results for the graph data to be queried, including: Taking the midpoint of the forward frontier set as the query starting point, performing forward indexing according to the outbound edge index key of the query starting point, determining the first neighbor vertex of the query starting point from the attribute graph stored in the base storage layer, and updating the forward frontier set according to the first neighbor vertex; Taking the midpoint of the reverse frontier set as the query endpoint, performing reverse indexing according to the incoming edge index key of the query endpoint, determining the second neighbor vertex of the query endpoint, and updating the reverse frontier set according to the second neighbor vertex; Determining whether the forward front set and the reverse front set intersect; According to the intersection judgment result, the query result of the map data to be queried is determined.

9. The method according to claim 8, wherein Determining the query result of the to-be-queried map data according to the intersection judgment result includes: If the intersection judgment result includes an intersection point, then the path is backtracked according to the intersection point, the target starting point and the target end point to obtain a complete path, and the complete path is used as the query result of the map data to be queried; If the intersection judgment result does not include an intersection point, an index query is performed again from the attribute graph according to the new forward front set and the new reverse front set.

10. The method according to claim 9, wherein: Performing path backtracking based on the intersection, the target starting point, and the target end point to obtain a complete path includes: By querying the edge metadata key and the edge attribute key, performing path mapping according to the intersection point and the target starting point, a forward path is determined; By querying the edge metadata key and the edge attribute key, performing path mapping according to the intersection point and the target end point, a reverse path is determined; A complete path is determined based on the forward path and the reverse path.

11. The method according to claim 7, wherein: If there are multiple vertices in the forward front set and / or multiple vertices in the reverse front set, a path query is performed from the attribute graph stored in the basic storage layer using the vertices in the forward front set and the vertices in the reverse front set as query starting points, respectively, to obtain query results for the graph data to be queried, including: Determine at least two sub-query tasks according to the number of vertices and the forward front set and the reverse front set, wherein the sub-query task includes a sub-forward front set and a sub-reverse front set; In parallel, taking the vertices in the sub-forward front set and the vertices in the sub-reverse front set as query starting points, respectively, performing path queries from the attribute graph stored in the basic storage layer to obtain sub-query results; Merge at least two sub-query results to obtain the query result of the graph data to be queried.

12. The method according to any one of claims 1 to 11, wherein The basic storage layer is the FoundationDB distributed KV engine.

13. A graph data processing device, configured in a graph data storage system, wherein: The graph data storage system includes an interface layer, a query processing layer, and a graph processing layer; wherein the graph processing layer includes a graph model layer, a logic storage layer, and a basic storage layer; the basic storage layer is used to store attribute graphs; The device comprises: A graph processing request response module, configured to respond to graph processing requests through an interface layer; The graph processing result determination module is used to perform graph data processing according to the graph processing request and the graph processing layer and / or the query processing layer to obtain a graph processing result.

14. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the graph data processing method according to any one of claims 1 to 12.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the graph data processing method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program, which, when executed by a processor, implements the graph data processing method according to any one of claims 1 to 12.

Images