A graph data processing method, device and equipment and computer storage medium

Abstract

Embodiments of the present application disclose a graph data processing method, device and equipment and a computer storage medium. The method comprises the following steps: receiving an access request; determining a target region from a storage node cluster based on the access request, so as to store target graph data in the access request in the target region, or read graph data meeting a query condition in the access request from the target region; wherein the graph database comprises a storage node cluster, the storage node cluster comprises at least two storage nodes, each of the storage nodes comprises at least two regions, and the storage structure of graph data in each of the regions is a linear partition structure.

Description

Technical Field

[0001] This application relates to the field of data, and includes, but is not limited to, a graph data processing method, apparatus, device, and computer storage medium. Background Technology

[0002] Most existing graph databases are single-machine solutions or single-machine hot standby, which have certain limitations in terms of use cases and read / write performance, and cannot meet the application needs of large-scale data. Summary of the Invention

[0003] In view of this, embodiments of this application provide a graph data processing method, apparatus, device, and computer storage medium.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a graph data processing method, the method comprising: receiving an access request; determining a target region from a storage node cluster based on the access request, so as to store the target graph data in the access request in the target region, or, reading graph data from the target region that satisfies the query conditions in the access request; wherein the graph database includes a storage node cluster, the storage node cluster includes at least two storage nodes, each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partition structure.

[0006] Secondly, embodiments of this application provide a graph data processing apparatus, the apparatus comprising: a receiving module for receiving an access request; and a first determining module for determining a target region from a storage node cluster based on the access request, so as to store the target graph data in the access request in the target region, or to read graph data satisfying the query conditions in the access request from the target region; wherein the graph database includes a storage node cluster, the storage node cluster includes at least two storage nodes, each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partition structure.

[0007] Thirdly, embodiments of this application provide an electronic device, which includes a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement the above-described method.

[0008] Fourthly, embodiments of this application provide a computer storage medium storing executable instructions for inducing a processor to execute the above-described method.

[0009] In this embodiment, based on the access request, a target region is determined from the storage node cluster. The graph database includes a storage node cluster, which comprises at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partitioning structure. Thus, on the one hand, by storing the graph data in linear partitions, the partitions can be quickly calculated and the graph data can be written quickly, improving the efficiency of writing graph data in the graph database. On the other hand, by determining the target region in the storage node cluster, the communication between nodes when reading graph data can be reduced based on the partitioning strategy, improving the efficiency of reading graph data in the graph database. Attached Figure Description

[0010] Figure 1 This provides a graph data storage model in related technologies;

[0011] Figure 2 This application provides a schematic diagram of interpersonal relationship graphs in daily life.

[0012] Figure 3 A schematic diagram of a graph database system provided in this application embodiment;

[0013] Figure 4A This is a schematic diagram of a graph data processing method provided in an embodiment of this application;

[0014] Figure 4B This is a schematic diagram of a graph data processing method provided in an embodiment of this application;

[0015] Figure 5A This is a schematic diagram of a graph data processing method provided in an embodiment of this application;

[0016] Figure 5B This is a schematic diagram of a graph data processing method provided in an embodiment of this application;

[0017] Figure 6 This is a schematic diagram of a graph data processing method provided in an embodiment of this application;

[0018] Figure 7A This is a schematic diagram of a graph data processing method provided in an embodiment of this application;

[0019] Figure 7B This is a schematic diagram of a graph data processing method provided in an embodiment of this application;

[0020] Figure 8 This application provides a schematic diagram of the composition structure of a graph data processing device according to an embodiment of the present application.

[0021] Figure 9 This is a schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of the invention will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0023] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0024] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that some embodiments described herein are merely used to explain the technical solutions of this application and are not intended to limit the technical scope of this application.

[0027] In recent years, with the information explosion and the rapid development of social networks, the application demand for graph databases has expanded dramatically to various production fields. However, there are currently few implementation solutions for graph databases. Well-known and widely used open-source databases include Neo4j and Baidu's HugeGraph, while commercial products include Alibaba's Graph Database (GDB) cloud database. Current industry solutions for graph data include:

[0028] (1) The graph data model is used to split and store the graph data and create an index. The split storage includes entities, entity relationships, labels, attributes, etc.

[0029] (2) Based on single-machine or distributed storage, adopt a non-relational KV storage structure and serialize and store data according to a custom serialization method;

[0030] The existing technology has the following drawbacks:

[0031] (1) Since graph data partitioning requires a large computational overhead, most traditional graph databases are single-machine solutions (such as Neo4j) or single-machine hot standby (such as Alibaba's GDB), which have certain limitations in terms of usage scenarios and cannot meet the application needs of large-scale data.

[0032] (2) Other distributed solutions, such as Baidu's HugeGraph and the open-source JanusGraph, although they implement distributed storage and can support mainstream distributed storage such as Cassandra and HBase, are limited in read and write performance due to their respective design schemes. JanusGraph suffers from severe performance jitter during large-scale data query operations due to the lack of a storage sharding strategy; HugeGraph has a built-in partitioning strategy, which can reduce inter-node communication during queries, but it also results in a slower write speed.

[0033] (3) In the data writing method of related technologies, nodes and relationships are relatively independent. For example, nodes must be created first, then relationships must be created, and then the execution plan is analyzed by Structured Query Language (SQL). In this process, the steps of creating nodes and relationships take little time, but the matching operation is very time-consuming.

[0034] To address the aforementioned problems, this application provides a graph data processing method. To aid in understanding the technical solutions of this application, the terms and concepts used in the embodiments are explained below. The terms and concepts used in the embodiments of this application are subject to the following interpretations:

[0035] I. Graph Database

[0036] Graph databases are databases used to store, retrieve, and analyze graph data. Based on graph theory, they are non-relational databases that store information about entities and the relationships between them. They are suitable for many scenarios requiring the recording of complex relationships, such as social networks, knowledge graphs, and resource management.

[0037] 1. Schema of a Graph Database: A schema can be used to define the indexes of a graph database. By creating a schema, a graph database can improve query performance.

[0038] 2. Graph Database Indexes: Used to improve query performance. Graph databases create indexes on one or more attributes of graph nodes. After the indexes are created, when the graph data is updated, the graph database automatically updates the indexes based on the updated content. When querying indexed attributes, the graph database automatically applies the indexes to improve query performance.

[0039] 3. Metadata Information of the Graph Database: This describes the graph data stored in the graph database. For example, in a personal information registration form, the name, gender, and education level entered are metadata information. The different values ​​entered by different users constitute graph data. When user A fills out the personal information registration form, their name is A, gender is B, and education level is C; the data (A, B, C) constitutes graph data.

[0040] 4. Graph Data Storage Model

[0041] Figure 1 This provides a graph data storage model in related technologies, such as Figure 1 As shown, a graph 10 records two types of data: nodes 11 and relations 12; among them,

[0042] Node 11 is used to describe an object in real life; Node 11 includes labels and attributes 13; the labels are used to group Node 11; the attributes are used to describe the characteristics of Node 11;

[0043] Relationship 12 is used to associate two identical or different nodes 11; Relationship 12 includes attribute 13 and relationship type; Relationship type is used to group relationships.

[0044] For example, as shown in Table 1:

[0045] Table 1 provides a node data storage format in the application embodiments.

[0046]

[0047] As shown in Table 1 above, each node record contains five parts, among which,

[0048] "In Use" indicates the status of this point, that is, whether the data at this point is available.

[0049] Label Count is used to calculate the specific length of a Node Entry, i.e., the number of Labels that identify that point; together with the Labels, it defines all the labels for that point.

[0050] Next Relation ID is a pointer used to identify the first relation, which is used for graph search queries.

[0051] Next Property ID is a pointer to the first property, which is used for graph search queries;

[0052] Label is a tag used to store the node;

[0053] Extra is used to store additional information about the node, and can provide a fixed length of additional information.

[0054] Table 2. A relational data storage format provided in the embodiments of this application.

[0055]

[0056] As shown in Table 2 above, the records of the relations use a fixed-length structure, where,

[0057] "In Use" indicates the current state of the relationship, that is, whether the relationship data is available.

[0058] First Node ID, used to point to the starting node, that is, a pointer that identifies the starting node of this relationship;

[0059] Second Node ID, used to point to the end node, that is, a pointer that identifies the end node of this relationship;

[0060] Relation Type is used to identify the type of relationship;

[0061] First Previous Relation ID, a pointer used to identify the previous relationship of the starting node;

[0062] First Next Relation ID, a pointer used to identify the next relationship after the starting node;

[0063] Second Previous Relation ID, a pointer used to identify the previous relationship of the ending node;

[0064] Second Next Relation ID, a pointer used to identify the next relationship after the end node;

[0065] Next Property ID, a pointer to the first property of the relationship.

[0066] Here, after the Relation Type, the IDs of the preceding and following relationships of the starting node and the ending node are recorded sequentially. Finally, the Next Property ID points to the first attribute of the relationship. Since the attributes are recorded separately in a linked list, it can be guaranteed that all attributes of the relationship can be traversed based on the relationship.

[0067] Table 3. An attribute storage structure provided in the embodiments of this application.

[0068]

[0069] As shown in Table 3 above, the attribute storage uses a doubly linked list structure, where...

[0070] "In Use" indicates the status of an attribute, that is, whether the attribute data is available.

[0071] Previous Property ID, a pointer used to identify the previous property;

[0072] Next Property ID, a pointer used to identify the next property;

[0073] Count is used to identify the number of fixed-length storage units (Blocks) occupied;

[0074] A block is used to identify a fixed-length storage unit.

[0075] Here, the Previous Property ID and Next Property ID point to the preceding and following properties, respectively. Considering that the storage length of different property types is also different, multiple fixed-length Blocks (e.g., fixed to one byte) plus a Count identifier are used to determine the storage length of a property.

[0076] For example, Figure 2 This application provides a schematic diagram of interpersonal relationship graphs in daily life, such as... Figure 2 As shown, the schematic diagram includes:

[0077] All node data is constructed based on the individual person 21. Taking Bob as an example, the data storage format of this node is as follows:

[0078] [In use]:1, [Label Count]:1, [Next Relation ID]:[Pointing to relationship Bob—>Alice], [Next Property ID]:[Pointing to property-gender], [Label]:[Student].

[0079] All relational data is constructed based on the relationships between people.22. Taking the relationship [Bob—>Alice] as an example, its data storage format is as follows:

[0080] [In Use]:1, [First Node ID]:[Points to node Bob], [Second Node ID]:[Points to node Alice], [Relation Type]:[Friend], [First Previous Relation ID]:null, [First Next Relation ID]:null, [Second Previous Relation ID]:[Points to relationship Jack -> Alice], [Second Next Relation ID]:[Points to relationship Alice -> Lucy], [Next Property ID]:[Points to property - time of acquaintance].

[0081] 23. Construct all attribute data based on the attribute list. Figure 2 Taking the attribute data of gender in a record as an example, the storage format for the node's attributes is as follows:

[0082] [In Use]:1, [Previous Property ID]:null, [Next Property ID]:[points to the next property - age], [Count]:1, [Block]:[Male].

[0083] During implementation, nodes, relationships, and attributes can be established simultaneously. Once the graph data is constructed and stored, graph traversal can be easily achieved. For example, the traversal process can be as follows: starting from node Bob, all attributes of this node can be traversed using the NextProperty ID field; the Next Relation ID field can be used to find a relationship [Bob—>Alice] starting from node Bob; and the Second Node ID field of this relationship data can be used to find node Alice. Repeating the above process completes the traversal of all stored information in the entire graph data.

[0084] II. Relevant Calculations

[0085] Modulo operation: a%p (or a mod p), which represents the remainder when a is divided by p.

[0086] III. Access Request

[0087] 1. The access request message structure includes a request header and request data.

[0088] 2. The request header includes the source IP address and user ID.

[0089] 3. The requested data includes the data to be processed in the access request. For example, the query conditions when querying the target data, or the storage location information when storing the target data.

[0090] Before introducing the graph data processing method provided in the embodiments of this application, the graph database in which the graph data processing method in the embodiments of this application is applied will be introduced. Figure 3 This application provides a schematic diagram of a graph database system, as shown in the embodiments below. Figure 3 As shown, the graph database system includes: a service layer, a computing layer, and a storage layer, wherein:

[0091] The service layer is used to provide user service interfaces, perform resource scheduling, and manage metadata.

[0092] The computing layer is used to execute read / write requests, index sharding, and data partitioning;

[0093] The storage layer is used for persistent storage of graph data.

[0094] It should be understood that users access the graph database through the user service interface of the service layer. In one possible implementation, the service layer includes an access API, an infrastructure manager, a metadata storage component, and a processor, wherein:

[0095] The access interface is used to receive access requests to the graph database;

[0096] The manager is used to expand computing nodes and manage resources;

[0097] The metadata storage component is used to manage the schema of the graph data;

[0098] The processor is used to fragment the access request and send the fragmented access request to different computing nodes in the computing layer.

[0099] It should be understood that the computing node is a node in the computing layer used to respond to the access request.

[0100] In one possible implementation, the manager manages resources, including: implementing cloud service-based resource management based on a containerized and automated operations and maintenance (DevOps) platform. For example, when scaling conditions are met, the manager scales up compute nodes based on user-defined policies. The user-defined policies can be elastic policies for modifying compute nodes. In another possible implementation, compute nodes are scaled up using native cloud service tools such as portable container orchestration management tools (Kubernetes, k8s).

[0101] In one feasible approach, a schema can be used to define indexes for graph data, such as the index schema and indexed fields, and other metadata information.

[0102] In one possible implementation, the processor fragments the access request based on a request-to-redirect (R / W Redirect) mechanism.

[0103] In one possible implementation, the sharding of the access request based on the request sharding mechanism includes: taking the modulo of the number of nodes in the computing node cluster (Brokers) based on the request parameters in the access request to obtain the target computing node; the request parameters include at least one of the following: user ID and source IP address.

[0104] For example, the Broker ID is obtained by taking the modulo of the user ID of the access request with the number of nodes in the Brokers cluster. If the user ID is 37 and the cluster consists of 6 compute nodes, taking the modulo of the user ID with the total number of compute nodes (37 divided by 6) results in a remainder of 1. Therefore, 1 represents the result of taking the modulo of the user ID with the total number of compute nodes. The access request for user ID 37 is then processed by the compute node with Broker ID 1.

[0105] For example, the BrokerID is obtained by taking the modulo of the source IP address of the access request with the number of nodes in the Brokers cluster. If the source IP address is 50, and the cluster consists of 6 compute nodes, taking the modulo of the user ID with the total number of compute nodes (50 divided by 6, resulting in a remainder of 2), then 2 represents the result of taking the modulo of the source IP address with the total number of compute nodes. The access request with source IP address 50 is then processed by the compute node with BrokerID 2.

[0106] In one possible implementation, the target computing node stores a preset number of data entries in its cache; the data is read from the target computing node more than a preset number of times.

[0107] For example, the target compute node (broker) caches a preset number of recently accessed hot data entries. Since when processing a read-type access request, the data related to the access request is first searched in the target compute node's cache, storing the preset number of recently accessed hot data entries in the target compute node's cache can reduce the probability of cache misses and improve data retrieval efficiency.

[0108] The request sharding mechanism described above allows a single compute node (broker) to respond to requests from the same user or from the same source IP address. This enables the rapid retrieval of data for both read and write access requests, improving data query efficiency.

[0109] In one possible implementation, the computing layer includes at least two computing nodes, each of which is used to perform data read and write operations based on the access request to obtain operation results; in order to process the access request.

[0110] For example, the computing layer includes multiple unified Brokers service nodes (computing nodes) responsible for performing data read and write operations.

[0111] In one possible implementation, the step of performing data read / write operations based on the access request to obtain an operation result includes: if the request parameters include target data, determining the metadata information of the target data based on the target data in the access request; taking the modulo of the number of nodes in the computing node cluster based on the encoding of the metadata information to obtain the target computing node; distributing the access request to the target computing node; and having the target computing node perform read / write operations on the target data to obtain an operation result in order to process the access request.

[0112] For example, each computing node in the computing layer performs data read and write operations, primarily by sharding data based on metadata information provided by the service layer. For instance, sharding may be based on index fields.

[0113] Graph databases encode the nodes and relationships of graph data (record IDs) and create indexes based on the encoding. For example, the encoded fields can be determined as index fields.

[0114] After sharding, access requests for different index fields are sent to the corresponding target compute nodes, which then process the access requests.

[0115] In the above, the use of sharding ensures that access requests for different index fields are sent to different compute nodes for processing, improving request efficiency. Furthermore, since no storage operations are involved during the computation on the target node, computation and storage are separated. Therefore, user-defined index fields do not affect the stored procedures. This allows for user-defined index fields and multiple indexing methods, with compute nodes calculating and retaining index information based on the user-configured graph database schema, providing multi-modal query capabilities.

[0116] In one possible implementation, when the access request is of the read type, the compute node is also used to read the target data based on a preset set of search algorithms; the preset set of search algorithms includes at least one of the following search algorithms: Dijkstra's algorithm and A* (A-Star) algorithm. It should be understood that Dijkstra's algorithm and A-Star algorithm are commonly used optimal path algorithms in the process of searching for target data in graph data.

[0117] In one possible implementation, the computing node includes a search query engine for managing the preset set of search algorithms.

[0118] In the above content, the target data is read based on a preset set of search algorithms. When handling high-concurrency read and write requests, the read and write efficiency of the graph database can be improved by expanding multiple computing nodes that include the preset set of search algorithms. The search algorithm can determine the shortest path to read the target data.

[0119] In one possible implementation, the computing node further includes a graph sharding engine, used to linearly partition the data in the graph data using the encoding of the nodes or the encoding of the relationships in the graph data. This allows the computing node responsible for handling write-type access requests to be determined based on the computed partitions, thus completing the data write operation.

[0120] In one possible implementation, the graph sharding engine includes a shuffle component for partitioning and storing cold data using edge-cutting or point-cutting methods based on a pre-defined period.

[0121] The storage layer is used for merging and storing graph data. In one possible implementation, the storage layer includes a GSM Storage Engine. The GSM Storage Engine is used to provide merged and stored graph structures for a cluster of multiple GraphKeeper instances.

[0122] It should be understood that for a cluster consisting of multiple Graph Keepers, any two Graph Keepers communicate via the Raft protocol. First, the physical nodes corresponding to data shards are mapped to Regions, which are managed by the ShardDaemon (SD) service. Each SD manages multiple Regions, and SDs use the Raft protocol to back up GSM and WAL files, maintaining data consistency and disaster recovery. Replicas are managed on a Region-by-Region basis. Multiple Regions on different nodes constitute a protocol cluster (Raft Group), where multiple Regions serve as replicas of each other, enabling the migration of the Raft Group Leader. When an SD resumes service, it initializes the index offsets in memory by reading the latest GSM file and simultaneously loads the latest WAL file to initialize the cache data.

[0123] In one possible implementation, the data stored in the Graph Keeper is divided into multiple shards. In another possible implementation, each shard includes a cache, a GSM file, and a compactor component. Each shard corresponds to a region, and each region is a physical node of a shard.

[0124] In one possible implementation, multiple fragments within the same Region share a single write-ahead logging (WAL) file, which records cached data. This allows the system to recover data not yet written to the GSM file upon restart, using the cached data recorded in the WAL file.

[0125] based on Figure 3 The graph database system shown in this application provides a graph data processing method. Figure 4A This is a schematic diagram of a graph data processing method provided in an embodiment of this application, such as... Figure 4A As shown, the method includes:

[0126] Step S401: Receive access request;

[0127] Here, the access request can include at least one of the following: a storage request and a read request. The access request can be in the form of at least an HTTP request.

[0128] Step S402: Based on the access request, determine the target region from the storage node cluster, so as to store the target graph data in the access request in the target region, or read the graph data that meets the query conditions in the access request from the target region;

[0129] The graph database includes a storage node cluster, which includes at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partition structure.

[0130] Here, the linear partitioning structure uses the encoding of nodes in the graph data, or the encoding of relationships in the graph data, to linearly partition and store the data in the graph data. This allows the system to determine the computational node responsible for handling write-type access requests based on the calculated partitions, thus completing the data writing process.

[0131] For example, Figure 4B This is a schematic diagram of a graph data processing method provided in an embodiment of this application, such as... Figure 4B As shown, the method includes: step S410, generating a primary key based on the encoding of nodes in the graph data, or the encoding of relationships between nodes in the graph data; step S420, determining a target region based on the primary key and a pre-defined partitioning strategy; step S430, determining the position of an index in the target region based on the position of the target region; and step S440, establishing a write stream for the graph data, or establishing a read stream for the graph data, based on the index position.

[0132] In this embodiment, based on the access request, a target region is determined from the storage node cluster. The graph database includes a storage node cluster, which comprises at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partitioning structure. Thus, on the one hand, by storing the graph data in linear partitions, the partitions can be quickly calculated and the graph data can be written quickly, improving the efficiency of writing graph data in the graph database. On the other hand, by determining the target region in the storage node cluster, the communication between nodes when reading graph data can be reduced based on the partitioning strategy, improving the efficiency of reading graph data in the graph database.

[0133] based on Figure 3 The graph database system shown in this application provides a graph data processing method. Figure 5AThis is a schematic diagram of a graph data processing method provided in an embodiment of this application, such as... Figure 5A As shown, the method includes:

[0134] Step S501: Receive access request;

[0135] Step S502: Based on the request header of the access request, determine the target computing node from the computing node cluster;

[0136] Step S503: Based on the request body of the access request, using the target computing node, determine the target region from the storage node cluster, so as to store the target graph data in the access request in the target region, or read the graph data that meets the query conditions in the access request from the target region.

[0137] The graph database includes a storage node cluster, which includes at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partition structure. The graph database also includes a computing node cluster, which includes at least two computing nodes.

[0138] In one possible implementation, the access request includes a read request, and the method further includes: if the target compute node's cache includes graph data that satisfies the query conditions in the read request, determining the graph data as a response to the read request; if the target compute node's cache does not include graph data that satisfies the query conditions in the read request, determining a target region from the storage node cluster based on the request body of the read request using the target compute node; and determining graph data that satisfies the query conditions in the graph structure merge file of the target region.

[0139] In one feasible way Figure 5B This is a schematic diagram of a graph data processing method provided in an embodiment of this application, such as... Figure 5B As shown, the access request includes a read request, and the method includes:

[0140] Step S510: Determine whether the cache of the target computing node contains graph data that satisfies the query conditions;

[0141] Here, if the target computing node's cache includes graph data that satisfies the query conditions in the read request, step S520 is executed; here, if the target computing node's cache includes graph data that satisfies the query conditions in the read request, step S530 is executed.

[0142] Step S520: Determine the graph data as a response to the read request;

[0143] In one possible implementation, determining graph data that satisfies the query conditions in the graph structure merge file of the target region includes: step S530, reading the graph structure merge file in the target region; the graph structure merge file includes at least one target graph data; step S540, generating an index offset based on the graph structure merge file; the index offset is used to record the address of the entity stored in the at least one target graph data; step S550, determining the graph data that satisfies the query conditions in the graph structure merge file based on the index offset.

[0144] Here, step S530, reading the graph structure merge file in the target area, can be achieved through binary search.

[0145] In one possible implementation, the entity may include: a node, a relationship between nodes, or an attribute, where Entry is an array that stores actual values ​​according to their corresponding IDs.

[0146] In one possible implementation, step S550, which determines the graph data that satisfies the query conditions in the graph structure merge file based on the index offset, includes: step S551, determining the storage block that satisfies the query conditions in the graph structure merge file based on the index offset; and step S552, reading the data fields in the storage block to obtain the graph data that satisfies the query conditions.

[0147] In one possible implementation, the method further includes: expanding the number of computing nodes in the computing node cluster based on a pre-defined expansion strategy; and / or expanding the number of storage nodes in the storage node cluster based on the pre-defined expansion strategy.

[0148] In one possible implementation, when the access request is of the read type, the compute node is also used to read the target data based on a preset set of search algorithms; the preset set of search algorithms includes at least one of the following search algorithms: Dijkstra's algorithm and A* (A-Star) algorithm. It should be understood that Dijkstra's algorithm and A-Star algorithm are commonly used optimal path algorithms for searching target data in attribute graph data.

[0149] Here, the computing node includes a search query engine, which manages the preset set of search algorithms.

[0150] In this embodiment, on the one hand, by using the target computing node based on the request body of the access request, the target region is determined from the storage node cluster, thereby realizing the separation of computing and storage. After the computing node determines the target region of the target graph data in the storage node cluster, the graph data is operated on in the target region, which is beneficial for expanding the nodes of the computing node cluster and the storage node cluster and improving the reading and writing efficiency.

[0151] On the other hand, by including graph data that satisfies the query conditions in the read request in the cache of the target computing node, the graph data is determined as the response to the read request. In this way, since it can be searched in the cache of the target computing node first, the probability of cache miss can be reduced and the query efficiency of graph data can be improved.

[0152] On the other hand, the number of computing nodes in the computing node cluster can be expanded based on a pre-defined expansion strategy; and / or, the number of storage nodes in the storage node cluster can be expanded based on the pre-defined expansion strategy. In this way, based on the design scheme of separating computing and storage, the independent scalability of read and write services and storage nodes can be supported, and the data read and write efficiency can be improved by expanding computing nodes.

[0153] based on Figure 3 The graph database system shown in this application provides a graph data processing method. Figure 6 This is a schematic diagram of a graph data processing method provided in an embodiment of this application, such as... Figure 6 As shown, the method includes:

[0154] Step S601: Receive access request;

[0155] Step S602: If the request header includes the user code ID, the user ID is modulo the number of nodes in the computing node cluster to obtain the target computing node; the request header includes: user code ID and source Internet Protocol IP address;

[0156] Step S603: If the request header includes the source IP address, the source IP address is modulo the number of nodes in the computing node cluster to obtain the target computing node;

[0157] Step S604: Based on the request body of the access request, using the target computing node, determine the target region from the storage node cluster, so as to store the target graph data in the access request in the target region, or read the graph data that meets the query conditions in the access request from the target region.

[0158] The graph database includes a storage node cluster, which includes at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partition structure. The graph database also includes a computing node cluster, which includes at least two computing nodes.

[0159] In one possible implementation, step S604, based on the request body of the access request, determines the target region from the storage node cluster using the target computing node, including:

[0160] S6041, Based on the target graph data in the request body, determine the metadata information of the target graph data;

[0161] S6042, Based on the metadata information, generate a record encoding ID using the target computing node;

[0162] S6043, Based on the record ID, the target region is determined from the storage node cluster using the target computing node.

[0163] In one feasible approach, the computation process at the target computing node employs a distributed hash algorithm. A hash value space [0~2] is used. 32 The data is distributed on a ring; each storage node is assigned a different weight based on performance conditions; the position of the storage node on the hash ring is determined according to the weight; in this way, the entire hash value space can be divided; based on the division result, each storage node is responsible for an adjacent data interval. Based on this data interval, the target region is determined. In one possible implementation, when adding or deleting storage nodes, only the data distribution of adjacent storage nodes needs to be recalculated, without recalculating the partitioning of the entire dataset.

[0164] In this embodiment, on one hand, if the request header includes the user code ID, the target computing node is obtained by taking the modulo of the user ID with the number of nodes in the computing node cluster; or, if the request header includes the source IP address, the target computing node is obtained by taking the modulo of the source IP address with the number of nodes in the computing node cluster. This allows for request sharding (R / WRedirect) processing, enabling the use of a specified target computing node to respond to the same user request.

[0165] On the other hand, based on the metadata information, a record encoding ID is generated using the target computing node; based on the record ID, a target region is determined from the storage node cluster using the target computing node. In this way, record IDs can be generated based on metadata information, and the system internally calculates shards based on the record IDs by default.

[0166] based on Figure 3 The graph database system shown in this application provides a graph data processing method. Figure 7A This is a schematic diagram of a graph data processing method provided in an embodiment of this application, such as... Figure 7A As shown, the method includes:

[0167] Step S701: Receive storage request;

[0168] Step S702: Based on the storage request, determine the target region from the storage node cluster, so as to store the target graph data in the storage request in the target region, or read the graph data that meets the query conditions in the storage request from the target region;

[0169] Step S703: Write the target graph data in the access request into the log file and the cache of the target region; the cache and the log file store the same graph data;

[0170] The graph database includes a storage node cluster, which includes at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partition structure.

[0171] In one possible implementation, the target graph data includes nodes and relationships between the nodes, where each relationship connects two nodes that share a common attribute. The method further includes step S704, in the cache, merging at least two nodes based on a pre-defined attribute in the common attribute.

[0172] In one possible implementation, the method further includes: step S705, when the number of target graph data in the cache reaches the persistence trigger condition, merging at least one target graph data in the cache to obtain merged graph data; step S706, storing the merged graph data in the graph structure merge file of the target region, so as to persistently store the merged graph data through the graph structure merge file.

[0173] Here, the merging process can be achieved through merge sort.

[0174] For example, such as Figure 3 As shown, steps S705 and S706 are implemented in the compression component.

[0175] In one possible implementation, the method further includes step S707, updating the index offset of the graph data after the graph data is stored.

[0176] In one possible implementation, each target region includes a graph structure merge file, and the method further includes: cutting the graph data in at least four graph structure merge files in the storage node cluster using an edge-cutting method or a point-cutting method based on a pre-set period; and storing the graph data that are related after cutting on the same storage node.

[0177] In one feasible way Figure 7B This is a schematic diagram of a graph data processing method provided in an embodiment of this application, such as... Figure 7B As shown, the method includes:

[0178] Step S710: Obtain the index position in the target area;

[0179] Step S720: Based on the index position, read the first position of the storage node from the target region;

[0180] Step S730: Based on the location of the storage node and a preset period, the graph data in at least four graph structure merged files in the storage node cluster is cut using the edge cutting method or the point cutting method; the graph data with correlation after cutting is stored in the second location of the storage node.

[0181] Step S740: Based on the second location, update the location of the storage node in the target area.

[0182] In this embodiment, on the one hand, the data in the cache is recorded through the WAL file to ensure that data that has not yet been written to the GSM file can be recovered through the file when the system restarts.

[0183] On the other hand, by merging at least two nodes in the cache based on the pre-defined attributes in the common attributes, the time for writing graph data can be saved by occupying storage space, thereby realizing real-time writing of graph data and solving the problem of low writing efficiency caused by the need to match relationships with nodes during the creation of relationships between nodes.

[0184] On the other hand, by storing the merged graph data in the graph structure merge file of the target area, the merged graph data can be persistently stored through the graph structure merge file. In this way, a large amount of randomly written graph data can be transformed into sequential writing, thereby improving the performance of data writing.

[0185] On the other hand, by using the edge-cutting method or the point-cutting method based on a pre-set period, the graph data in at least four graph structure merge files in the storage node cluster is cut; the graph data with correlation after cutting is stored on the same storage node. In this way, after storing the graph data, the stored graph data can be sorted out, which improves the writing efficiency and the physical continuity on the storage node.

[0186] Figure 3 This application provides a schematic diagram of a graph database system, as shown in the embodiments below. Figure 3 As shown, the graph database system includes:

[0187] The service layer is responsible for providing user service interfaces externally and implementing resource scheduling and metadata management internally. It can be implemented based on cloud services. The service layer includes an access API, an infrastructure manager, metadata storage, and a request sharding (R / WRedirect) mechanism.

[0188] The access interface is responsible for accepting unified database access requests;

[0189] The basic manager is responsible for configuring and managing compute resource scaling and load balancing strategies, implementing cloud-native resource management capabilities based on containerization and specific development platforms. For example, the system can modify the elasticity policy of compute nodes according to user settings, and automatically scale up the compute nodes once the conditions are met using cloud-native tools.

[0190] Metadata storage is responsible for the schema management of graph data. The schema can be used to define the index schema and index fields of graph data, as well as other metadata information.

[0191] When the service layer forwards a request to the computing layer, there is a request sharding mechanism. For example, the number of computing units in the computing unit (Broker) cluster in the computing layer can be moduloed based on the user identifier or the source Internet Protocol (IP) address of the request to obtain the Broker identifier. In this way, the specified Broker is used to respond to the same user request. The Broker will cache a preset number of recently operated hot data. Since read requests can first look up in the cache file in the computing layer, the probability of cache miss can be reduced and the data query efficiency can be improved.

[0192] The computation layer is responsible for executing read and write requests and implementing index sharding and data partitioning calculations. The computation layer includes at least one Broker and a Graph Sharding Engine.

[0193] The Broker is responsible for performing data read and write operations. It can perform index calculations and sharding based on metadata information provided by the service layer. For example, the computation layer can, by default, calculate shards and create indexes based on the record identifiers of graph data nodes and relationships. Due to its computation and storage separation design, it supports user-defined index patterns. The Broker calculates and retains the index information based on the user-configured schema, providing multi-pattern query capabilities. For read requests, the Broker also provides a Search Query Engine, configuring and managing data search methods, and supporting configurable search algorithms such as shortest path algorithms, Dijkstra's algorithm, and A* algorithm. Based on these design methods, database read and write efficiency can be improved by expanding the computation nodes when dealing with large-scale data and a large number of access requests.

[0194] The graph partitioning engine can use node or relation identifiers to linearly partition data by default. The advantage of this is that during write operations, partitions can be quickly calculated and data is written. To reduce inter-node communication when reading large-scale graph data in a distributed system, a periodic shuffle mechanism is built in, providing edge-cutting or point-cutting methods to repartition and store cold data from a certain period of time ago.

[0195] The storage layer is responsible for the persistence of graph data records. It includes at least one storage unit, namely the graph administrator, which includes at least one shard. Each shard includes a cache, index offset, graph structure merge file, and compression component. The physical nodes corresponding to the shards are mapped to the same region, and multiple shards in the same region share a single write-ahead logging (WAL).

[0196] The core of this database system is to cache a portion of the latest write requests in the memory of the computing layer. On the one hand, when writing data, nodes and relationships are created directly (the start and end points of the relationships have common attributes with the created nodes). Then, nodes are merged through a specific attribute. This is a space-for-time approach to write data in real time, solving the problem of low write efficiency caused by the matching operation required during the relationship creation process.

[0197] When the data cached in the computation layer reaches the trigger condition, the in-memory data is merged and appended to the storage layer cache using merge sort. This aims to convert a large number of random writes into sequential writes, thereby improving data write performance. The data stored in the graph administrator is divided into multiple fragments. Each fragment includes a cache, a graph structure merge file, a compression component, and a write-ahead log. The physical nodes corresponding to the fragments are mapped to the same region. Multiple fragments in the same region share a single write-ahead log. The write-ahead log records the data in the cache, ensuring that data not yet written to the graph structure merge file can be recovered during system restarts.

[0198] Table 4 shows a write-ahead log storage structure provided in the embodiments of this application.

[0199]

[0200] The storage structure of the write-ahead log is the same as that of the cache. Its purpose is to persist this part of the data in the cache. Since the data is inserted into the write-ahead log in the order of the index, the efficiency of batch writing to data fragments managed by the same fragment is very high.

[0201] The graph structure merge file is used to store database table data. The format of the graph structure merge file is shown in Table 5 below.

[0202] Table 5 shows the format of the merged file.

[0203] Header Blocks Index Footer

[0204] As shown in Table 5 above, the graph structure merged file consists of four parts: Header, Blocks, Index, and Footer. The Header is used to identify version information and reserved words, while the Blocks are used to record data content. The formats of the Header and Blocks are shown in Table 6 below.

[0205] Table 6 Format of Header and Blocks

[0206]

[0207] Blocks consist of a series of consecutive blocks. Each block is divided into two parts: a CRC32 value and data. The CRC32 value is used to verify the integrity of the data. The length of the data is recorded in the subsequent Index section. The data content includes the Entry ID and the actual Value. Since the Entry ID is an ordered value, only an offset is needed as an index to quickly locate the corresponding Block, facilitating data reading operations.

[0208] The data structure of the index is shown in Table 7 below. In order to quickly locate a certain block record, a two-level index is actually used. That is, the records in the index are first sorted according to the lexicographical order of the key, and then sorted according to the entry ID of the corresponding record.

[0209] Table 7 Data Structure of Index

[0210]

[0211] As shown in Table 7 above, in the data structure of Index,

[0212] Key Len specifies the length of the adjacent field Key;

[0213] The key definition is consistent with the cache definition;

[0214] Type is used to identify the type of data within Data in a Block;

[0215] Count indicates the number of indexes contained in this record segment;

[0216] The last four parts (Min Seq, Max Seq, Offset, and Size) are the block index information, which will repeat according to the count, and are sorted by Entry ID.

[0217] Min Seq is used to identify the smallest Entry ID in the Block;

[0218] Max Seq is used to identify the maximum Entry ID of data in a Block;

[0219] Offset is used to identify the offset of the corresponding Block in the entire graph structure merge file;

[0220] Size is used to record the size of the Block, so that the data of the corresponding Block can be quickly read based on the Offset and the Size field;

[0221] The Footer section at the end of the graph structure merge file stores Index Offsets, which is the offset of the starting position of the Index content within the graph structure merge file. This improves query efficiency and facilitates loading the index information into memory, thereby creating an indirect index.

[0222] See the data read request process. Figure 5B ,like Figure 5B As shown, the method includes:

[0223] During initialization, the index offsets are generated by reading the GSM file and loading it into memory. The Compactor needs to update this indirect index every time it persists the GSM file.

[0224] Each read request first checks the cache. If the data exists in memory, it is returned directly; otherwise, it is searched using an indirect index.

[0225] Since Index Offsets records the offset information of each key in the Index table of the GSM File, the key corresponding to the index is first obtained by binary search. Based on lexicographical comparison, the storage range of the corresponding block in the GSM File can be located. Since the blocks are arranged in order according to the Sequence Code, the corresponding block can be found by binary search, and the content of a block can be quickly read from the GSM File.

[0226] Based on the foregoing embodiments, this application provides a graph data processing device, which includes various modules, each module including sub-modules, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0227] Figure 8 This is a schematic diagram of the composition structure of a graph data processing device provided in an embodiment of this application, such as... Figure 8 As shown, the device 800 includes:

[0228] Receiver module 801 is used to receive access requests;

[0229] The first determining module 802 is used to determine a target region from the storage node cluster based on the access request, so as to store the target graph data in the access request in the target region, or to read graph data that meets the query conditions in the access request from the target region.

[0230] The graph database includes a storage node cluster, which includes at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partition structure.

[0231] In one possible implementation, the graph database further includes a computing node cluster, which includes at least two computing nodes. The first determining module 802 includes: a first determining submodule, used to determine a target computing node from the computing node cluster based on the request header of the access request; and a second determining submodule, used to determine a target region from the storage node cluster based on the request body of the access request and using the target computing node.

[0232] In one possible implementation, the request header includes: a user code ID and a source Internet Protocol (IP) address. The first determining submodule includes: a first modulo unit, configured to, when the request header includes the user code ID, take the user ID modulo the number of nodes in the computing node cluster to obtain a target computing node; and a second modulo unit, configured to, when the request header includes the source IP address, take the source IP address modulo the number of nodes in the computing node cluster to obtain a target computing node.

[0233] In one possible implementation, the second determining submodule based on the request body of the access request includes: a first determining unit, configured to determine metadata information of the target graph data based on the target graph data in the request body; a generating unit, configured to generate a record encoding ID using the target computing node based on the metadata information; and a second determining unit, configured to determine a target region from the storage node cluster based on the record ID using the target computing node.

[0234] In one possible implementation, the access request includes a storage request, and the apparatus further includes: a first storage module for writing target graph data from the storage request into a log file and a cache of the target region; the cache and the log file store the same graph data.

[0235] In one possible implementation, the target graph data includes nodes and relationships between the nodes, where each relationship connects two nodes that share a common attribute. The apparatus further includes a first merging module, configured to merge at least two nodes in the cache based on a pre-defined attribute among the common attributes.

[0236] In one possible implementation, the apparatus further includes: a second merging module, configured to merge at least one target graph data in the cache when the number of target graph data in the cache reaches the persistence trigger condition, to obtain merged graph data; and a second storage module, configured to store the merged graph data in the graph structure merge file of the target region, so as to persistently store the merged graph data through the graph structure merge file.

[0237] In one possible implementation, each target region includes a graph structure merge file, and the apparatus further includes: a cutting module for cutting graph data in at least four graph structure merge files in the storage node cluster using an edge-cutting method or a point-cutting method based on a pre-set period; and a third storage module for storing the graph data that are related after cutting on the same storage node.

[0238] In one possible implementation, the access request includes a read request, and the apparatus further includes: a second determining module, configured to determine the graph data as a response to the read request if the cache of the target computing node includes graph data that satisfies the query conditions in the read request; a third determining module, configured to determine a target region from the storage node cluster based on the request body of the read request, using the target computing node, if the cache of the target computing node does not include graph data that satisfies the query conditions; and a fourth determining module, configured to determine graph data that satisfies the query conditions in the graph structure merge file of the target region.

[0239] In one possible implementation, the apparatus further includes: an expansion module for expanding the number of computing nodes in the computing node cluster based on a pre-defined expansion strategy; and / or, expanding the number of storage nodes in the storage node cluster based on the pre-defined expansion strategy.

[0240] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0241] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause electronic devices (such as mobile phones, tablets, laptops, desktop computers, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0242] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the graph data processing method provided in the above embodiments.

[0243] Correspondingly, embodiments of this application provide an electronic device, Figure 9 A schematic diagram of a hardware entity of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the hardware entity of the device 900 includes a memory 901 and a processor 902. The memory 901 stores a computer program that can run on the processor 902. When the processor 902 executes the program, it implements the steps in the graph data processing method provided in the above embodiments.

[0244] The memory 901 is configured to store instructions and applications executable by the processor 902, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the processor 902 and various modules in the electronic device 900. It can be implemented by flash memory or random access memory (RAM).

[0245] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0246] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0247] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0248] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0249] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0250] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0251] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0252] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, laptop computer, desktop computer, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0253] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0254] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0255] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0256] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A graph data processing method, characterized in that, Applied to graph databases, the method includes: Receive access requests; Based on the access request, a target region is determined from the storage node cluster so as to store the target graph data in the access request in the target region, or to read graph data that meets the query conditions in the access request from the target region. The graph database includes a storage node cluster, which includes at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partitioning structure. The graph database also includes a computing node cluster, which includes at least two computing nodes. The step of determining the target region from the storage node cluster based on the access request includes: Based on the request header of the access request, the target computing node is determined from the computing node cluster; the request header includes: user code ID and source Internet Protocol IP address; Based on the request body of the access request, the target region is determined from the storage node cluster using the target computing node; The step of determining the target computing node from the computing node cluster based on the request header of the access request includes: If the request header includes the user-coded ID, the user-coded ID is moduloed by the number of nodes in the computing node cluster to obtain the target computing node; If the request header includes the source Internet Protocol (IP) address, the source IP address is modulo the number of nodes in the computing node cluster to obtain the target computing node.

2. The method as described in claim 1, characterized in that, The request body based on the access request, using the target computing node, determines the target region from the storage node cluster, including: Based on the target graph data in the request body, determine the metadata information of the target graph data; Based on the metadata information, a record encoding ID is generated using the target computing node; Based on the record encoding ID, the target region is determined from the storage node cluster using the target computing node.

3. The method as described in claim 1, characterized in that, The accessed requests include storage requests, and the method further includes: The target graph data in the storage request is written to the log file and the cache of the target region; the cache and the log file store the same graph data.

4. The method as described in claim 3, characterized in that, The target graph data includes nodes and the relationships between the nodes, where each relationship connects two nodes that share a common attribute. The method further includes: In the cache, at least two nodes are merged based on pre-defined attributes in the common attributes.

5. The method as described in claim 4, characterized in that, The method further includes: When the number of target graph data in the cache reaches the persistence trigger condition, at least one target graph data in the cache is merged to obtain merged graph data. The merged graph data is stored in the graph structure merge file of the target region, so as to persistently store the merged graph data through the graph structure merge file.

6. The method as described in claim 5, characterized in that, Each target region includes a graph structure merge file, and the method further includes: Based on a pre-set period, the graph data in at least four graph structure merge files in the storage node cluster is cut using the edge cutting method or the point cutting method. Graph data that are related after being cut are stored on the same storage node.

7. The method as described in claim 1, characterized in that, The access request includes a read request, and the method further includes: If the target computing node's cache contains graph data that satisfies the query conditions in the read request, then the graph data is determined as the response to the read request; If the target computing node's cache does not contain graph data that meets the query conditions in the read request, the target region is determined from the storage node cluster based on the request body of the read request, using the target computing node. Determine the graph data that meets the query conditions in the graph structure merge file of the target region.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Based on a pre-defined expansion strategy, the number of computing nodes in the computing node cluster is expanded; and / or, based on the pre-defined expansion strategy, the number of storage nodes in the storage node cluster is expanded.

9. A graph data processing apparatus, characterized in that, The device includes: The receiving module is used to receive access requests; The first determining module is used to determine a target region from the storage node cluster based on the access request, so as to store the target graph data in the access request in the target region, or to read graph data that meets the query conditions in the access request from the target region; The graph database includes a storage node cluster, which includes at least two storage nodes. Each storage node includes at least two regions, and the storage structure of the graph data in each region is a linear partitioning structure. The graph database also includes a computing node cluster, which includes at least two computing nodes. The first determining module is further configured to determine a target computing node from the computing node cluster based on the request header of the access request; the request header includes: user code ID and source Internet Protocol IP address; and determine a target region from the storage node cluster based on the request body of the access request using the target computing node. The first determining module is further configured to, when the request header includes the user code ID, take the user code ID modulo the number of nodes in the computing node cluster to obtain the target computing node; and when the request header includes the source Internet Protocol IP address, take the source Internet Protocol IP address modulo the number of nodes in the computing node cluster to obtain the target computing node.

10. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer storage medium, characterized in that, The device stores executable instructions for causing a processor to execute, thereby implementing the steps of the method according to any one of claims 1 to 8.

Images