Graph Data Storage Method, System and Computer Device

Abstract

The present invention discloses a graph data storage method, system and computer device, the method comprising: decomposing and processing graph data to be stored to obtain vertex identification data, attribute data and relationship identification data, wherein the vertex identification data comprises start vertex identification data and end vertex identification data; creating relationship bucket metadata comprising a plurality of relationship buckets, wherein each relationship bucket stores the graph data to be stored with the same relationship identification data in the form of a KV key; creating a vertex attribute set based on the vertex identification data and the attribute data and creating a relationship attribute set based on the attribute data and the relationship identification data; distributively storing the vertex attribute set, the relationship attribute set, the attribute inverted index and the relationship bucket metadata in a persistent memory medium; utilizing persistent memory and unique data to store only one copy of key-value data, storing the relationship network in buckets for easy horizontal expansion, directly obtaining key data by identification during query, making the query time independent of the size of the data set, and processing graph data storage and query more quickly.

Description

Technical Field

[0001] The present invention relates to the field of graph data processing, and in particular to a graph data storage method, system and computer equipment. Background Art

[0002] With the rapid development of industries like social networking, e-commerce, finance, retail, and the Internet of Things, a vast and complex web of relationships has emerged. Traditional databases struggle to handle relational operations, creating an urgent need for databases that support relational operations on massive amounts of complex data. This is where graph databases emerged. For any data of a certain size or value, graph databases are the best way to present and query this relational data.

[0003] Graph storage is a crucial part of graph databases, typically including native graph storage such as adjacency matrices or adjacency lists, and non-native graph storage such as JanusGraph. Native graph storage requires special customization and optimization, while non-native graph storage has varying degrees of read amplification and other consumption. Currently, the relationships between data that need to be processed in the big data industry are growing exponentially with the amount of data, and the requirements for efficiency are increasing day by day. Existing graph storage methods face huge challenges in breaking through graph data storage and query performance. Summary of the Invention

[0004] The purpose of the present invention is to provide a graph data storage method, system and computer equipment that can improve the efficiency of graph data storage and query.

[0005] The technical solution of the present invention is as follows: In a first aspect, the present invention provides a method for storing graph data, the method comprising:

[0006] Decomposing and processing the graph data to be stored to obtain vertex identification data, attribute data, and relationship identification data, wherein the vertex identification data includes start vertex identification data and end vertex identification data;

[0007] Creating relationship bucket metadata including a plurality of relationship buckets, each of the relationship buckets storing the graph data to be stored with the same relationship identification data in a KV key format;

[0008] Creating a vertex attribute set based on the vertex identification data and the attribute data and creating a relationship attribute set based on the attribute data and the relationship identification data;

[0009] The vertex attribute set, the relationship attribute set, the attribute inverted index, and the relationship bucket metadata are distributed and stored in a persistent memory medium.

[0010] In a preferred embodiment, before creating the relationship bucket metadata including a plurality of relationship buckets, and each of the relationship buckets storing the graph data to be stored having the same relationship identification data in the form of a KV key, the method further comprises:

[0011] KV key format data is constructed using the combination of the start vertex identification data and the relationship identification data as a key and the end vertex identification data as a value.

[0012] In a preferred embodiment, the step of creating relationship bucket metadata including a plurality of relationship buckets, wherein each relationship bucket stores the graph data to be stored with the same relationship identification data in the form of a KV key, includes:

[0013] Create a relationship bucket that stores the graph data to be stored with the same relationship identification data;

[0014] Establishing an ID for each of the relationship buckets and associating the ID of the relationship bucket with the relationship identification data of the graph data to be stored stored in the relationship bucket;

[0015] Create relationship bucket metadata including all the relationship buckets and store the ID-associated content of the relationship bucket in the relationship bucket metadata.

[0016] In a preferred embodiment, the creating a vertex attribute set based on the vertex identification data and the attribute data and the creating a relationship attribute set based on the attribute data and the relationship identification data includes:

[0017] Creating a vertex attribute set based on the vertex identification data and the attribute data to store the vertex identification data and the attribute data in a key-value model, wherein the vertex attribute set uses the vertex identification data as a key and the attribute data as a value;

[0018] A relationship attribute set is created based on the attribute data and the relationship identification data to store the attribute data and the relationship identification data in a key-value model, wherein the relationship attribute set uses the relationship bucket ID and the relationship identification data as keys and the attribute data as values.

[0019] In a preferred embodiment, the content associated with the relationship bucket ID includes at least: the relationship type, the amount of data corresponding to the relationship type, the IDs of all relationship buckets associated with the relationship type, and the positions of all relationship buckets associated with the relationship type.

[0020] In a preferred embodiment, after creating a vertex attribute set based on the vertex identification data and the attribute data and creating a relationship attribute set based on the attribute data and the relationship identification data, the method further comprises:

[0021] An attribute inverted index is created based on the vertex attribute set and the relationship attribute set.

[0022] In a preferred embodiment, the method further comprises:

[0023] Get the graph data to be written;

[0024] Decomposing the graph data to be written to obtain start vertex identification data of the graph data to be written, relationship identification data of the graph data to be written, attribute data of the graph data to be written, and end vertex identification data of the graph data to be written;

[0025] Searching for a corresponding target relationship bucket based on the relationship identification data;

[0026] Using the start vertex identification data of the graph data to be written and the relationship identification data corresponding to the relationship identification data of the graph data to be written as a key, and using the end vertex identification data of the graph data to be written as a value to construct KV key form data and write the data into the target relationship bucket;

[0027] The attribute data of the graph data to be written is written into the relationship attribute set.

[0028] In a preferred embodiment, the method further comprises:

[0029] Receive a graph data query request, wherein the graph data query request includes at least a query type, target query vertex identification data, and target relationship identification data;

[0030] Locating a target relationship bucket based on the target query vertex identification data and the target relationship identification data;

[0031] A target query result is obtained based on the query type, the target relationship vertex ID, and the target relationship bucket.

[0032] In a second aspect, the present invention further provides a graph data storage system, the system comprising:

[0033] a decomposition processing module, configured to decompose and process the graph data to be stored to obtain vertex identification data, attribute data, and relationship identification data, wherein the vertex identification data includes start vertex identification data and end vertex identification data;

[0034] A first creation module is configured to create relationship bucket metadata comprising a plurality of relationship buckets, each of which stores the graph data to be stored with the same relationship identification data in a KV key format;

[0035] A second creation module, configured to create a vertex attribute set based on the vertex identification data and the attribute data, and to create a relationship attribute set based on the attribute data and the relationship identification data;

[0036] A distributed storage module is used to distribute and store the vertex attribute set, the relationship attribute set, the attribute inverted index and the relationship bucket metadata in a persistent memory medium.

[0037] In a third aspect, the present invention further provides a computer device, comprising:

[0038] one or more processors; and

[0039] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute any method according to the first aspect.

[0040] The advantages of the present invention are: improving a graph data storage method, system and computer device, the method including: decomposing and processing the graph data to be stored to obtain vertex identification data, attribute data and relationship identification data, the vertex identification data including start vertex identification data and end vertex identification data; creating relationship bucket metadata containing several relationship buckets, each relationship bucket storing the graph data to be stored with the same relationship identification data in the form of KV keys; creating a vertex attribute set based on the vertex identification data and attribute data and creating a relationship attribute set based on the attribute data and relationship identification data; distributing and storing the vertex attribute set, relationship attribute set, attribute inverted index and relationship bucket metadata in a persistent memory medium; utilizing the DAX feature of persistent memory to directly store data of a specific structure, storing only one key-value data for unique data, storing the relationship network in buckets for easy horizontal expansion, and recursively iterating the data in the buckets. When querying, key data is directly obtained by identification, so that the time required for complex relationship network queries is independent of the size of the data set, and graph data storage and query can be processed at a faster speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 This is an architectural diagram for graph data storage in this application;

[0043] Figure 2 A flow chart of the graph data storage method provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the data structure stored in the relationship bucket in the graph data storage method provided in an embodiment of the present application;

[0045] Figure 4 A first flowchart of the graph data storage method provided in the embodiment of the present application for querying target graph data based on current storage content;

[0046] Figure 5A second flow chart of querying target graph data based on current storage content in the graph data storage method provided in an embodiment of the present application;

[0047] Figure 6 A third flow chart of the method for storing graph data in an embodiment of the present application, wherein the method queries target graph data based on the current stored content;

[0048] Figure 7 This is an architecture diagram of the graph data storage system provided in the embodiment of the present application;

[0049] Figure 8 This is a diagram of the computer device architecture provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] As described in the background, graph data, or graph data, is a data structure used to store and query data, rather than storing image data. Its data model is primarily based on nodes and relationships (edges). Current graph data storage typically uses native large-scale graph storage, such as adjacency matrices or adjacency lists, and non-native graph storage, such as JanusGraph. These storage methods require extensive I / O (computer data read and write operations) during query scanning, resulting in low query efficiency.

[0052] To solve the above problems, the present application proposes a graph data storage method, system, and computer device, which is a new graph storage data structure based on relationship buckets, wherein the storage of graph data mainly includes a vertex attribute set, a relationship attribute set, and several relationship buckets. The relationship bucket stores graph relationship data in a key-value structure with a start vertex identifier and a relationship identifier as the key and an end vertex identifier as the value. Unique data only stores one copy of the key-value data, and the relationship network is stored in buckets for easy horizontal expansion. The query time is independent of the size of the data set, and graph data storage and query can be processed more quickly. The database query tuning method, system, and computer device proposed in this application will be introduced in conjunction with specific embodiments below.

[0053] Example 1: This example introduces the architecture for storing graph data in this application.

[0054] Reference Figure 1As shown, the architecture includes: a persistent memory (PMem) storage medium, which distributes the storage of vertex attribute sets, relationship attribute sets, attribute inverted indexes, and relationship bucket metadata in the persistent memory storage medium. Among them, the relationship bucket metadata includes several relationship buckets, and a relationship bucket stores graph relationship data of the same relationship. When the amount of graph relationship data of a certain relationship is particularly large, it can be split into multiple sub-buckets for storage, and the relationship bucket metadata stores the split sub-bucket storage records. In addition, each relationship bucket has its own unique ID, and the relationship bucket metadata also stores the relationship bucket ID-related content. The relationship bucket ID-related content includes at least: the relationship type, the amount of data corresponding to the relationship type, the IDs of all relationship buckets associated with the relationship type, and the positions of all relationship buckets associated with the relationship type.

[0055] Example 2: Based on the architecture for storing graph data introduced in Example 1 above, this example provides a reference Figure 2 As shown, the process of the image data storage method in this application is introduced.

[0056] Specifically, refer to Figure 2 As shown, the process of performing graph data storage in the graph data storage method provided in this embodiment includes:

[0057] S210 , decompose and process the graph data to be stored to obtain vertex identification data, attribute data, and relationship identification data, where the vertex identification data includes start vertex identification data and end vertex identification data.

[0058] Specifically, the graph data to be stored typically includes two relationship entities and the relationship between them. For example, if a graph data item is "Xiao Wang is Xiao Ming's father," this graph data can be decomposed into the starting vertex identifier data (Xiao Wang), the ending vertex identifier data (Xiao Ming), the attribute data (Xiao Wang and Xiao Ming's attributes), and the relationship identifier data (the parent-child relationship identifier data). The relationship identifier data and the relationship content are generated in a preset and one-to-one correspondence. For example, relationship identifier data 0001 represents father and son, relationship identifier data 0002 represents friends, and relationship identifier data 0003 represents mother and child.

[0059] Preferably, before S220, the method includes:

[0060] SA1. Construct KV key format data using the combination of the start vertex identification data and the relationship identification data as the key and the end vertex identification data as the value.

[0061] Specifically, a key-value KV structure is constructed, where the key (K) is composed of the starting vertex identification data and the relationship identification data. The vertex identifier is the identification data that uniquely points to the vertex and can be regarded as the equivalent of an address pointer. The vertex attributes can be directly obtained, and conversely, the unique identifier can also be quickly obtained from the vertex set; the relationship identifier is a fixed-bit serial number belonging to the vertex that represents a certain relationship, and the value (V) is the ending vertex identifier.

[0062] More preferably, a fixed-length start vertex identifier and a fixed-length relationship identifier in the graph data are combined as the key, and the end vertex identifier is used as the value to construct the KV key form data. Dividing the KV key form data by relationship facilitates the subsequent distributed storage of graph relationship data of the same relationship.

[0063] S220 , creating relationship bucket metadata including several relationship buckets, where each relationship bucket stores the graph data to be stored with the same relationship identification data in the form of a KV key.

[0064] Preferably, this step includes:

[0065] S221. Create a relationship bucket that stores the graph data to be stored with the same relationship identification data.

[0066] Specifically, KV keyed data is divided by relationship type. Graph data of the same relationship is stored in a relationship bucket. That is, the relationship identifiers of the graph data in a relationship bucket are the same. The relationship identifiers of the graph data in a relationship bucket are determined by the type of the bucket. The design of the relationship bucket (RB, meaning relations bucket) is as follows:

[0067] RB=[K->V]=[(P1+Rs)->P2], where RB represents the relation bucket, K represents the key, V represents the value, P1 represents the start vertex identifier, Rs represents the relation identifier, and P2 represents the end vertex identifier. Figure 3 shown.

[0068] S222: Create an ID for each relationship bucket and associate the ID of the relationship bucket with the relationship identification data of the graph data to be stored stored in the relationship bucket.

[0069] S223: Create relationship bucket metadata including all relationship buckets and store the ID-related content of the relationship bucket in the relationship bucket metadata.

[0070] Specifically, the content associated with a relationship bucket ID includes at least the relationship type, the amount of data corresponding to the relationship type, the IDs of all relationship buckets associated with the relationship type, and the locations of all relationship buckets associated with the relationship type. In other words, the content represented by the relationship bucket ID is recorded in the relationship bucket metadata, including but not limited to the type of relationship represented by the relationship attribute set key, the total number of records of that type, the relationship buckets in which that type is distributed, and the location of the relationship buckets.

[0071] S230: Create a vertex attribute set based on the vertex identification data and the attribute data, and create a relationship attribute set based on the attribute data and the relationship identification data.

[0072] Specifically, the vertex identification data uniquely points to a vertex, and illustratively, the vertex identification data is a vertex ID. The vertex attribute set stores the vertex ID and attributes, and the relationship attribute set stores attributes and relationship IDs.

[0073] Preferably, this step includes:

[0074] S231 . Create a vertex attribute set based on the vertex identification data and the attribute data, storing the vertex identification data and the attribute data in a key-value model. The vertex attribute set uses the vertex identification data as a key and the attribute data as a value.

[0075] Specifically, the vertex attribute set uses the vertex ID as the key and the content is the persistent memory ID (PMEMoid) of the data object, which is usually 64 bits in size. The persistent memory ID can be easily converted into a memory pointer; the attribute value of the vertex attribute set is a separate label value and opaque binary data with an unfixed size.

[0076] S232: Create a relationship attribute set based on the attribute data and the relationship identification data, storing the attribute data and the relationship identification data in a key-value model. The relationship attribute set uses the relationship bucket ID and the relationship identification data as keys and the attribute data as values.

[0077] Specifically, the key in a relationship attribute set consists of a 16-bit relationship bucket ID and a 16-bit relationship identifier, for a total of 32 bits. The most significant bit of the relationship bucket ID, 0 or 1, indicates whether the relationship is unidirectional or bidirectional, such as parent-child or friend. The remaining 7 bits form a sequence number. The data structure of the relationship attribute set attribute values is similar to that of the vertex attribute set.

[0078] Preferably, after S230, the method further includes:

[0079] SA2. Create an attribute inverted index based on the vertex attribute set and the relationship attribute set.

[0080] Specifically, an inverted index of attributes is generated based on the data stored in the vertex attribute set and the relationship attribute set, so as to facilitate locating and searching the data in the vertex attribute set and the relationship attribute set, and to facilitate searching the attribute values in the vertex attribute set.

[0081] S240 , distributing and storing the vertex attribute set, the relationship attribute set, the attribute inverted index, and the relationship bucket metadata in a persistent memory medium.

[0082] Persistent Memory (PMem) is a new generation of storage media that is byte-addressable, has high read and write performance, and offers advantages over traditional DRAM (Dynamic Random Access Memory), such as non-volatility during power failure, high storage density, and low static power consumption.

[0083] Preferably, the method further comprises:

[0084] SA3. Write the graph data to be written based on the current storage content, including:

[0085] SA31. Obtain the graph data to be written.

[0086] SA32. Decompose and process the graph data to be written to obtain the starting vertex identification data of the graph data to be written to, the relationship identification data of the graph data to be written to, the attribute data of the graph data to be written to, and the ending vertex identification data of the graph data to be written to.

[0087] Exemplarily, the graph data to be written is the graph relationship data that Xiao Wang is Xiao Ming's father. The graph data to be written is decomposed and processed to obtain the starting vertex identification data of the graph data to be written - Xiao Wang ID, the relationship identification data of the graph data to be written - the identification of the father relationship 0000 0001, the attribute data of the graph data to be written - the year of becoming a father: 2020 and the ending vertex identification data of the graph data to be written - Xiao Ming ID.

[0088] For example, if Xiao Wang is Xiao Ming's father, first find the "Dad" relationship bucket, then write the key-value pair data such as (Xiao Wang ID + "00000001") -> (Xiao Ming ID), and at the same time record the specific attributes of this relationship in Rc. For example, the attribute of "Dad-0000 0001" is the year of becoming a father: 2020.

[0089] SA33. Search for the corresponding target relationship bucket based on the relationship identification data.

[0090] The ID of the relationship bucket is associated with the relationship identification data of the graph data stored in the relationship bucket, so the corresponding target relationship bucket can be quickly found through the relationship identification data.

[0091] SA34. Use the relationship identifier corresponding to the starting vertex identification data of the graph data to be written and the relationship identification data of the graph data to be written as the key, and the ending vertex identification data of the graph data to be written as the value to construct KV key form data and write it into the target relationship bucket.

[0092] Specifically, following the above example, a KV key-value pair data such as (Xiao Wang ID + "00000001")->(Xiao Ming ID) is constructed and written into the target relationship bucket with the ID association relationship identification data of "00000001".

[0093] SA35. Write the attribute data of the graph data to be written into the relationship attribute set.

[0094] Specifically, continuing with the above example, the specific attributes of the graph data to be written are recorded in the relationship attribute set. Since the attribute of "Dad-00000001" is the year of becoming a father: 2020, the content in the relationship attribute set is based on the relationship bucket ID and relationship identification data as keys and the attribute data as values. The ID of the target relationship bucket is A01, so the content written in the relationship attribute set is (00000001+A01)->(2020).

[0095] Preferably, the method further comprises:

[0096] SA4. Query target graph data based on current storage content, including:

[0097] SA41. Receive a graph data query request, which includes at least a query type, target query vertex identification data, and target relationship identification data.

[0098] For example, in a specific embodiment, referring to Figure 4 As shown, the query type is a second-degree query, the target query vertex data is the vertex ID of A, and the target relationship identification data is the identification data of the "friend" relationship, that is, the graph data query requests to check whether there is someone named X among the friends of A's friends.

[0099] SA42. Locate the target relationship bucket based on the target query vertex identification data and the target relationship identification data.

[0100] Continuing with the above example, specifically, the target relationship bucket storing the friend relationship graph data is queried and located according to the relationship identification data corresponding to the friend relationship.

[0101] SA43: Obtain a target query result based on the query type, the target relationship vertex ID, and the target relationship bucket.

[0102] Continuing with the above example, this step specifically includes searching the target relationship bucket for all vertex ID series B pointed to by relationships starting with A's vertex ID, then obtaining all vertex ID series C pointed to by series B from this bucket, and obtaining the name attribute corresponding to vertex series C from the vertex attribute set.

[0103] In another embodiment, referring to Figure 5 As shown, the query type is a statistical query. The target vertex identifier is A's vertex ID, and the target relationship identifier is the "friend" relationship identifier. In other words, the query is to find out how many friends A has. Based on the "friend" relationship identifier, the query locates the target relationship bucket storing the friend relationship graph data. Within the target relationship bucket, all relationship identifier sequences beginning with A's vertex ID are searched. The query result is the number of data in this sequence. If there is more than one relationship bucket, simply add the values together.

[0104] In another embodiment, referring to Figure 6 As shown, the query type is a call range query within a month. The target query vertex identifier is A's vertex ID, and the target relationship identifier is the identifier of the "call" relationship. In other words, the query is to find who A has called the most times in the past month. To do this, obtain the relationship bucket ID from the relationship bucket metadata. From the relationship attribute set, filter the relationship entries starting with the bucket ID. Then, from the "call" relationship bucket, select the vertex series H containing A's ID and all vertices in the T series. The most frequent occurrence of Y in the H series is the result.

[0105] The graph data storage method provided in this embodiment includes: decomposing and processing the graph data to be stored to obtain vertex identification data, attribute data and relationship identification data, where the vertex identification data includes start vertex identification data and end vertex identification data; creating relationship bucket metadata containing several relationship buckets, each relationship bucket storing the graph data to be stored with the same relationship identification data in the form of KV keys; creating a vertex attribute set based on the vertex identification data and attribute data and creating a relationship attribute set based on the attribute data and relationship identification data; distributing and storing the vertex attribute set, relationship attribute set, attribute inverted index and relationship bucket metadata in a persistent memory medium; utilizing the DAX feature of persistent memory to directly store data of a specific structure, storing only one key-value data for unique data, storing the relationship network in buckets for easy horizontal expansion, and recursively iterating the data in the buckets. When querying, key data is directly obtained by using the identifier, so that the time required for complex relationship network queries is independent of the size of the data set, and graph data storage and query can be processed at a faster speed.

[0106] Example 3: Corresponding to the above-mentioned Example 1 and Example 2, the following will be combined Figure 7, the graph data storage system provided by this application is introduced. The system can be implemented by hardware or software, or by a combination of hardware and software, which is not limited by this application.

[0107] In an example such as Figure 4 As shown, the present application provides a graph data storage system, the system comprising:

[0108] Decomposition processing module 710, used to decompose and process the graph data to be stored to obtain vertex identification data, attribute data and relationship identification data, wherein the vertex identification data includes start vertex identification data and end vertex identification data;

[0109] A first creation module 720 is configured to create relationship bucket metadata comprising a plurality of relationship buckets, each of which stores the graph data to be stored with the same relationship identification data in a KV key format;

[0110] A second creation module 730, configured to create a vertex attribute set based on the vertex identification data and the attribute data, and to create a relationship attribute set based on the attribute data and the relationship identification data;

[0111] The distributed storage module 740 is configured to distribute and store the vertex attribute set, the relationship attribute set, the attribute inverted index, and the relationship bucket metadata in a persistent memory medium.

[0112] In a specific embodiment, the system further comprises:

[0113] The data construction module 750 is used to create relationship bucket metadata containing several relationship buckets in the first creation module 720. Before each relationship bucket stores the graph data to be stored with the same relationship identification data in the form of KV keys, the KV key form data is constructed with the combination of the starting vertex identification data and the relationship identification data as the key and the ending vertex identification data as the value.

[0114] Preferably, the first creation module 720 includes:

[0115] The first creating unit 721 is configured to create a relationship bucket storing graph data to be stored that has the same relationship identification data;

[0116] An association unit 722 is configured to establish an ID for each of the relationship buckets and associate the ID of the relationship bucket with the relationship identification data of the graph data to be stored stored in the relationship bucket;

[0117] The second creating unit 723 is configured to create relationship bucket metadata including all the relationship buckets and store the ID-associated content of the relationship bucket in the relationship bucket metadata.

[0118] More preferably, the second creation module 730 includes:

[0119] A third creating unit 731 is configured to create, based on the vertex identification data and the attribute data, a vertex attribute set storing the vertex identification data and the attribute data in a key-value model, wherein the vertex attribute set uses the vertex identification data as a key and the attribute data as a value;

[0120] The fourth creation unit 732 is used to create a relationship attribute set based on the attribute data and relationship identification data, which stores the attribute data and relationship identification data in a key-value model, wherein the relationship attribute set uses the relationship bucket ID and relationship identification data as keys and the attribute data as values.

[0121] In a specific embodiment, the system further comprises:

[0122] The third creation module 760 is used to create an attribute inverted index based on the vertex attribute set and the relationship attribute set after the second creation module 730 creates a vertex attribute set based on the vertex identification data and attribute data and creates a relationship attribute set based on the attribute data and relationship identification data.

[0123] In a specific embodiment, the system further comprises:

[0124] The writing module 770 includes:

[0125] A first acquiring unit 771 is used to acquire the graph data to be written;

[0126] A decomposition unit 772 is configured to decompose the graph data to be written to obtain start vertex identification data of the graph data to be written, relationship identification data of the graph data to be written, attribute data of the graph data to be written, and end vertex identification data of the graph data to be written;

[0127] A search unit 773 is configured to search for a corresponding target relationship bucket based on the relationship identification data;

[0128] A first writing unit 774 is configured to construct KV key format data using the start vertex identification data of the graph data to be written and the relationship identification data corresponding to the relationship identification data of the graph data to be written as a key and the end vertex identification data of the graph data to be written as a value, and write the data into the target relationship bucket;

[0129] The second writing unit 775 is configured to write the attribute data of the graph data to be written into the relationship attribute set.

[0130] In a specific embodiment, the system further comprises:

[0131] The query module 780 includes:

[0132] The receiving unit 781 is configured to receive a graph data query request, wherein the graph data query request includes at least a query type, target query vertex identification data, and target relationship identification data;

[0133] A positioning unit 782 is configured to locate a target relationship bucket based on the target query vertex identification data and the target relationship identification data;

[0134] The second acquiring unit 783 is configured to acquire a target query result based on the query type, the target relationship vertex ID, and the target relationship bucket.

[0135] Example 4: Corresponding to the above examples 1 to 3, the following will be combined Figure 8 , the computer device provided by this application is introduced. In an example such as Figure 8 As shown, the present application provides a computer device, which includes:

[0136] one or more processors; and

[0137] A memory associated with the one or more processors, the memory being configured to store program instructions, wherein the program instructions, when read and executed by the one or more processors, perform the following operations:

[0138] Decomposing and processing the graph data to be stored to obtain vertex identification data, attribute data, and relationship identification data, wherein the vertex identification data includes start vertex identification data and end vertex identification data;

[0139] Creating relationship bucket metadata including a plurality of relationship buckets, each of the relationship buckets storing the graph data to be stored with the same relationship identification data in a KV key format;

[0140] Creating a vertex attribute set based on the vertex identification data and the attribute data and creating a relationship attribute set based on the attribute data and the relationship identification data;

[0141] The vertex attribute set, the relationship attribute set, the attribute inverted index, and the relationship bucket metadata are distributed and stored in a persistent memory medium.

[0142] in, Figure 8 The architecture of a computer device is exemplarily shown, and may specifically include a processor 810, a video display adapter 811, a disk drive 812, an input / output interface 813, a network interface 814, and a memory 820. The processor 810, the video display adapter 811, the disk drive 812, the input / output interface 813, the network interface 814, and the memory 820 may be communicatively connected via a communication bus 830.

[0143] Among them, the processor 810 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in this application.

[0144] The memory 820 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 820 can store an operating system 821 for controlling the operation of the computer device 800, and a basic input and output system (BIOS) 822 for controlling the low-level operations of the computer device 800. In addition, a web browser 823, a data storage management 824, and an icon font processing system 825, etc. can also be stored. The above-mentioned icon font processing system 825 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 820 and is called and executed by the processor 810.

[0145] The input / output interface 813 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0146] The network interface 814 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0147] The bus 830 comprises a pathway for transmitting information between the various components of the device (eg, the processor 810 , the video display adapter 811 , the disk drive 812 , the input / output interface 813 , the network interface 814 , and the memory 820 ).

[0148] In addition, the computer device 800 can also obtain information on specific collection conditions from the virtual resource object collection condition information database 841 for use in condition judgment, etc.

[0149] It should be noted that although the computer device 800 shown above only includes a processor 810, a video display adapter 811, a disk drive 812, an input / output interface 813, a network interface 814, a memory 820, a bus 830, etc., in a specific implementation, the computer device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0150] Through the description of the above implementation methods, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a cloud server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0151] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system embodiment described above is merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0152] It should also be noted that the terms "first" and "second" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. Of course, the above embodiments are only for illustrating the technical concept and features of the present invention. Its purpose is to enable people familiar with this technology to understand the content of the present invention and implement it accordingly, and it cannot be used to limit the scope of protection of the present invention. Any modifications made according to the spirit of the main technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A graph data storage method, characterized in that: The method comprises: Decomposing and processing the graph data to be stored to obtain vertex identification data, attribute data, and relationship identification data, wherein the vertex identification data includes start vertex identification data and end vertex identification data; Constructing KV key format data using the combination of the start vertex identification data and the relationship identification data as a key and the end vertex identification data as a value; Creating a relationship bucket metadata comprising a plurality of relationship buckets, each of the relationship buckets storing the graph data to be stored having the same relationship identification data in a KV key format, wherein one of the relationship buckets stores graph relationship data of the same relationship, the relationship bucket includes a plurality of sub-buckets, and a split sub-bucket storage record is stored in the relationship bucket metadata; Creating a vertex attribute set based on the vertex identification data and the attribute data and creating a relationship attribute set based on the attribute data and the relationship identification data; Creating an attribute inverted index based on the vertex attribute set and the relationship attribute set; The vertex attribute set, the relationship attribute set, the attribute inverted index, and the relationship bucket metadata are distributed and stored in a persistent memory medium.

2. The graph data storage method according to claim 1, characterized in that: The step of creating relationship bucket metadata including a plurality of relationship buckets, wherein each relationship bucket stores the graph data to be stored with the same relationship identification data in the form of a KV key, includes: Create a relationship bucket that stores the graph data to be stored with the same relationship identification data; Establishing an ID for each of the relationship buckets and associating the ID of the relationship bucket with the relationship identification data of the graph data to be stored stored in the relationship bucket; Create relationship bucket metadata including all the relationship buckets and store the ID-associated content of the relationship bucket in the relationship bucket metadata.

3. The graph data storage method according to claim 2, characterized in that: The step of creating a vertex attribute set based on the vertex identification data and the attribute data and creating a relationship attribute set based on the attribute data and the relationship identification data comprises: Creating a vertex attribute set based on the vertex identification data and the attribute data to store the vertex identification data and the attribute data in a key-value model, wherein the vertex attribute set uses the vertex identification data as a key and the attribute data as a value; A relationship attribute set is created based on the attribute data and the relationship identification data to store the attribute data and the relationship identification data in a key-value model, wherein the relationship attribute set uses the relationship bucket ID and the relationship identification data as keys and the attribute data as values.

4. The graph data storage method according to claim 2, characterized in that: The content associated with the relationship bucket ID includes at least: the relationship type, the amount of data corresponding to the relationship type, the IDs of all relationship buckets associated with the relationship type, and the positions of all relationship buckets associated with the relationship type.

5. The graph data storage method according to claim 1, characterized in that: The method further comprises: Get the graph data to be written; Decomposing the graph data to be written to obtain start vertex identification data of the graph data to be written, relationship identification data of the graph data to be written, attribute data of the graph data to be written, and end vertex identification data of the graph data to be written; Searching for a corresponding target relationship bucket based on the relationship identification data; Using the start vertex identification data of the graph data to be written and the relationship identification data corresponding to the relationship identification data of the graph data to be written as a key, and using the end vertex identification data of the graph data to be written as a value to construct KV key form data and write the data into the target relationship bucket; The attribute data of the graph data to be written is written into the relationship attribute set.

6. The graph data storage method according to claim 1, characterized in that: The method further comprises: Receive a graph data query request, wherein the graph data query request includes at least a query type, target query vertex identification data, and target relationship identification data; Locating a target relationship bucket based on the target query vertex identification data and the target relationship identification data; A target query result is obtained based on the query type, the target relationship vertex ID, and the target relationship bucket.

7. A graph data storage system, characterized in that: The system comprises: a decomposition processing module, configured to decompose and process the graph data to be stored to obtain vertex identification data, attribute data, and relationship identification data, wherein the vertex identification data includes start vertex identification data and end vertex identification data, and construct KV key format data using the combination of the start vertex identification data and the relationship identification data as a key and the end vertex identification data as a value; A first creation module is configured to create relationship bucket metadata comprising a plurality of relationship buckets, each of the relationship buckets storing the graph data to be stored having the same relationship identification data in a KV key format, wherein one relationship bucket stores graph relationship data of the same relationship, the relationship bucket includes a plurality of sub-buckets, and the relationship bucket metadata stores split sub-bucket storage records; A second creation module is configured to create a vertex attribute set based on the vertex identification data and the attribute data, create a relationship attribute set based on the attribute data and the relationship identification data, and create an attribute inverted index based on the vertex attribute set and the relationship attribute set; A distributed storage module is used to distribute and store the vertex attribute set, the relationship attribute set, the attribute inverted index and the relationship bucket metadata in a persistent memory medium.

8. A computer device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the method according to any one of claims 1 to 6 is executed.

Images