Graph database updating method and apparatus

Abstract

The application relates to a graph database updating method and device, equipment and a storage medium, and particularly relates to the field of databases. The method comprises the following steps: obtaining target data; the target data comprises a target primary key; performing a query operation on an in-memory database according to the target primary key, and when the query result indicates that the target primary key exists in the in-memory database, obtaining a target point data ID corresponding to the target data; the target point data ID is generated when the target data is imported into the graph database; judging whether the target point data ID exists in the graph database, and performing data updating on the graph database based on the judgment result and the target data. In the above scheme, the target primary key is set in the target data, and the correspondence between the in-memory database indicating the primary key and the point data ID is set; since the point data ID is an index generated by the graph database, the point data can be quickly queried through the point data ID, so that the updating speed of the graph database is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, in particular to a graph database updating method and device. BACKGROUND

[0002] The graph database is a database for efficiently processing complex relationship network based on the idea and algorithm of graph theory in mathematics. The graph database is good at efficiently processing a large amount of complex, interconnected and variable data, and the calculation efficiency is much higher than that of the traditional relational database.

[0003] When updating the graph database, whether it is point data update, deletion, query operation or various operations on edge data, the point data in the graph database needs to be queried. For example, when the edge data needs to be imported, the source vertex data and the target vertex data corresponding to the edge data are queried through the unique attribute value, so as to determine the position of the edge data to be written in the graph database.

[0004] In the above scheme, the efficiency of traversing and querying each point data in the graph database through the unique attribute value is low, which leads to low updating speed of the graph database. SUMMARY

[0005] The present application provides a graph database updating method, device, equipment and storage medium, which improves the updating speed of the graph database. The technical scheme is as follows.

[0006] On the one hand, a graph database updating method is provided, which comprises:

[0007] Obtaining target data; the target data contains a target primary key;

[0008] According to the target primary key, the memory database is queried, and when the query result indicates that the target primary key exists in the memory database, the target point data ID corresponding to the target data is obtained; the target point data ID is generated when the target data is imported into the graph database;

[0009] Judging whether the target point data ID exists in the graph database, and based on the judgment result and the target data, updating the data of the graph database.

[0010] On the other hand, a graph database updating device is provided, which comprises:

[0011] A target data obtaining unit is configured to obtain target data; the target data contains a target primary key;

[0012] The first query module is configured to perform a query in the in-memory database according to the target primary key, and when the query result indicates that the target primary key exists in the in-memory database, obtain a target point data ID corresponding to the target data; the target point data ID is generated when the target data is imported into the graph database.

[0013] The first update module is configured to determine whether the target point data ID exists in the graph database, and perform data update on the graph database based on the determination result and the target data.

[0014] In a possible implementation, the apparatus further includes:

[0015] The second query module is configured to perform a first query operation in the in-memory database according to the target primary key.

[0016] The third query module is configured to perform a second query operation in the graph database according to the target primary key when the result of the first query operation indicates that the target primary key does not exist in the in-memory database.

[0017] The correspondence relationship construction module is configured to, when the result of the second query operation indicates that data containing the target primary key exists in the graph database, import the target point data ID stored in the graph database into the in-memory database, to construct a correspondence relationship between the target primary key and the target point data ID.

[0018] In a possible implementation, the in-memory database contains Hash-type correspondence relationship data; the correspondence relationship data contains a type label value, a target primary key, and a point data ID.

[0019] In a possible implementation, the target primary key in the correspondence relationship data corresponds to a specified bit of an encrypted value; and the type label value includes the specified bit of the encrypted value of the target primary key in the correspondence relationship data.

[0020] In a possible implementation, the first update module includes:

[0021] The attribute information update unit is configured to, when the determination result indicates that the target point data ID exists in the graph database, update attribute information of point data containing the target point data ID in the graph database according to the target data.

[0022] The point data creation unit is configured to, when the determination result indicates that the target point data ID does not exist in the graph database, create point data in the graph database according to each attribute information contained in the target data.

[0023] In a possible implementation, the apparatus further includes:

[0024] The fourth query module performs a third query operation in the in-memory database according to a target primary key in the target data; the target data is point data;

[0025] The fifth query module is configured to perform a fourth query operation in the graph database according to the target primary key when a result of the third query operation indicates that the target primary key does not exist in the in-memory database.

[0026] The point data creation module is configured to create point data in the graph database according to each attribute information in the target data when a result of the fourth query operation indicates that point data containing the target primary key does not exist in the graph database.

[0027] In a possible implementation, when the target data is edge data, the target point data ID is used to indicate at least one of source vertex data and target vertex data corresponding to the target data; and the first update module further includes:

[0028] The edge data writing unit is configured to write the target data as edge data corresponding to point data containing the target point data ID in the graph database when the result of the judgment indicates that the target point data ID exists in the graph database.

[0029] The error information generation unit is configured to generate error information indicating an update exception of the graph database when the result of the judgment indicates that the target point data ID does not exist in the graph database.

[0030] In a possible implementation, the apparatus further includes:

[0031] The sixth query module performs a fifth query operation in the in-memory database according to a target primary key in the target data; the target data is edge data;

[0032] The seventh query module is configured to perform a sixth query operation in the graph database according to the target primary key when a result of the fifth query operation indicates that the target primary key does not exist in the in-memory database.

[0033] The error information generation module is configured to generate error information indicating an update exception of the graph database when a result of the sixth query operation indicates that point data containing the target primary key does not exist in the graph database.

[0034] In another aspect, a computer device is provided, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the above-mentioned graph database update method.

[0035] In yet another aspect, a computer-readable storage medium is provided, the storage medium having stored therein at least one instruction, the at least one instruction being loadable by a processor and executable by the processor to implement the graph database updating method described above.

[0036] In yet another aspect, a computer program product is provided, the computer program product comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the graph database updating method described above.

[0037] The technical solutions provided in the present application can include the following beneficial effects:

[0038] In the process of updating the graph database, the target data obtained contains a target primary key. When querying in the in-memory database according to the target primary key, the in-memory database contains the correspondence between the target point data ID generated when each data is imported into the graph database and the primary key of each data. Therefore, when the target data exists in the graph database, the target point data ID can be queried, and when the target data does not exist in the graph database, the target point data ID cannot be queried. At this time, according to the judgment result of whether the target point data ID exists and the target data, the operation of updating the graph database can be determined, thereby realizing the updating process of the graph database. The above scheme sets the target primary key in the target data and sets the correspondence between the in-memory database indicating the primary key and the point data ID. Since the point data ID is an index generated by the graph database, the point data can be quickly queried through the point data ID, thereby improving the updating speed of the graph database. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the specific embodiments or prior art of the present application, the drawings needed in the specific embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0040] Figure 1 is a structural schematic diagram of a graph database updating system according to an exemplary embodiment.

[0041] Figure 2 A graph database data storage schematic diagram related to an embodiment of the present application is shown.

[0042] Figure 3is a method flow chart of a graph database updating method according to an example embodiment.

[0043] Figure 4 A graph database operation flow chart is shown.

[0044] Figure 5 is a method flow chart of a graph database updating method according to an example embodiment.

[0045] Figure 6 A storage structure schematic diagram of a memory database related to the embodiments of the present application is shown.

[0046] Figure 7 A point data warehousing flow schematic diagram related to the embodiments of the present application is shown.

[0047] Figure 8 is a method flow chart of a graph database updating method according to an example embodiment.

[0048] Figure 9 A point data warehousing flow schematic diagram related to the embodiments of the present application is shown.

[0049] Figure 10 is a structure block diagram of a graph database updating apparatus according to an example embodiment.

[0050] Figure 11 is a computer device schematic diagram according to an example embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0052] It should be understood that the "indication" mentioned in the embodiments of the present application can be direct indication, indirect indication, or can be an indication of an associated relationship. For example, A indicates B, which can mean that B can be obtained through A; or it can mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; or it can mean that A and B have an associated relationship.

[0053] In the description of the embodiments of the present application, the term "corresponding" can mean that there is a direct or indirect corresponding relationship between the two, or it can mean that there is an associated relationship between the two, or it can mean an indication and being indicated, configuration and being configured, etc.

[0054] In the embodiments of this application, "predefined" can be achieved by pre-storing corresponding codes, tables or other means that can be used to indicate relevant information in the device (e.g., including terminal devices and network devices). This application does not limit the specific implementation method.

[0055] Before describing the various embodiments shown in this application, several concepts involved in this application will be introduced first.

[0056] 1) Graph Database

[0057] Graph databases, also known as graph databases, are a type of NoSQL database that uses graph theory to store information about relationships between entities. Graph databases are a type of non-relational database that uses graph theory to store information about relationships between entities. The most common example is the relationships between people in a social network. Relational databases are not very effective at storing "relational" data; their queries are complex, slow, and often exceed expectations. The unique design of graph databases precisely compensates for this deficiency. When representing many-to-many relationships, it is often necessary to create an association table to record the many-to-many relationships between different entities, and these association tables are often not used to record information. If two entities have multiple relationships, then multiple association tables need to be created between them. In a graph database, however, it is only necessary to indicate that different relationships exist between them. If you want to establish a bidirectional relationship between two sets of nodes, you need to define a relationship for each direction. That is to say, compared to the various association tables in relational databases, relationships in graph databases can provide a richer way of representing relationships through the ability of relationships to include attributes. Therefore, compared to relational databases, users of graph databases have an additional weapon when abstracting things: rich relationships.

[0058] 2) Primary key

[0059] A primary key is one or more fields in a table whose values ​​are used to uniquely identify a record in the table. In a relationship between two tables, the primary key is used to reference a specific record from one table to another. A primary key is a unique key and is part of the table definition. A table's primary key can consist of multiple keys, and the primary key column cannot contain null values.

[0060] Figure 1 This is a schematic diagram illustrating the structure of a graph database update system according to an exemplary embodiment. The graph database update system includes a server 110 and a terminal 120.

[0061] Optionally, the server 110 comprises a graph database and an in-memory database. For example, the graph database can be built based on JanusGraph, and the in-memory database can be built based on redis (Remote Dictionary Server).

[0062] Optionally, when the terminal 120 acquires target data that needs to be imported into the graph database, the terminal 120 can transmit the target data to the server 110 loaded with the graph database through a communication network. The server updates the graph database in the server 110 according to the target data according to a preset import process.

[0063] In the graph database, there are point data and attribute data corresponding to the point data, and edge data and attribute data corresponding to the edge data. Please refer to Figure 2 , which shows a graph database data storage diagram related to an embodiment of the present application. As shown in Figure 2 , in the graph database built based on JanusGraph, data is stored in a way of cutting edges around nodes. For example, in Hbase, the ID of a node is used as the Rowkey of HBase, and each attribute on the node and each edge are used as an independent Cell of the Rowkey row. As shown in Figure 2 , in the graph database, the storage of the graph is divided into three parts: vertex id (point data ID), property (attribute), and edge (edge). The vertex id is the unique id of the corresponding node, which represents the Rowkey of the current row if the underlying storage uses Hbase, and uniquely represents a node. The property represents the attribute of the node; and the edge represents the corresponding edge of the node.

[0064] Therefore, in the graph database as shown in Figure 2 , since an edge is used to connect two nodes, any edge needs to be stored in the storage part corresponding to the two nodes, that is, any edge is stored twice in the graph database. When querying edge data in the graph database, the storage space of the point data corresponding to the two nodes (source node and target node) of the edge data needs to be queried.

[0065] Optionally, the above-mentioned server can be a standalone physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms, etc. technical computing services.

[0066] Optionally, the system may also include a management device for managing the system (such as managing the connection status between each module and the server), and the management device is connected to the server via a communication network. Optionally, the communication network may be a wired network or a wireless network.

[0067] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any other network, including but not limited to any combination of local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), mobile, wired or wireless networks, private networks, or virtual private networks (VPNs). In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), VPNs, and Internet Protocol (IP) security can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0068] Figure 3 This is a flowchart illustrating a graph database update method according to an exemplary embodiment. The method is executed by a computer device, which may be, for example... Figure 1 The diagram shows a server or terminal in the database update system. For example... Figure 3 As shown, the graph database update method may include the following steps:

[0069] Step 301: Obtain the target data.

[0070] In this embodiment, the target data includes a target primary key. The target primary key is one or more fields in the target data, and serves as a unique value to identify the target data.

[0071] Step 302: Query the memory database based on the target primary key. If the query result indicates that the target primary key exists in the memory database, obtain the target point data ID corresponding to the target data.

[0072] The target point data ID is generated when the target data is imported into the graph database; the in-memory database contains the correspondence between the point data IDs generated when each data is imported into the graph database and the primary keys of each data.

[0073] After the computer device acquires the target primary key in the target data, the target primary key is used to perform a query operation in the in-memory database. Since the target database contains the correspondence between each point data ID and the primary key, the target primary key can be used to determine whether the point data ID corresponding to the target primary key exists in the target database. When the target primary key exists in the target database, the target point data ID corresponding to the target data can be acquired according to the correspondence.

[0074] In step 303, it is determined whether the target point data ID exists in the graph database, and the graph database is updated based on the determination result and the target data.

[0075] When the computer device queries the target point data ID corresponding to the target primary key from the in-memory database, the target point data ID can be used to query the graph database. There are two results of the query. When the point data corresponding to the target point data ID is queried from the graph database using the target point data ID, the point data corresponding to the target point data ID is updated according to the target data. When the point data corresponding to the target point data ID is not queried from the graph database using the target point data ID, it means that the graph database deletes the point data for some reason. In this case, the point data can be reconstructed in the graph database according to the target data.

[0076] Figure 4 A graph database operation flowchart is shown. As shown in Figure 4 Before at least one of updating attributes, querying attributes, deleting attributes, and deleting points of a point object (i.e., point data) is performed, the point object needs to be queried. Before at least one of adding edges, updating edges, querying edges, querying attributes, deleting edges, and deleting attributes of an edge object (i.e., edge data) is performed, the source vertex object and the target vertex object corresponding to the edge object also need to be queried. Therefore, in the graph database operation, the query of the point object is a very frequently used operation. Generally, only by using a unique attribute value and performing a traversal operation in the graph database, the point object can be queried, and a large amount of time and computing resources are consumed.

[0077] In the embodiment of the present application, the point data ID is automatically generated when the data is imported into the graph database, and the point data ID is imported into the in-memory database, so that the correspondence between the primary key and the point data ID is generated in the in-memory database. When the update operation of the graph database is performed according to the target data, the in-memory database is queried according to the target primary key in the target data, so that the target point data ID corresponding to the target primary key is determined, as shown in Figure 2In the shown graph database storage structure, the query process can be implemented by comparing the target point data ID with the point data ID saved in the graph database, without traversing each attribute value saved by the point data, which greatly improves the query efficiency and thus the speed of the graph database updating process.

[0078] To sum up, when updating the graph database, the target data obtained contains a target primary key. When querying in the in-memory database according to the target primary key, the in-memory database contains the correspondence between the target point data ID generated when each data is imported into the graph database and the primary key of each data. Therefore, when the target data exists in the graph database, the target point data ID can be queried, and when the target data does not exist in the graph database, the target point data ID cannot be queried. At this time, according to the judgment result of whether the target point data ID exists and the target data, the operation of updating the graph database can be determined, thereby realizing the updating process of the graph database. In the above scheme, the target primary key is set in the target data, and the correspondence between the in-memory database indicating primary key and point data ID is set. Since the point data ID is an index generated by the graph database, the point data can be quickly queried through the point data ID, thereby improving the updating speed of the graph database.

[0079] Figure 5 FIG. 1 is a method flowchart of a graph database updating method according to an example embodiment. The method is executed by a computer device, which can be a server or a terminal in the graph database updating system as shown in FIG. 1. Figure 1 As shown in FIG. 1, the graph database updating method can include the following steps: Figure 5

[0080] Step 501, obtaining target data.

[0081] In the embodiments of the present application, the target data is of a point data type, and the target primary key in the target data is a unique identifier for indicating the target data.

[0082] Step 502, querying in an in-memory database according to the target primary key, and when the in-memory database contains the target primary key, obtaining a target point data ID corresponding to the target data.

[0083] In a possible implementation manner, the in-memory database contains Hash type correspondence data; the correspondence data contains a type label value, a target primary key, and a point data ID.

[0084] ​For example, when the in-memory database is a redis database, in order to reduce the storage space occupation, the redis database can use a Hash type to store data. At this time, it can be considered that the redis database contains various Hash tables (i.e., corresponding relationship data) for storing the corresponding relationship between the target primary key and the point data ID. For example, in any one corresponding relationship data, a type label value, a target primary key, and a point data ID are included, wherein the type label value is used to represent that the corresponding relationship data includes various target primary keys satisfying the type label value and the point data ID corresponding to the target primary key.

[0085] In a possible implementation, the specified bits of the encrypted value of the target primary key in the corresponding relationship data are the same; and the type label value includes the specified bits of the encrypted value of the target primary key in the corresponding relationship data.

[0086] Please refer to Figure 6 , which shows a storage structure diagram of an in-memory database related to an embodiment of the present application. As shown in Figure 6 , in order to reduce the size of a single Hash value, MD5 operation is performed according to the value of the primary key, and the last four characters are appended to the key of the Hash, so as to reduce the size of the value of a single hash type. For example, in a possible implementation, the primary keys with the same last four bits of the encrypted value obtained by MD5 calculation of the primary keys can be merged into a hash type data structure. The smaller the number of bits, the larger the amount of data in a hash type data structure. In the case of 4 bits, there can be 36 4th powers at most, which can support a graph database of hundreds of millions of levels. The hash here is a hash storage type of redis, and redis is a key value database, the key is a label and the last four bits of MD5, and the value is similar to Map or Dict, which is also a key-value structure. The label is a label name defined by a user for data in a graph database. The MD5 here is the MD5 of the primary value, and after MD5, the last four bits of the primary value will certainly be the same. Therefore, the primary values with the same last four bits of the MD5 are merged together and placed in the same label+MD5 hash structure data, so as to reduce the consumption of memory.

[0087] Step 503A, when the judgment result indicates that the target point data ID exists in the graph database, the attribute information of the point data containing the target point data ID in the graph database is updated according to the target data.

[0088] When the target data to be imported is point data, and according to the target point data ID queried from the in-memory database, the corresponding point data is queried from the graph database, then the attribute information of the point data containing the target point data ID in the graph database can be updated according to the target data.

[0089] It should be noted that in the process of updating the attribute information of the point data containing the target point data ID in the graph database, since the primary key corresponds to the target point data ID, the primary key information in the point data cannot be changed under the premise that the target point data ID is unchanged, and since the target data and the primary key in the point data containing the target point data ID in the graph database should be the same, the primary key information in the point data should also not be changed.

[0090] Step 503B, when the judgment result indicates that the target point data ID does not exist in the graph database, create point data in the graph database according to each attribute information contained in the target data.

[0091] When the target data to be imported is point data, and according to the target point data ID queried from the in-memory database, the corresponding point data is not queried from the graph database, which means that the point data identified by the target primary key of the target data does not exist in the graph database. At this time, the graph database can directly create new point data according to each attribute information in the target data.

[0092] In one possible implementation, a first query operation is performed in the in-memory database according to the target primary key in the target data; when the result of the first query operation indicates that the target primary key does not exist in the in-memory database, a second query operation is performed in the graph database according to the target primary key; when the result of the second query operation indicates that the target data containing the target primary key exists in the graph database, the target point data ID corresponding to the target data is imported into the in-memory database to construct the correspondence between the target primary key and the target point data ID.

[0093] Before the graph database is updated according to the target data, a first query operation is performed in the in-memory database according to the target primary key in the target data, and the result of the first query indicates that the target primary key does not exist in the in-memory database. This means that the computer device cannot determine the point data ID corresponding to the target primary key through the in-memory database. At this time, there can be two cases: one is that the point data containing the target primary key does not exist in the graph database, and the other is that the point data containing the target primary key exists in the graph database, but the in-memory database does not save the correspondence between the target primary key and the point data ID.

[0094] To further check whether the point data containing the target primary key exists in the graph database, at this time, the target primary key can be used to perform a traversal query in the graph database (i.e., a second query operation). When the result of the second query operation indicates that the target data containing the target key exists in the graph database, at this time, the point data (i.e., the target data) containing the target primary key exists in the graph database, but the corresponding relationship between the target primary key and the target point data ID is not saved in the memory database. Therefore, the computer device can import the target primary key and the target point data ID into the memory database to construct the corresponding relationship between the target primary key and the target point data ID. Thus, the computer device can subsequently query the target point data ID through the target primary key in the memory database, and then query in the graph database through the target point data ID, thereby improving the query efficiency in the graph database.

[0095] In a possible implementation, a third query operation is performed in the memory database according to the target primary key in the target data; the target data is point data; when the result of the third query operation indicates that the target primary key does not exist in the memory database, a fourth query operation is performed in the graph database according to the target primary key; and when the result of the fourth query operation indicates that the target data containing the target primary key does not exist in the graph database, point data is created in the graph database according to each attribute information in the target data.

[0096] Similarly, before the graph database needs to be updated according to the target data, but a third query operation is performed in the memory database according to the target primary key in the target data, and the result of the third query operation indicates that the target primary key does not exist in the memory database. This indicates that the computer device cannot determine the point data ID corresponding to the target primary key through the memory database. At this time, there can be two cases: one is that the point data containing the target primary key does not exist in the graph database, and the other is that the point data containing the target primary key exists in the graph database, but the corresponding relationship between the target primary key and the point data ID is not saved in the memory database.

[0097] When a traversal query (i.e., a fourth query operation) is directly performed in the graph database according to the target primary key, and the result of the fourth query operation indicates that the target data containing the target primary key does not exist in the graph database, at this time, point data corresponding to the target data is created in the graph database according to each attribute information in the target data.

[0098] Optionally, after the point data is created in the graph database, a point data ID used to indicate the point data corresponding to the target data is generated in the graph database. At this time, the computer device imports the point data ID used to indicate the point data corresponding to the target data and the target primary key of the target data into the memory database to construct the corresponding relationship therebetween.

[0099] Please refer to Figure 7Fig. 1 shows a point data storage process diagram according to an embodiment of the present application. As shown in Fig. 1, after the computer device reads the point data, the computer device searches in the memory database redis according to the label and the primary key. Figure 7

[0100] When the point exists in the redis, the point ID in the redis is obtained, and the point ID is checked in the graph database to determine whether the point exists in the graph database (the operation is searched by the ID, and the speed is very fast, and will not become a bottleneck).

[0101] When the point does not exist in the redis, it can be selected whether to check in the graph database. According to the test, the safety and stability of the redis can be ensured, and the possibility of data loss is small. When the data is stored for the first time or the proportion of the known new data is too large, the checking stage can be skipped, and a function of periodically repairing the cache database in batches can also be selected.

[0102] When the point exists in the graph database, it is indicated that the point already exists, and the point is updated.

[0103] When the point does not exist in the graph database, it is indicated that the point does not exist, and the point is added.

[0104] In summary, when the graph database is updated, the target data obtained contains a target primary key. When the target data is searched in the memory database according to the target primary key, since the memory database contains the correspondence between the target point data ID generated when each data is imported into the graph database and the primary key of each data, when the target data exists in the graph database, the target point data ID can be queried, and when the target data does not exist in the graph database, the target point data ID cannot be queried. At this time, according to the judgment result of whether the target point data ID exists and the target data, the operation of updating the graph database can be determined, so that the updating process of the graph database is realized. The above scheme sets the target primary key in the target data, and sets the correspondence between the memory database indicating the primary key and the point data ID. Since the point data ID is an index generated by the graph database, the point data can be quickly queried through the point data ID, so that the updating speed of the graph database is improved.

[0105] Figure 8 Fig. 2 shows a method flowchart of a graph database updating method according to an example embodiment. The method is performed by a computer device, which can be a server or a terminal in a graph database updating system as shown in Fig. 2. Figure 1 Figure 8 As shown in Fig. 2, the graph database updating method can include the following steps:

[0106] Step 801, obtaining target data. ​​

[0107] In the embodiment of the present application, the target data is an edge data type, and the target primary key in the target data is used to indicate at least one of the source vertex data and the target vertex data corresponding to the target data.

[0108] Optionally, when the target data is an edge data type, there are two target primary keys in the target data, which are respectively used to indicate the source vertex data corresponding to the target data and the target vertex data corresponding to the target data. Since the computer execution processes corresponding to the two target primary keys are the same, the subsequent steps are explained by taking any one of the target primary keys as an example.

[0109] In step 802, a query operation is performed in the in-memory database according to the target primary key, and the target point data ID corresponding to the target data is obtained.

[0110] Step 802 is similar to step 502, and will not be described here.

[0111] In step 803A, when the judgment result indicates that the target point data ID exists in the graph database, the target data is written into the graph database as the edge data corresponding to the point data containing the target point data ID in the graph database.

[0112] When the judgment result indicates that the target point data ID exists in the graph database, the point data containing the target point data ID in the graph database can be one of the source vertex data or the target vertex data. Taking the point data containing the target point data ID as the target vertex data as an example, when the target vertex data is queried according to the target point data ID, since the target data is the edge data related to the target vertex data, the target data can be written into the graph database as the edge data related to the target vertex data, and saved to the storage area corresponding to the target vertex data.

[0113] In step 803B, when the judgment result indicates that the target point data ID does not exist in the graph database, error information indicating that the graph database update is abnormal is generated.

[0114] When the judgment result indicates that the target point data ID does not exist in the graph database, i.e., the target data as edge data cannot find the source vertex data and the target vertex data corresponding thereto in the graph database, it is obvious that the target data cannot be imported into the graph database at this time. At this time, the computer device generates error information indicating that the graph database update is abnormal.

[0115] In a possible implementation, the fifth query operation is performed in the in-memory database according to a target primary key in target data; the target data is edge data; when the result of the fifth query operation indicates that the target primary key does not exist in the in-memory database, a sixth query operation is performed in the graph database according to the target primary key; when the result of the sixth query operation indicates that the point data containing the target primary key does not exist in the graph database, error information indicating that the graph database update is abnormal is generated.

[0116] When the target data is edge data and the graph database needs to be imported to update the graph database, when the result of the fifth query operation indicates that the target primary key does not exist in the in-memory database, it indicates that there is no corresponding relationship between the target primary key and the point data ID in the in-memory database, at this time, in order to further determine whether the point data corresponding to the target primary key exists in the graph database, the target primary key needs to be used to perform a traversal query (i.e., the sixth query operation) in the graph database, and when the query result indicates that the graph database also does not contain the target primary key, that is, the target data as edge data cannot find the source vertex data and the target vertex data corresponding to the target data in the graph database, obviously, the target data cannot be imported into the graph database at this time, and therefore the computer device also generates error information indicating that the graph database update is abnormal.

[0117] Please refer to Figure 9 , which shows a point data storage process according to an embodiment of the present application. As shown in Figure 9 , after reading the point data, the point data is searched in the in-memory database redis according to a label and a primary key;

[0118] When the point exists in the redis, the point ID in the redis is obtained, and the point ID is used to check whether the point exists in the graph database (this operation is searched by the ID, and the speed is very fast and will not become a bottleneck);

[0119] When the point does not exist in the redis, it can be selected whether to check in the graph database. According to tests, the safety and stability of the redis can be guaranteed, and the possibility of data loss is small. When the data is stored for the first time or the proportion of the newly added data is too large, the checking stage can be skipped, and a function of periodically repairing the cache database in batches can also be selected;

[0120] When the point exists in the graph, the point object is obtained according to the ID, and the edge operation is performed according to the point object.

[0121] The present application improves the retrieval speed of the point by adding the in-memory database redis in the architecture of the graph database, so as to optimize the operation performance of the graph data. The beneficial effects of the improvement are as follows:

[0122] 1. No restrictions are imposed on the backend storage and backend index engine of a certain JanusGraph, and the backend storage and engine are transparent to the storage scheme, and the storage scheme uses the high-level API (Application Programming Interface) of JanusGraph to perform data operations;

[0123] 1. The storage performance is greatly improved, and the performance comparison is as follows:

[0124]

[0125]

[0126] It can be found that the speed of importing edge data using the batch storage scheme of the present application is significantly improved, and the speed does not decrease significantly when the data volume increases. When using a graph database, the amount of edge data is often much larger than that of point data. Optimizing the storage speed of edge data can improve the overall storage speed.

[0127] In summary, when updating the graph database, the target data obtained contains a target primary key. When querying in the in-memory database according to the target primary key, the in-memory database contains the correspondence between the target point data ID generated when each data is imported into the graph database and the primary key of each data. Therefore, when the target data exists in the graph database, the target point data ID can be queried, and when the target data does not exist in the graph database, the target point data ID cannot be queried. At this time, according to the judgment result of whether the target point data ID exists and the target data, the operation of updating the graph database can be determined, thereby realizing the updating process of the graph database. The above scheme sets the target primary key in the target data and sets the correspondence between the in-memory database indicating the primary key and the point data ID. Since the point data ID is an index generated by the graph database, the point data can be quickly queried through the point data ID, thereby improving the updating speed of the graph database.

[0128] Figure 10 is a structural block diagram of a graph database updating device according to an exemplary embodiment. The device comprises:

[0129] The target data acquisition unit 1001 is configured to acquire target data; the target data contains a target primary key;

[0130] The first query module 1002 is configured to perform a query operation in the in-memory database according to the target primary key, and when the in-memory database contains the target primary key, the target point data ID corresponding to the target data is obtained; the target point data ID is generated when the target data is imported into the graph database;

[0131] The first updating module 1003 is configured to determine whether the target point data ID exists in the graph database, and perform data updating on the graph database based on the determination result and the target data.

[0132] In a possible implementation, the apparatus further includes:

[0133] The second querying module is configured to perform a first querying operation on the in-memory database according to the target primary key.

[0134] The third querying module is configured to perform a second querying operation on the graph database according to the target primary key when the result of the first querying operation indicates that the target primary key does not exist in the in-memory database.

[0135] The correspondence constructing module is configured to import the target point data ID stored in the graph database into the in-memory database to construct the correspondence between the target primary key and the target point data ID when the result of the second querying operation indicates that the data containing the target primary key exists in the graph database.

[0136] In a possible implementation, the in-memory database contains Hash type correspondence data; the correspondence data contains a type label value, a target primary key, and a point data ID.

[0137] In a possible implementation, the target primary key in the correspondence data corresponds to a specified bit of an encrypted value; and the type label value includes the specified bit of the encrypted value of the target primary key in the correspondence data.

[0138] In a possible implementation, the first updating module includes:

[0139] The attribute information updating unit is configured to update attribute information of point data containing the target point data ID in the graph database according to the target data when the determination result indicates that the target point data ID exists in the graph database.

[0140] The point data creating unit is configured to create point data in the graph database according to each attribute information contained in the target data when the determination result indicates that the target point data ID does not exist in the graph database.

[0141] In a possible implementation, the apparatus further includes:

[0142] The fourth querying module is configured to perform a third querying operation on the in-memory database according to a target primary key in target data; the target data is point data.

[0143] The fifth query module is configured to perform a fourth query operation in the graph database according to the target primary key when the result of the third query operation indicates that the target primary key does not exist in the memory database.

[0144] The point data creation module is configured to create point data in the graph database according to each attribute information in the target data when the result of the fourth query operation indicates that the point data containing the target primary key does not exist in the graph database.

[0145] In a possible implementation, when the target data is edge data, the target point data ID is used to indicate at least one of source vertex data and target vertex data corresponding to the target data; and the first update module further includes:

[0146] The edge data writing unit is configured to write the target data as edge data corresponding to point data containing the target point data ID in the graph database when the result of the judgment indicates that the target point data ID exists in the graph database.

[0147] The error information generation unit is configured to generate error information indicating an abnormal graph database update when the result of the judgment indicates that the target point data ID does not exist in the graph database.

[0148] In a possible implementation, the apparatus further includes:

[0149] The sixth query module is configured to perform a fifth query operation in the memory database according to a target primary key in target data; and the target data is edge data.

[0150] The seventh query module is configured to perform a sixth query operation in the graph database according to the target primary key when the result of the fifth query operation indicates that the target primary key does not exist in the memory database.

[0151] The error information generation module is configured to generate error information indicating an abnormal graph database update when the result of the sixth query operation indicates that point data containing the target primary key does not exist in the graph database.

[0152] To sum up, when the graph database is updated, the target data obtained contains a target primary key, and when the target primary key is queried in the in-memory database, the in-memory database contains the correspondence between the target point data ID generated when each data is imported into the graph database and the primary key of each data. Therefore, when the target data exists in the graph database, the target point data ID can be queried, and when the target data does not exist in the graph database, the target point data ID cannot be queried. At this time, according to the judgment result of whether the target point data ID exists and the target data, it can be determined that the operation of updating the graph database is performed, so as to realize the updating process of the graph database. The above scheme sets the target primary key in the target data and sets the correspondence between the in-memory database indicating the primary key and the point data ID. Since the point data ID is an index generated by the graph database, the point data can be quickly queried through the point data ID, so as to improve the updating speed of the graph database.

[0153] Referring to Figure 11 According to an exemplary embodiment of the present application, a computer device is provided, and a schematic diagram of the computer device is shown in FIG. 1. The computer device includes a memory and a processor. The memory is configured to store a computer program. When the computer program is executed by the processor, the graph database updating method described above is implemented.

[0154] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, or combinations thereof.

[0155] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiments of the present application. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the method in the method embodiments described above.

[0156] The memory can include a program storage area and a data storage area, where the program storage area can store an operating system, application programs required by at least one function, and the like, and the data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some embodiments, the memory can optionally include a memory that is remotely located with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] In an exemplary embodiment, a computer readable storage medium storing at least one computer program is also provided, the at least one computer program is loaded and executed by the processor to implement all or part of the steps of the above method. For example, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0158] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the application be limited only by the scope of the claims, which will be readily appreciated by those skilled in the art. The specification and examples are to be considered exemplary only, with the true scope and spirit of the application indicated by the following claims.

[0159] It should be understood that the application is not limited to the precise construction that has been described above and illustrated in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A graph database update method, characterized in that, The method includes: Obtain target data; the target data includes a target primary key; A query is performed in the in-memory database based on the target primary key. If the query result indicates that the target primary key exists in the in-memory database, the target point data ID corresponding to the target data is obtained. The target point data ID is generated when the target data is imported into the graph database. The in-memory database contains the correspondence between the point data IDs generated when each piece of data is imported into the graph database and the primary keys of each piece of data. Determine whether the target point data ID exists in the graph database, and update the graph database based on the determination result and the target data; If a first query operation is performed in the memory database based on the target primary key, and the result of the first query operation indicates that the target primary key does not exist in the memory database, a second query operation is performed in the graph database based on the target primary key. When the result of the second query operation indicates that there is data containing the target primary key in the graph database, the target point data ID stored in the graph database is imported into the memory database to establish the correspondence between the target primary key and the target point data ID.

2. The method according to claim 1, characterized in that, The in-memory database contains Hash-type mapping data; the mapping data includes type label values, target primary keys, and point data IDs.

3. The method according to claim 2, characterized in that, The specified bits of the encrypted value corresponding to the target primary key in the correspondence data are the same; the type tag value includes the specified bits of the encrypted value of the target primary key in the correspondence data.

4. The method according to any one of claims 1 to 3, characterized in that, When the target data is point data, the step of updating the graph database based on the judgment result and the target data includes: When the judgment result indicates that a target point data ID exists in the graph database, the attribute information of the point data containing the target point data ID in the graph database is updated according to the target data. When the judgment result indicates that there is no target point data ID in the graph database, point data is created in the graph database according to the various attribute information contained in the target data.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: A third query operation is performed in the in-memory database based on the target primary key in the target data; the target data is point data. When the result of the third query operation indicates that the target primary key does not exist in the memory database, a fourth query operation is performed in the graph database based on the target primary key. When the result of the fourth query operation indicates that there is no point data containing the target primary key in the graph database, the point data is created in the graph database according to the attribute information in the target data.

6. The method according to any one of claims 1 to 3, characterized in that, When the target data is edge data, the target vertex data ID is used to indicate at least one of the source vertex data and the target vertex data corresponding to the target data; The step of updating the graph database based on the judgment result and the target data includes: When the judgment result indicates that a target point data ID exists in the graph database, the target data is written into the graph database as the edge data corresponding to the point data containing the target point data ID. When the judgment result indicates that the target point data ID does not exist in the graph database, an error message indicating that the graph database update is abnormal is generated.

7. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Based on the target primary key in the target data, a fifth query operation is performed in the in-memory database; the target data is edge data. When the result of the fifth query operation indicates that the target primary key does not exist in the memory database, a sixth query operation is performed in the graph database based on the target primary key. When the result of the sixth query operation indicates that there is no point data containing the target primary key in the graph database, an error message indicating an abnormal update of the graph database is generated.

8. A graph database updating device, characterized in that, The device includes: A target data acquisition unit is used to acquire target data; the target data includes a target primary key. The first query module is used to query the memory database based on the target primary key, and when the query result indicates that the target primary key exists in the memory database, obtain the target point data ID corresponding to the target data; the target point data ID is generated when the target data is imported into the graph database; the memory database contains the correspondence between the point data IDs generated when each piece of data is imported into the graph database and the primary key of each piece of data; The first update module is used to determine whether the target point data ID exists in the graph database, and to update the graph database based on the determination result and the target data. The second query module performs a first query operation in the memory database based on the target primary key; The third query module is used to perform a second query operation in the graph database based on the target primary key when the result of the first query operation indicates that the target primary key does not exist in the memory database. The correspondence construction module is used to import the target point data ID stored in the graph database into the memory database when the result of the second query operation indicates that there is data containing the target primary key in the graph database, so as to construct the correspondence between the target primary key and the target point data ID.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the graph database update method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement the graph database update method as described in any one of claims 1 to 7.

Images