Graph data query method, system, computer device and readable storage medium

Abstract

The application relates to a graph data query method, a system, computer equipment and a readable storage medium, wherein the graph data query method comprises the following steps: a metadata synchronization step, obtaining data of a target graph database and synchronizing metadata of point data and / or edge data in the graph data to a shunt database in real time; a point and edge data synchronization step, synchronizing point data and / or edge data in the graph database to the shunt database and updating in real time; a data query acceleration step, obtaining a data query request initiated by an application and screening a point and edge direct query request in the data query request, directly obtaining a query result corresponding to the point and edge direct query request in the shunt database and outputting. The application solves the problem that a graph database cannot quickly query point and edge data, and improves the point data and edge data query speed in the graph database.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a graph data query method, system, computer device, and computer-readable storage medium. Background Art

[0002] A graph database (GDB for short) is a database that uses a graph structure for semantic queries. It uses points, edges, and attributes to represent and store data. Real-world entities are mapped to points in the graph database, and the relationships between entities are mapped to edges. Points and edges each have attributes. Thanks to its storage structure, graph databases have a clear advantage in query speed for associated data compared to other types of databases. However, graph databases do not have an advantage over other types of databases in direct queries of point and edge data. Under complex query conditions, the query performance of graph databases is generally worse than that of other types of databases. Currently, data queries are generally accelerated by providing data indexes.

[0003] An index is a decentralized storage structure created by transforming data through various computational methods. Indexes are stored independently of the data and contain logical pointers to associated data entries. Index design considers common query scenarios, ensuring that queries are as likely to directly hit index entries as possible. This allows for a logical pointer to the data, quickly retrieving the data storage location.

[0004] Graph database indexes can be stored independently from vertex-edge data, but they still need to be stored within the graph database and adhere to its storage architecture. Because graph database storage design focuses on querying data relationships, even after using indexes to improve query speed, complex queries directly on vertex-edge data are still slower than those in other databases. Currently, index-based graph database queries are not a viable solution for fast vertex-edge data queries. Summary of the Invention

[0005] Embodiments of the present application provide a graph data query method, system, computer device, and computer-readable storage medium to improve the query speed of midpoint data and edge data in a graph database by providing a technical solution for diverting graph data.

[0006] In a first aspect, an embodiment of the present application provides a graph data query method, including:

[0007] A metadata synchronization step, obtaining data from a target graph database and synchronizing metadata of point data and / or edge data in the graph data to a shunt database in real time;

[0008] The step of synchronizing point and edge data is to synchronize the point data and / or edge data in the graph database to the diversion database and update them in real time; specifically, when the point data and / or edge data are added, modified or deleted in the graph data, the diversion database will correspondingly add, modify or delete the point data and / or edge data in real time.

[0009] The data query acceleration step obtains the data query request initiated by the application and filters the point-edge direct query request in the data query request, directly obtains the query result corresponding to the point-edge direct query request in the diversion database and outputs it.

[0010] Based on the above steps, this application combines the use of the offload database and the graph database to directly complete the query of point data and edge data in the graph database using the offload database, thereby effectively improving the query efficiency of the graph database and reducing the query time.

[0011] In some embodiments, the metadata synchronization step includes:

[0012] A data table creation step is to create a data table in the diversion database for each type of point data and / or edge data based on the metadata, and create data columns of corresponding data types in the data table based on the attributes of the point data and / or edge data; wherein the attributes include character type, numerical type, date type and other attributes of the point and edge.

[0013] In the data table updating step, when the metadata is added, modified or deleted in the graph database, the diversion database correspondingly adds, modifies or deletes the corresponding data table or data column in real time.

[0014] In some embodiments, the data query acceleration step specifically includes:

[0015] A query request judgment step is to obtain the data query request initiated by the application and determine whether it is a direct vertex-edge query request;

[0016] In the query result acquisition step, if the result of the query request judgment step is yes, the query result in the diversion database is directly queried and returned to the application; otherwise, the query result in the graph database is queried and returned to the application.

[0017] In some embodiments, the offload database is ClickHouse, and queries are performed from ClickHouse when querying points and edges, thereby utilizing the high query performance of ClickHouse to accelerate the query.

[0018] In a second aspect, an embodiment of the present application provides a graph data query system, including:

[0019] A metadata synchronization module, configured to obtain data from a target graph database and synchronize metadata of vertex data and / or edge data in the graph data to a shunt database in real time;

[0020] The point-edge data synchronization module is used to synchronize the point data and / or edge data in the graph database to the diversion database and update it in real time; specifically, when the point data and / or edge data are added, modified or deleted in the graph data, the diversion database will correspondingly add, modify or delete the point data and / or edge data in real time.

[0021] The data query acceleration module is used to obtain data query requests initiated by applications and filter point-edge direct query requests in the data query requests, directly obtain query results corresponding to the point-edge direct query requests in the offload database and output them.

[0022] Based on the above modules, this application combines the use of the diversion database and the graph database to directly complete the query of point data and edge data in the graph database using the diversion database, thereby effectively improving the query efficiency of the graph database and reducing the query time.

[0023] In some embodiments, the metadata synchronization module specifically includes:

[0024] A data table creation module is used to create a data table in the diversion database for each type of point data and / or edge data based on the metadata, and to create data columns of corresponding data types in the data table based on the attributes of the point data and / or edge data; wherein the attributes include character type, numerical type, date type and other attributes of the point and edge.

[0025] The data table update module is used to add, modify or delete the corresponding data table or data column in the offload database in real time when the metadata is added, modified or deleted in the graph database.

[0026] In some embodiments, the data query acceleration module specifically includes:

[0027] The query request judgment module is used to obtain the data query request initiated by the application and determine whether it is a direct query request for the vertex and edge;

[0028] The query result acquisition module is used to directly query the query result in the diversion database and return it to the application when the result of the query request judgment step is yes; otherwise, query the query result in the graph database and return it to the application.

[0029] In some embodiments, the offload database is ClickHouse, and queries are performed from ClickHouse when querying points and edges, thereby accelerating queries by utilizing the high query performance of ClickHouse.

[0030] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the graph data query method as described in the first aspect above is implemented.

[0031] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the graph data query method described in the first aspect above is implemented.

[0032] Compared with related technologies, the graph data query method, system, computer device and computer-readable storage medium provided by the embodiments of the present application specifically relate to the field of data capabilities. By diverting the direct query of point and edge data of the graph database, the query efficiency of the graph database is improved, the query time is reduced, and the performance of the graph database is improved.

[0033] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0035] Figure 1 is a flowchart of a graph data query method according to an embodiment of the present application;

[0036] Figure 2 is a structural block diagram of a graph data query system according to an embodiment of the present application;

[0037] Figure 3 This is a schematic diagram of the metadata structure in the graph data query method according to the preferred embodiment of the present application;

[0038] Figure 4 This is the ClickHouse data representation scheme in the graph data query method according to the preferred embodiment of the present application;

[0039] Figure 5 This is a schematic diagram of metadata synchronization timing according to a preferred embodiment of the present application;

[0040] Figure 6This is a schematic diagram of the timing principle of point-edge data synchronization according to a preferred embodiment of the present application;

[0041] Figure 7 It is a step-by-step flowchart of the graph data query method according to the preferred embodiment of the present application.

[0042] in:

[0043] 1. Metadata synchronization module; 2. Point-edge data synchronization module; 3. Data query acceleration module; 101. Data table creation module; 102. Data table update module; 301. Query request judgment module; 302. Query result acquisition module. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0045] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can apply the present application to other similar scenarios based on these drawings without inventive effort. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0046] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive with other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0047] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0048] ClickHouse is a column-based database designed for online analytical processing. It utilizes a distributed computing architecture and features a native vectorized computing engine. Thanks to its well-designed architecture and storage model, ClickHouse achieves industry-leading query performance.

[0049] To improve the speed of complex queries of point data and edge data in a graph database, the present invention proposes a graph data query acceleration solution based on a distributed database. Specifically, the present invention provides a graph data query method, system, computer device, and computer-readable storage medium.

[0050] First, the graph data query method provided by this embodiment is explained. Figure 1 is a flowchart of a graph data query method according to an embodiment of the present application. Figure 1 As shown, the process includes the following steps:

[0051] Metadata synchronization step S1, obtaining data from the target graph database and synchronizing metadata of point data and / or edge data in the graph data to a diversion database in real time;

[0052] The metadata synchronization step S1 includes:

[0053] In the data table creation step S101, a data table is created in the diversion database for each type of point data and / or edge data according to the metadata, and a data column of the corresponding data type is created in the data table according to the attributes of the point data and / or edge data; wherein the attributes include the character type, numerical type, date type and other attributes of the point and edge.

[0054] In the data table updating step S102 , when metadata is added, modified, or deleted in the graph database, the offload database will correspondingly add, modify, or delete the corresponding data table or data column in real time.

[0055] In step S2 of synchronizing point and edge data, the point data and / or edge data in the graph database are synchronized to the diversion database and updated in real time. Specifically, when point data and / or edge data are added, modified or deleted in the graph data, the diversion database will correspondingly add, modify or delete the point data and / or edge data in real time.

[0056] In the data query acceleration step S3, the data query request initiated by the application is obtained and the point-edge direct query request in the data query request is filtered, and the query result corresponding to the point-edge direct query request is directly obtained in the offload database and outputted.

[0057] The data query acceleration step S3 specifically includes:

[0058] Query request determination step S301: obtaining a data query request initiated by an application and determining whether it is a point-edge direct query request;

[0059] In the query result acquisition step S302, if the result of the query request judgment step is yes, the query result in the offload database is directly queried and returned to the application; otherwise, the query result in the graph database is queried and returned to the application.

[0060] Based on the above steps, this application combines the use of the offload database and the graph database to directly complete the query of point data and edge data in the graph database using the offload database, thereby effectively improving the query efficiency of the graph database and reducing the query time.

[0061] The following describes and illustrates the embodiments of the present application through preferred embodiments. In this preferred embodiment, ClickHouse is used as the shunting database, and the high query performance of ClickHouse is used to further improve the query efficiency of the graph database and reduce the query time. It is worth noting that in addition to ClickHouse, other databases (such as POSTGreSQL) or software can also be used to shunt the direct query of the point edge of the graph database using a similar process.

[0062] Specifically, in step S1, refer to Figure 3The metadata structure in the graph database shown in the figure creates a data table in ClickHouse for each type of vertex and / or edge, and creates columns in the data table based on the attributes associated with the vertex and / or edge. The data type of the column is consistent with the data type of the attribute in the graph database, and the result is as follows: Figure 4 ClickHouse data table shown. When the metadata in the graph database is updated, the corresponding update operation is performed on the data table in ClickHouse, such as Figure 5 The implementation code example of the above process is as follows:

[0063] / / Call the graph database interface to create a new point type

[0064] graphDataBase.createVertexSchema(vertexSchema);

[0065] / / Create the corresponding table in ClickHouse

[0066] clickhouse.execute('CREATE TABLE vertex_table(......)');

[0067] / / Call the graph database interface to delete the point type

[0068] graphDataBase.deleteVertexSchema(vertexSchema);

[0069] / / Delete the corresponding table in ClickHouse

[0070] clickhouse.execute('DROP TABLE vertex_table');

[0071] / / Call the graph database interface to modify the point metadata definition

[0072] graphDataBase.modifyVertexSchema(vertexSchema);

[0073] / / Delete the corresponding table in ClickHouse

[0074] clickhouse.execute('ALTER TABLE vertex_table(......)');

[0075] In step S2, the data in ClickHouse is also kept consistent with the data in the graph database. When data is added, modified, or deleted in the graph database, the same operation is performed on ClickHouse, such as Figure 6 The implementation code example of the above process is as follows:

[0076] / / Call the graph database interface to add new data

[0077] graphDataBase.addVertex(vertex);

[0078] / / Add new data in ClickHouse

[0079] clickhouse.execute('INSERTINTOvertex_table(......');

[0080] / / Call the graph database interface to delete data

[0081] graphDataBase.deleteVertex(vertex);

[0082] / / Delete data in ClickHouse

[0083] clickhouse.execute('ALTERTABLEvertex_tableDELETE...');

[0084] / / Call the graph database interface to modify the data

[0085] graphDataBase.modifyVertexSchema(vertexSchema);

[0086] / / Modify data in ClickHouse

[0087] clickhouse.execute('ALTERTABLEvertex_tableUPDATE......');

[0088] refer to Figure 7 As shown, in step S3, the data query request initiated by the application is obtained and it is determined whether it is a direct query request for a vertex or edge; if the result of the query request determination step is yes, the query result in ClickHouse is directly queried and returned to the application; otherwise, the query result in the graph database is queried and returned to the application.

[0089] This embodiment integrates ClickHouse and a graph database, utilizing ClickHouse's high query performance to improve the query efficiency of the graph database, reduce query time, and achieve performance improvement.

[0090] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0091] This embodiment also provides a graph data query system, which is used to implement the above-mentioned embodiments and preferred implementation methods. The details that have been explained will not be repeated here. As used below, the terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that implements the predetermined functions. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceivable. Figure 2 is a structural block diagram of a graph data query system according to an embodiment of the present application, such as Figure 2 As shown, the system includes a metadata synchronization module 1, a point-edge data synchronization module 2, and a data query acceleration module 3. Specifically:

[0092] The metadata synchronization module 1 is used to obtain the data of the target graph database and synchronize the metadata of the point data and / or edge data in the graph data to a diversion database in real time; wherein, the metadata synchronization module 1 includes: a data table creation module 101, which is used to create a data table in the diversion database for each type of point data and / or edge data according to the metadata, and create a data column of the corresponding data type in the data table according to the attributes of the point data and / or edge data; wherein, the attributes include the character type, numerical type, date type and other attributes of the point and edge. The data table update module 102 is used to add, modify or delete the corresponding data table or data column in the diversion database in real time when metadata is added, modified or deleted in the graph database. By way of example and not limitation, the diversion database is ClickHouse, and queries are performed from ClickHouse when querying points and edges, so as to achieve accelerated queries using the high query performance of ClickHouse.

[0093] The point-edge data synchronization module 2 is used to synchronize the point data and / or edge data in the graph database to the diversion database and update it in real time; specifically, when point data and / or edge data are added, modified or deleted in the graph data, the diversion database will correspondingly add, modify or delete the point data and / or edge data in real time.

[0094] Data query acceleration module 3 is used to obtain data query requests initiated by applications and filter the data query requests for direct point-edge queries, directly obtaining and outputting the query results corresponding to the direct point-edge queries in the offload database. Specifically, data query acceleration module 3 includes: a query request judgment module 301, which is used to obtain data query requests initiated by applications and determine whether they are direct point-edge queries; a query result acquisition module 302, which is used to directly query the offload database for query results and return them to the application if the result of the query request judgment step is yes; otherwise, it queries the graph database for query results and returns them to the application.

[0095] It should be noted that in addition to ClickHouse, other databases or software such as POSTGreSQL can also be used to implement direct point-edge query diversion of graph databases.

[0096] Based on the above modules, this application combines the use of the diversion database and the graph database to directly complete the query of point data and edge data in the graph database using the diversion database, thereby effectively improving the query efficiency of the graph database and reducing the query time.

[0097] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0098] In addition, combined Figure 1 The graph data query method described in the embodiment of the present application can be implemented by a computer device. The computer device may include a processor and a memory storing computer program instructions.

[0099] Specifically, the processor may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0100] Among them, the memory may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside the data processing device. In a specific embodiment, the memory is a non-volatile memory. In a specific embodiment, the memory includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0101] The memory may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor.

[0102] The processor implements any one of the graph data query methods in the above embodiments by reading and executing computer program instructions stored in the memory.

[0103] In addition, in conjunction with the graph data query method in the above embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the graph data query methods in the above embodiments is implemented.

[0104] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A graph data query method, characterized in that: include: A metadata synchronization step, obtaining data from a target graph database and synchronizing metadata of point data and / or edge data in the graph data to a shunt database in real time; The metadata synchronization step further includes: a data table creation step, creating a data table in the diversion database for each type of point data and / or edge data based on the metadata, and creating data columns of corresponding data types in the data table based on attributes of the point data and / or edge data; wherein the attributes include character type of point data, character type of edge data, numeric type, and date type; A data table updating step, in which when the metadata is added, modified or deleted in the graph database, the diversion database correspondingly adds, modifies or deletes the corresponding data table or data column in real time; A node-edge data synchronization step, synchronizing node data and / or edge data in the graph database to the offload database and updating them in real time; The data query acceleration step obtains the data query request initiated by the application and filters the point-edge direct query request in the data query request, directly obtains the query result corresponding to the point-edge direct query request in the diversion database and outputs it.

2. The graph data query method according to claim 1, characterized in that: The data query acceleration step specifically includes: A query request judgment step is to obtain the data query request initiated by the application and determine whether it is a point-edge direct query request; In the query result acquisition step, if the result of the query request judgment step is yes, the query result in the diversion database is directly queried and returned to the application; otherwise, the query result in the graph database is queried and returned to the application.

3. The graph data query method according to any one of claims 1-2, characterized in that: The diversion database is ClickHouse.

4. A graph data query system, characterized in that: include: A metadata synchronization module, configured to obtain data from a target graph database and synchronize metadata of vertex data and / or edge data in the graph data to a shunt database in real time; The metadata synchronization module further includes: a data table creation module, configured to create a data table in the diversion database for each type of point data and / or edge data based on the metadata, and to create data columns of corresponding data types in the data table based on the attributes of the point data and / or edge data; wherein the attributes include the character type of the point data, the character type of the edge data, the numeric type, and the date type; A data table update module, configured to cause the offload database to add, modify or delete corresponding data tables or data columns in real time when the metadata is added, modified or deleted in the graph database; A node-edge data synchronization module, used to synchronize node data and / or edge data in the graph database to the offload database and update them in real time; The data query acceleration module is used to obtain data query requests initiated by applications and filter point-edge direct query requests in the data query requests, directly obtain query results corresponding to the point-edge direct query requests in the offload database and output them.

5. The graph data query system according to claim 4, characterized in that: The data query acceleration module specifically includes: The query request judgment module is used to obtain the data query request initiated by the application and determine whether it is a direct query request for the vertex and edge; The query result acquisition module is used to directly query the query result in the diversion database and return it to the application when the result of the query request judgment step is yes; otherwise, query the query result in the graph database and return it to the application.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the graph data query method according to any one of claims 1 to 3 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the graph data query method according to any one of claims 1 to 3 is implemented.

Images