A knowledge graph mass data graph query method and a system having the same

Abstract

The application relates to the technical field of knowledge graph query, in particular to a knowledge graph mass data graph query method and a system with the same. The knowledge graph mass data graph query method comprises the following steps: acquiring a starting and ending entity list of the data graph, wherein the entity list comprises a hive table of entities and edges from the starting to the ending; filtering the hive table of the entities and the edges according to preset filtering conditions; determining a set of the entities and the edges according to the filtered hive table of the entities and the edges; putting the attributes of the entities into the attributes of the edges, importing the set of the edges (containing entity information) into a GraphFrame framework, and determining join data volume and times; and querying the data graph according to the join data volume and times. Therefore, the query efficiency can be greatly improved under the condition that the sum of the entities and the edges exceeds 10 billion data volume, the storage space is reduced, the join logic is accelerated, and the join times are reduced.

Description

Technical Field

[0001] This invention relates to the field of knowledge graph query technology, and in particular to a method for querying massive amounts of knowledge graph data and a system thereof. Background Technology

[0002] A knowledge graph, also known as a scientific knowledge graph, or in the library and information science field as a knowledge domain visualization or knowledge domain mapping map, is a series of different graphics that display the development process and structural relationships of knowledge. It uses visualization technology to describe knowledge resources and their carriers, and to mine, analyze, construct, draw and display knowledge and the interrelationships between them.

[0003] When the total amount of entities and relationships constituting a knowledge graph exceeds 10 billion data points, and when performing K-layer expansion of more than 10,000 nodes and batch offline queries of many-to-many entity relationships of degree 1 or higher, the query function originally provided by the graph database takes more than 60 minutes to return results, resulting in low search efficiency and even failure to retrieve results, causing the graph database to crash and failing to meet user needs.

[0004] Therefore, there is an urgent need for a method that can quickly query data graphs within knowledge graphs formed from massive amounts of data. Summary of the Invention

[0005] The present invention provides a method and system for querying massive data graphs in a knowledge graph, which can solve the problem of low efficiency in querying data graphs in massive knowledge graphs.

[0006] The first aspect of this application provides a method for querying massive amounts of knowledge graph data, the method including:

[0007] Obtain the start and end entity lists of the data graph, wherein the entity lists from the start to the end include Hive tables of entities and edges;

[0008] Filter the Hive tables of the entities and the edges according to preset filtering conditions;

[0009] Based on the filtered Hive table of the entities and edges, determine the set of the entities and edges;

[0010] The attributes of the entity are placed into the attributes of the edge, the set of edges is imported into the GraphFrame framework, and the amount and number of joins are determined. The attributes of the edges in the set of edges contain entity information.

[0011] The data graph is queried based on the GraphFrame framework, which defines the amount and frequency of joins.

[0012] In one possible implementation, the step of obtaining the list of entities at the beginning and end of the data graph includes:

[0013] Obtain multiple Hive tables for the entities in the entity list and multiple Hive tables for the edges between the entities, wherein the Hive tables for the entities include an ID field, and the Hive tables for the edges include SRC and DST fields.

[0014] In one possible implementation, the step of filtering the Hive table of the entity and the edge according to preset filtering conditions includes:

[0015] Import the list of the start and end entities of the data graph into the Spark application;

[0016] Load the entities and edges to be used in calculating the start entity list to the end entity list into the Spark application;

[0017] Based on preset filtering conditions, entities and edges that do not meet the filtering conditions are filtered out in the Spark application.

[0018] In one possible implementation, the step of determining the set of entities and edges based on the filtered Hive table of entities and edges includes:

[0019] The fixed attributes of the Hive tables of the entity and the edge are encoded, wherein the fixed attributes include at least the ID, SRC and DST of the Hive table;

[0020] Encode the 32-bit random string number into two long integers for the entity and the edge;

[0021] The encoded entities and edges are respectively used to form a set of entities and a set of edges.

[0022] In one feasible approach, the steps of placing the entity's attributes into the edge's attributes, importing the set of edges into a GraphFrame, and determining the amount and number of joins include:

[0023] The attributes of the entity are placed into the attributes of the edge, and the set of edges is joined.

[0024] The number of edges is limited based on the starting and ending lists of entities.

[0025] In one possible implementation, the steps of importing the set of entities and the set of edges into a GraphFrame, placing the attributes of the entities into the attributes of the edges, and determining the amount and number of joins further include:

[0026] If each path does not contain duplicate entities, then when calculating each 1-degree path, paths with duplicate entities are filtered out, where each 1-degree path represents a relationship of 1 degree with the entity.

[0027] In one feasible approach, the step of querying the data graph based on the GraphFrame framework of the join data volume and number of joins includes:

[0028] Based on the GraphFrame framework with the amount and number of joins, calculate the number of entities and edges, export all paths of the data graph query, export all related attributes of the entities and edges, and export JSON data for data graph display.

[0029] A second aspect of this application provides a knowledge graph massive data graph query system, applied to the aforementioned knowledge graph massive data graph query method, the system comprising:

[0030] The acquisition unit is used to acquire the list of entities at the beginning and end of the data graph, wherein the list of entities from the beginning to the end includes Hive tables of entities and edges;

[0031] A filtering unit is used to filter the Hive tables of the entities and the edges according to preset filtering conditions.

[0032] The entity and edge set unit is used to determine the set of entities and edges based on the filtered Hive table of entities and edges;

[0033] A GraphFrame unit is used to put the attributes of the entity into the attributes of the edge, import the set of edges into the GraphFrame, and determine the amount and number of joins. The attributes of the edges in the set of edges contain entity information.

[0034] The result unit is used to query the data graph based on the GraphFrame framework of the join data volume and number of joins.

[0035] A third aspect of this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned knowledge graph massive data graph query method.

[0036] A fourth aspect of this application provides a computer storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the aforementioned knowledge graph massive data graph query method.

[0037] Beneficial effects of this invention:

[0038] This application provides a method and system for querying massive data graphs in a knowledge graph. First, it obtains a list of starting and ending entities based on the data graph to be queried. Then, based on preset filtering conditions, it filters out some data from entities and edges, reducing the amount of computational data. Next, based on the filtered entity and edge Hive tables, it determines the set of entities and edges, places the attributes of the entities into the attributes of the edges, and imports the set of edges into a GraphFrame framework, reducing the amount and number of joins. When the edge join is complete, the entity information is expanded, which is equivalent to reducing one large table join operation, saving computation time. Finally, based on the GraphFrame framework with the reduced join amount and number of joins, the data graph is queried. By encoding some large fields that occupy a lot of space in the above way, a large amount of storage space is reduced and the join logic is accelerated. For example, when performing batch offline queries on more than 10,000 nodes with a total of over 10 billion entities and edges, and on many-to-many entity relationships of degree 1 or higher, the graph can be generated within 10 minutes according to the requirements, greatly improving query efficiency. Attached Figure Description

[0039] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a method for querying massive amounts of knowledge graph data according to the present invention;

[0041] Figure 2 This is a flowchart illustrating a method for querying massive amounts of knowledge graph data according to the present invention, which uses filters to remove non-working data from entities and edges.

[0042] Figure 3 This is a flowchart illustrating the encoding of fixed attributes of entities and edges in a Hive table for a knowledge graph massive data graph query method according to the present invention.

[0043] Figure 4 This is a flowchart illustrating the process of importing sets of entities and edges into a GraphFrame framework, as part of a knowledge graph massive data graph query method according to the present invention.

[0044] Figure 5 This invention provides a method for querying massive amounts of knowledge graph data, including the software operation flow for batch offline querying of massive knowledge graph data. Detailed Implementation

[0045] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0047] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0048] The following explanations of some terms used in this application are provided to facilitate understanding of the technical solutions presented herein:

[0049] Hive tables are data warehouse tools based on Hadoop that can map structured data files to a database table and provide SQL-like query functionality.

[0050] SRC stands for source, which means source file.

[0051] DST stands for destination, which means destination.

[0052] Spark applications refer to Spark programs, which are a unified analytics engine for large-scale data processing. They can be used for batch processing, interactive queries (Spark SQL), real-time stream processing (Spark Streaming), machine learning (Spark MLlib), and graph computation (GraphX).

[0053] I / O stands for Input / Output, which represents the input and output of data.

[0054] GraphFrame is a graph processing library built on top of DataFrame. It refers to the graph processing framework on Apache Spark, which can leverage the good scalability and powerful performance of DataFrame, while also providing a unified graph processing API for Scala, Java and Python. GraphFrame can unify the graph algorithms in Spark to the graph operation API interface of DataFrame.

[0055] A degree of 1 refers to the number of entities that an entity is associated with through an edge. For example, if an entity is associated with a second entity through an edge, and then that second entity is associated with a third entity through an edge, this is a degree of 2. A degree of 1 represents all entities that the first entity is associated with through an edge.

[0056] This application provides a method for querying massive amounts of knowledge graph data, decomposing the multi-degree queries performed within the original graph database into multiple 1-degree K-layer expansions. Simultaneously, a series of optimizations are made to the computation process to accelerate computation speed; only entity and edge data are required for rapid querying, making it simple to use.

[0057] like Figure 1 As shown, a method for querying massive amounts of knowledge graph data includes:

[0058] S100: Get the list of the start and end entities of the data graph.

[0059] The entity list consists of Hive tables containing both entities and edges, from beginning to end. For example, the entity list starts at A and ends at B, with multiple Hive tables containing both entities and edges from A to B.

[0060] Specifically, entities are connected by edges, and edges represent the relationships between entities. The entities, edges, and relationships between entities can be referred to as entity relationships.

[0061] Obtain the start and end entity lists of the data graph in the graph database. Based on the start and end of the entity lists, the intermediate path data can be obtained.

[0062] It should be noted that this involves retrieving multiple Hive tables for entities in the entity list, as well as multiple Hive tables for edges between entities.

[0063] The entity's Hive table includes an ID field, while the edge's Hive table includes SRC and DST fields. The ID field in the entity's Hive table enables location, while the SRC and DST fields in the edge's Hive table indicate the source operand and destination operand. The source operand represents the "number" to be operated on, utilizing the edge operations described in this application. The destination operand is the result of the operation performed on the source operand, outputting it to a specific address; that address is the target operand.

[0064] S200: Filter the Hive tables of entities and edges according to preset filtering conditions.

[0065] The filtering conditions are equivalent to setting filters, which remove non-working data from entities and edges, thus cleaning the data in entities and edges. For example... Figure 2 As shown, specifically, it includes steps S201 to S203.

[0066] S201: Import the list of the start and end entities of the data graph into the Spark application.

[0067] S202: Load the entities and edges to be used to compute the start entity list to the end entity list into the Spark application.

[0068] S203: Based on preset filtering conditions, filter out entities and edges in the Spark application that do not meet the filtering conditions.

[0069] The process begins by launching the Spark application, loading the start and end entities into the application. Then, the Spark application calculates the path from start to end, loading all necessary entities and edges into the application. Next, based on filtering criteria for entities and edges, data from entities and edges that do not meet the criteria are filtered out to reduce the amount of data required for computation.

[0070] S300: Determine the set of entities and edges based on the filtered entity and edge Hive table.

[0071] The Hive tables for entities and edges include fixed attributes, which are encoded for location and lookup purposes. For example... Figure 3 As shown, the specific steps include S301 to S303.

[0072] S301: Encode the fixed attributes of the Hive tables for entities and edges.

[0073] Among them, the fixed attributes include at least the ID, SRC, and DST of the hive table.

[0074] S302: Encodes a 32-bit random string number into two long integers for the entity and the edge.

[0075] S303: The encoded entities and edges are respectively mapped to form sets of entities and sets of edges.

[0076] Specifically, the ID, SRC, DST, and other fixed entity and edge attribute fields of the Hive tables are encoded. For example, a 32-bit random string number is encoded into two long integers, one for the entity and one for the edge. Using this 32-bit random string encoding reduces storage space for entities and edges, decreasing data I / O during Spark joins and speeding up the process. Furthermore, other numeric fields, such as object_key and table name, are also encoded as long types. This allows entities and edges to be grouped into separate sets for use in subsequent steps.

[0077] S400: Put the entity's attributes into the edge's attributes, import the set of edges into the GraphFrame, and determine the amount of data to be joined and the number of joins.

[0078] This is achieved by incorporating entity attributes into edge attributes, thus imbuing the edge attributes with entity information. Then, the collection of edges is imported into the GraphFrame framework, as shown below. Figure 4 As shown, the specific steps include S401 and S402.

[0079] S401: Put the attributes of the entity into the attributes of the edge, and join the set of edges.

[0080] S402: Limit the number of edges based on the starting and ending lists of entities.

[0081] Specifically, the computation logic of GraphFrame is optimized. For example, current methods for calculating massive amounts of data in knowledge graphs involve joining entities and edges, then joining the entities again to form a 1-degree path. For each additional degree, an additional join of edges and entities is required, meaning two more joins are needed for each additional degree of path. This results in slow data queries, high performance at best, but low efficiency. This application only performs joins on edges, and the entity's attributes are placed into the edge's attributes. Once the edge join is complete, the entity information is expanded. This effectively reduces one large table join operation, saving computation time.

[0082] In addition, limiting the number of join edges based on the starting and ending entities will also significantly shorten the overall join time.

[0083] It should be noted that if each path does not contain duplicate entities, then when calculating each 1-degree path, paths containing duplicate entities are filtered out.

[0084] Each 1-degree path represents a 1-degree relationship with an entity.

[0085] If a path contains multiple duplicate entities, it increases both computational complexity and storage space. Therefore, to ensure that each calculated path does not contain duplicate entities, paths with duplicate entities are filtered out when calculating each degree of the path. Subsequent joins will then have significantly less duplicate data, thus speeding up computation.

[0086] S500: GraphFrame framework for querying data graphs based on the amount and number of joins.

[0087] Specifically, based on the GraphFrame framework with the amount and number of joins, the number of entities and edges is calculated, and all paths of the data graph query are exported, along with the relevant attributes of all entities and edges, and JSON data for data graph display.

[0088] Specifically, it calculates a dataframe containing all paths, and can calculate the number of all entities and edges as needed, export all paths, export the relevant attributes of all entities and edges, and export JSON data for graph display.

[0089] It should be noted that in the aforementioned steps, the data has already been encoded into two long integers using 32-bit random string numbers. Therefore, after the calculation, these fields need to be encoded back in reverse order to restore the entities and edges.

[0090] like Figure 5 As shown, the exemplary workflow of the batch offline query software for massive knowledge graph data is as follows:

[0091] The query conditions are assembled by Program-01 and then handed over to Program-02 for step 2 query.

[0092] Program-02 uses Spark GraphFrame distributed computing technology to query entity and relationship data step by step and assemble the final result.

[0093] In summary, this invention can leverage the features of Spark to perform graph query computations on massive datasets. It utilizes SparkGraphFrame for graph queries and optimizes numerous customized query logics. Furthermore, it encodes some large fields that consume significant space, reducing storage space and accelerating join logic. As a result, it can generate graphs within 10 minutes when performing K-level expansions of over 10,000 nodes and batch offline queries of many-to-many entity relationships with a degree of 1 or higher, even with a total of over 10 billion entities and relationships. This significantly improves query efficiency.

[0094] A second aspect of this application provides a knowledge graph massive data graph query system, applied to the aforementioned knowledge graph massive data graph query method, the system comprising:

[0095] The acquisition unit is used to acquire the list of entities at the beginning and end of the data graph, wherein the list of entities from the beginning to the end includes Hive tables of entities and edges;

[0096] A filtering unit is used to filter the Hive tables of the entities and the edges according to preset filtering conditions.

[0097] The entity and edge set unit is used to determine the set of entities and edges based on the filtered Hive table of entities and edges;

[0098] A GraphFrame unit is used to put the attributes of the entity into the attributes of the edge, import the set of edges into the GraphFrame, and determine the amount and number of joins. The attributes of the edges in the set of edges contain entity information.

[0099] The result unit is used to query the data graph based on the GraphFrame framework of the join data volume and number of joins.

[0100] A third aspect of this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned knowledge graph massive data graph query method.

[0101] A fourth aspect of this application provides a computer storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the aforementioned knowledge graph massive data graph query method.

[0102] In the description of the embodiments of the present invention, those skilled in the art should understand that the embodiments of the present invention can be implemented as methods, apparatuses, electronic devices, and computer-readable storage media. Therefore, the embodiments of the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Furthermore, in some embodiments, the embodiments of the present invention can also be implemented as a computer program product in one or more computer-readable storage media, the computer-readable storage media containing computer program code.

[0103] The aforementioned computer-readable storage medium may be any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any combination thereof. In embodiments of the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0104] The computer program code contained in the aforementioned computer-readable storage medium may be transmitted using any suitable medium, including wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0105] Computer program code for performing the operations of the embodiments of the present invention can be written in assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The computer program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer or an external computer via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0106] The embodiments of the present invention describe the provided methods, apparatus, and electronic devices through flowcharts and / or block diagrams.

[0107] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine that, when executed by a computer or other programmable data processing apparatus, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0108] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to function in a particular manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction apparatus product that includes the functions / operations specified in the blocks of a flowchart and / or block diagram.

[0109] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable data processing apparatus provide a process for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0110] In the specification and claims of this invention, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0111] In embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0112] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for querying massive amounts of knowledge graph data, characterized in that, The methods include: Obtain the start and end entity lists of the data graph, wherein the entity lists from the start to the end include Hive tables of entities and edges; Filter the Hive tables of the entities and the edges according to preset filtering conditions; Based on the filtered Hive table of the entities and edges, determine the set of the entities and edges; The attributes of the entity are placed into the attributes of the edge, the set of edges is imported into the GraphFrame framework, and the amount and number of joins are determined. The attributes of the edges in the set of edges contain entity information. Based on the GraphFrame framework of the join data volume and number of joins, query the data graph; The step of determining the set of entities and edges based on the filtered Hive table of entities and edges includes: The fixed attributes of the Hive tables of the entity and the edge are encoded, wherein the fixed attributes of the entity include at least the ID of the Hive table, and the fixed attributes of the edge include at least the SRC and DST of the Hive table. Encode the 32-bit random string number into two long integers, with the two long integers corresponding to the entity and the edge, respectively; The encoded entities and edges are respectively used to form a set of entities and a set of edges; The steps of placing the entity's attributes into the edge's attributes, importing the set of edges into a GraphFrame, and determining the amount and number of joins include: The attributes of the entity are placed into the attributes of the edge, and the set of edges is joined. The number of edges is limited based on the list of starting and ending entities; The step of querying the data graph based on the GraphFrame framework with the amount and number of joins includes: Based on the GraphFrame framework with the amount and number of joins, calculate the number of entities and edges, export all paths of the data graph query, export all related attributes of the entities and edges, and export JSON data for data graph display.

2. The method for querying massive data graphs in a knowledge graph according to claim 1, characterized in that, The step of obtaining the list of the start and end entities of the data graph includes: Obtain multiple Hive tables for the entities in the entity list and multiple Hive tables for the edges between the entities, wherein the Hive tables for the entities include an ID field, and the Hive tables for the edges include SRC and DST fields.

3. The method for querying massive data graphs in a knowledge graph according to claim 1, characterized in that, The step of filtering the Hive table of the entity and the edge according to preset filtering conditions includes: Import the list of the start and end entities of the data graph into the Spark application; Load the entities and edges to be used in calculating the start entity list to the end entity list into the Spark application; Based on preset filtering conditions, entities and edges that do not meet the filtering conditions are filtered out in the Spark application.

4. The method for querying massive data graphs in a knowledge graph according to claim 1, characterized in that, The steps of importing the set of entities and the set of edges into a GraphFrame, placing the attributes of the entities into the attributes of the edges, and determining the amount and number of joins further include: If each path does not contain duplicate entities, then when calculating each 1-degree path, paths with duplicate entities are filtered out, where each 1-degree path represents a relationship of 1 degree with the entity.

5. A knowledge graph massive data graph query system, characterized in that, The system, which is applied to the knowledge graph massive data graph query method according to any one of claims 1-4, comprises: The acquisition unit is used to acquire the list of entities at the beginning and end of the data graph, wherein the list of entities from the beginning to the end includes Hive tables of entities and edges; A filtering unit is used to filter the Hive tables of the entities and the edges according to preset filtering conditions. The entity and edge set unit is used to determine the set of entities and edges based on the filtered Hive table of entities and edges; A GraphFrame unit is used to put the attributes of the entity into the attributes of the edge, import the set of edges into the GraphFrame, and determine the amount and number of joins. The attributes of the edges in the set of edges contain entity information. The result unit is used to query the data graph based on the GraphFrame framework of the join data volume and number of joins.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the knowledge graph massive data graph query method according to any one of claims 1 to 4.

7. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the knowledge graph massive data graph query method as described in any one of claims 1 to 4.

Images