Knowledge graph extension updating method and device

Abstract

The application discloses an extension updating method of a knowledge graph. When new nodes need to be added in an existing knowledge graph, it is judged whether the number of the new nodes is less than or equal to a preset parameter N; if yes, all the new nodes are put into a priority calculation queue, and the new nodes in the priority calculation queue are extended and updated into the existing knowledge graph in a real-time calculation mode. If no, N nodes are randomly selected from the new nodes and put into the priority calculation queue, and the remaining part of the new nodes is put into a common calculation queue; the new nodes in the priority calculation queue are extended and updated into the existing knowledge graph in a real-time calculation mode; and the new nodes in the common calculation queue wait for hardware calculation resources to be idle and then are calculated and extended and updated into the existing knowledge graph. The application shortens the time for a user to experience the partially extended and updated knowledge graph, and enables the extension and update of the knowledge graph to be applied to more real-time business scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to a method of graph computing, and in particular to a method of extending and updating a knowledge graph. BACKGROUND

[0002] A knowledge graph is a graph in which entities are represented by nodes and relationships between entities are represented by edges, and can be used for graph query, graph analysis or graph computing. After the initial construction of a knowledge graph, it will still be continuously extended and updated, that is, new nodes are added to the current knowledge graph, and new edges are added between the newly added nodes, and / or new edges are added between the newly added nodes and the original nodes.

[0003] One existing method of extending and updating a knowledge graph is to use real-time updating. The disadvantage is that if the amount of data to be updated is large, the time-consuming of extending and updating will be long, and during the extending and updating, the hardware computing resources will be greatly occupied, so that the use demand of graph query, graph analysis or graph computing cannot be responded in time.

[0004] Another existing method of extending and updating a knowledge graph is to use non-real-time updating, that is, to perform extending and updating when the hardware computing resources are idle. The disadvantage is that if the knowledge graph is in a busy state of graph query, graph analysis or graph computing for a long time, the work of extending and updating will continue to wait for the hardware computing resources to be idle, so that when the extending and updating is performed, the timeliness has been lost. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a method of extending and updating a knowledge graph, which preferentially calculates part of the newly added nodes to achieve the purpose of providing a part of the extended and updated knowledge graph as soon as possible, and then calculates the remaining newly added nodes. For this purpose, the present application also provides an apparatus for extending and updating a knowledge graph.

[0006] To solve the above technical problems, the knowledge graph extension updating method provided by the present application comprises the following steps. Step S10: When new nodes need to be added in the existing knowledge graph, it is judged whether the number of the added nodes is less than or equal to a preset parameter N; if yes, step S20 is entered; if no, step S30 is entered. Step S20: All the added nodes are put into a priority calculation queue, and the added nodes in the priority calculation queue are extended and updated to the existing knowledge graph in a real-time calculation manner. Step S30: N nodes are randomly selected from the added nodes and put into the priority calculation queue, and the remaining part of the added nodes is put into a normal calculation queue; the added nodes in the priority calculation queue are extended and updated to the existing knowledge graph in a real-time calculation manner; the added nodes in the normal calculation queue wait for the hardware computing resources to be idle before being calculated and extended and updated to the existing knowledge graph. The above method can provide the user with a part of the extended and updated knowledge graph in real time, attract the user, and at the same time avoid causing a large computing burden and a large computing time consumption to the graph database where the knowledge graph is located.

[0007] Preferably, in the step S10, a Neo4j graph database is used to store the knowledge graph.

[0008] Preferably, in the step S10, the edges between the nodes in the knowledge graph are calculated by Apache Spark.

[0009] Further, in the step S20, while the priority calculation queue is processed, new edges are added between the added nodes in the priority calculation queue, and / or new edges are added between the added nodes in the priority calculation queue and the original nodes.

[0010] Further, in the step S30, while the normal calculation queue is processed, new edges are added between the added nodes in the normal calculation queue, and / or new edges are added between the added nodes in the normal calculation queue and the existing nodes.

[0011] Further, when new nodes are added in the existing knowledge graph, the number of the added nodes that can be processed by the hardware computing resources within a preset time is set as the parameter N.

[0012] Further, the parameter N represents the calculation speed, and the larger the parameter N is, the longer the calculation time is.

[0013] The application further provides an extension updating device of a knowledge graph, which comprises a judging unit, a real-time calculation unit and an idle calculation unit. The judging unit is used for judging whether the number of new nodes to be added in the existing knowledge graph is less than or equal to a preset parameter N when the new nodes are needed to be added in the existing knowledge graph; if yes, all the new nodes are sent to the real-time calculation unit; if no, N nodes are randomly selected from the new nodes and sent to the real-time calculation unit, and the remaining part of the new nodes is sent to the idle calculation unit. The real-time calculation unit is used for extending and updating the received new nodes to the existing knowledge graph in a real-time calculation mode. The idle calculation unit is used for calculating and extending and updating the received new nodes to the existing knowledge graph after the hardware calculation resource is idle. The above device can provide the user with the partially extended and updated knowledge graph in real time, attract the user, and meanwhile avoid causing large calculation burden and large calculation time consumption to the graph database where the knowledge graph is located.

[0014] Further, the new nodes received by the real-time calculation unit are used as a priority calculation queue; the real-time calculation unit is further used for adding new edges between the new nodes in the priority calculation queue in a real-time calculation mode, and / or adding new edges between the new nodes in the priority calculation queue and the original nodes.

[0015] Further, the new nodes received by the idle calculation unit are used as a common calculation queue; the idle calculation unit is further used for adding new edges between the new nodes in the common calculation queue when the hardware calculation resource is idle, and / or adding new edges between the new nodes in the common calculation queue and the existing nodes.

[0016] The application achieves the technical effect of greatly shortening the time for the user to experience the partially extended and updated knowledge graph, enabling the extension and update of the knowledge graph to be applied to more real-time business scenarios, and improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of the extension updating method of the knowledge graph provided by the application.

[0018] Figure 2 is a structural diagram of the extension updating device of the knowledge graph provided by the application.

[0019] In the drawing, the reference signs are as follows: 10 is a judging unit, 20 is a real-time calculation unit, and 30 is an idle calculation unit. DETAILED DESCRIPTION

[0020] Please refer to Figure 1 The extension updating method of the knowledge graph provided by the application comprises the following steps.

[0021] Step S10: When the existing knowledge graph needs to add new nodes, determine whether the number of new nodes is less than or equal to a preset parameter N. If yes, go to step S20. If no, go to step S30.

[0022] In this step, the existing knowledge graph is stored in a graph database, which refers to a database that stores and queries data in the form of a graph data structure. For example, this step uses a Neo4j graph database to store the existing knowledge graph, and the relationships (edges) between the nodes in the existing knowledge graph are calculated by Apache Spark; then the relationship path can be queried by the query language of Neo4j and the corresponding algorithm. Apache Spark is a unified analysis engine for large-scale data processing.

[0023] Step S20: Put all the new nodes into the priority calculation queue, and use real-time calculation to extend and update the new nodes in the priority calculation queue to the existing knowledge graph; at the same time, add new edges between the new nodes in the priority calculation queue, and / or add new edges between the new nodes in the priority calculation queue and the original nodes.

[0024] Step S30: Randomly select N nodes from the new nodes and put them into the priority calculation queue, and put the remaining new nodes into the ordinary calculation queue. The new nodes in the priority calculation queue are extended and updated to the existing knowledge graph by real-time calculation; at the same time, new edges are added between the new nodes in the priority calculation queue, and / or new edges are added between the new nodes in the priority calculation queue and the original nodes. The new nodes in the ordinary calculation queue wait for the hardware computing resources to be idle before being calculated and extended and updated to the existing knowledge graph; at the same time, new edges are added between the new nodes in the ordinary calculation queue, and / or new edges are added between the new nodes in the ordinary calculation queue and the existing nodes (including the original nodes and the new nodes in the priority calculation queue).

[0025] The parameter N in the step S10 represents the calculation speed, and the larger N is, the longer the calculation time is. The size of the parameter N depends on the processing capacity of the hardware computing resource, and can be fixed or variable. Preferably, when new nodes are added to the existing knowledge graph, the hardware computing resource (such as the processor, memory, etc. of the server cluster) can process the number of new nodes in a short time (such as 1 second, 5 seconds, 10 seconds, 15 seconds, 30 seconds, 1 minute, etc.) and the number of new nodes that can be processed in a short time is set as N. In the step S30, the priority calculation queue is used to complete the processing of part of the new nodes in a short time, so that the user can be provided with part of the extended and updated knowledge graph in time, the waiting time of the user is reduced, and the user is prevented from leaving due to too long waiting time; the user is allowed to perform graph query, graph analysis or graph calculation in the part of the extended and updated knowledge graph. The part of the extended and updated knowledge graph has the characteristic of high timeliness, and the completeness of the new data is sacrificed. The general calculation queue is used to process all new nodes, so that the user can be provided with all of the extended and updated knowledge graph; the user is allowed to perform graph query, graph analysis or graph calculation in all of the extended and updated knowledge graph. The all of the extended and updated knowledge graph has the completeness of the new data, and the timeliness is sacrificed.

[0026] In a typical application scenario, a knowledge graph is used to record the social connections (also referred to as interpersonal relationships, interpersonal networks, relation, relationship, connection). In the constructed knowledge graph, the nodes are divided into unit nodes and personal nodes; the first type of edge between the personal nodes represents that one party saves the other party in the friend relationship (such as the publicly available friend relationship in microblog, LinkedIn, etc.); the second type of edge between the personal nodes and the unit nodes represents the employment relationship of the personal in the unit. All data is obtained on the premise of obtaining the consent of the user, so as to fully respect the privacy of the user data. When new nodes need to be added to the existing knowledge graph, if the number of new nodes is large (more than N), the existing method of extending and updating the knowledge graph needs a long calculation time. After the method of extending and updating the knowledge graph of the present application is used, the first N new nodes are calculated in real time, and the remaining new nodes are calculated in idle time; so that part of the new social connection data is provided to the user in time, a good user experience is provided, and all of the new social connection data is provided to the user subsequently.

[0027] Please refer to Figure 2 The device for extending and updating the knowledge graph of the present application includes a judgment unit 10, a real-time calculation unit 20 and an idle calculation unit 30.

[0028] The judging unit 10 is used for judging whether the number of the new nodes to be added in the existing knowledge graph is less than or equal to a preset parameter N when the existing knowledge graph needs to add new nodes. If yes, all the added nodes are sent to the real-time computing unit 20. If no, N nodes are randomly selected from the added nodes and sent to the real-time computing unit 20, and the remaining part of the added nodes is sent to the idle computing unit 30.

[0029] The real-time computing unit 20 is used for extending and updating the received added nodes to the existing knowledge graph in a real-time computing manner, and meanwhile adding new edges between the added nodes in the priority computing queue and / or adding new edges between the added nodes in the priority computing queue and the original nodes. The added nodes received by the real-time computing unit 20 are as the priority computing queue.

[0030] The idle computing unit 30 is used for waiting for the hardware computing resources to be idle before computing and extending and updating the received added nodes to the existing knowledge graph, and meanwhile adding new edges between the added nodes in the ordinary computing queue and / or adding new edges between the added nodes in the ordinary computing queue and the existing nodes (including the original nodes and the added nodes in the priority computing queue). The added nodes received by the idle computing unit 30 are as the ordinary computing queue.

[0031] In the application, if the number of the added nodes is less than or equal to the parameter N, all the added nodes are extended and updated to the existing knowledge graph in a real-time computing manner; if the number of the added nodes is greater than the parameter N, N added nodes are randomly selected and extended and updated to the existing knowledge graph in a real-time computing manner, and the remaining added nodes are computed and extended and updated to the existing knowledge graph when the hardware computing resources are idle. In this way, the user can be provided with the partially extended and updated knowledge graph in time, and the user can be provided with the real-time experience; and then the user can be provided with the completely extended and updated knowledge graph, and the user can be provided with the experience of complete data volume.

[0032] The above is only the preferred embodiment of the application, and is not used to limit the application. The application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for extended updating of a knowledge graph, characterized in that, Comprising the following steps: Step S10: When new nodes need to be added in the existing knowledge graph, the number of new nodes that can be processed by hardware computing resources within a preset time is set as parameter N, and it is judged whether the number of new nodes is less than or equal to the preset parameter N; If yes, go to step S20; if no, go to step S30; Step S20: Put all the new nodes into the priority calculation queue, and extend and update the new nodes in the priority calculation queue to the existing knowledge graph in a real-time calculation manner; Step S30: Randomly select N nodes from the new nodes and put them into the priority calculation queue, and put the remaining part of the new nodes into the ordinary calculation queue; extend and update the new nodes in the priority calculation queue to the existing knowledge graph in a real-time calculation manner; The new nodes in the ordinary calculation queue wait for the hardware computing resources to be idle before being calculated and extended and updated to the existing knowledge graph. 2.The method of claim 1, wherein, In the step S10, the Neo4j graph database is used to store the knowledge graph.

3. The method of claim 1, wherein the knowledge graph is updated based on the identified new entity and the identified new relationship. In the step S10, the edges between the nodes in the knowledge graph are calculated by Apache Spark. 4.The method of claim 1, wherein, In the step S20, while processing the priority calculation queue, new edges are added between the new nodes in the priority calculation queue, and / or new edges are added between the new nodes in the priority calculation queue and the original nodes. 5.The method of claim 4, wherein, In the step S30, while processing the ordinary calculation queue, new edges are added between the new nodes in the ordinary calculation queue, and / or new edges are added between the new nodes in the ordinary calculation queue and the existing nodes. 6.The method of claim 1, wherein, The parameter N represents the calculation speed, and the larger the parameter N is, the longer the calculation time is.

7. An apparatus for extended update of a knowledge graph, characterized by Comprising a judgment unit, a real-time calculation unit and an idle calculation unit; The judgment unit is used to set the number of new nodes that can be processed by hardware computing resources within a preset time as parameter N when new nodes need to be added in the existing knowledge graph, and judge whether the number of new nodes is less than or equal to the preset parameter N; If yes, all the new nodes are sent to the real-time calculation unit; if no, N nodes are randomly selected from the new nodes and sent to the real-time calculation unit, and the remaining part of the new nodes is sent to the idle calculation unit; The real-time calculation unit is used to extend and update the received new nodes to the existing knowledge graph in a real-time calculation manner; The idle calculation unit is used to wait for the hardware computing resources to be idle before calculating and extending and updating the received new nodes to the existing knowledge graph.

8. The apparatus according to claim 7, characterized in that, The new nodes received by the real-time calculation unit serve as the priority calculation queue; the real-time calculation unit is also used to add new edges between the new nodes in the priority calculation queue in a real-time calculation manner, and / or add new edges between the new nodes in the priority calculation queue and the original nodes. 9.The device for extended update of a knowledge graph according to claim 7, characterized in that, The new nodes received by the idle calculation unit serve as the ordinary calculation queue; the idle calculation unit is also used to add new edges between the new nodes in the ordinary calculation queue when the hardware computing resources are idle, and / or add new edges between the new nodes in the ordinary calculation queue and the existing nodes.

Images