Knowledge graph generation task construction method and device, electronic equipment and storage medium

Abstract

The application discloses a knowledge graph generation task construction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a preset knowledge graph ontology and a knowledge graph construction data source, determining a mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source according to user configuration information, and determining a graph construction subtask executed by at least one calculation node based on the preset knowledge graph ontology and the mapping relationship. According to the embodiment of the application, the single task of constructing the knowledge graph is divided into a plurality of graph construction subtasks which can be independently executed in series or in parallel, so that the construction efficiency and fault tolerance of the knowledge graph can be improved. Meanwhile, the construction process of the subtask is relatively simple and flexible, and the practicality is higher.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and in particular to a knowledge graph generation task construction method and device, electronic equipment and a storage medium. BACKGROUND

[0002] The development of artificial intelligence has three important stages: operational intelligence, perceptual intelligence and cognitive intelligence. Cognitive intelligence refers to the realization of information cognition, understanding, reasoning and decision-making, and the realization of the cognition and collaboration of intelligent entities such as people, things and enterprises. In the field of cognitive intelligence, knowledge graph plays an important role. Knowledge graph is a knowledge organization form based on natural language processing (NLP) and combined with applied mathematics, graphics, information visualization technology and information science. It can be regarded as a large semantic knowledge network, and it has a wide range of applications in intelligent search, intelligent recommendation, financial risk control and smart city.

[0003] The existing knowledge graph generation task construction method, whether based on distributed clusters or single construction, mostly adopts a one-time construction method for complete knowledge graphs. This type of method is relatively complex in construction process and is not flexible enough, resulting in low efficiency in constructing knowledge graphs. At the same time, parallel execution of sub-tasks can lead to low fault tolerance in knowledge graph construction, which in turn leads to poor practicality. That is, when a sub-task fails, all parallel tasks need to be executed again to construct a complete knowledge graph. Therefore, it is of great significance to study the knowledge graph generation task construction method. SUMMARY

[0004] The present application provides a knowledge graph generation task construction method, device, electronic equipment and storage medium to solve the problem of complex construction process, poor flexibility, low efficiency and fault tolerance of knowledge graph construction caused by the existing knowledge graph construction method which can only construct single task.

[0005] According to an aspect of the present application, a knowledge graph generation task construction method is provided, wherein the method comprises:

[0006] Obtaining a preset knowledge graph ontology and a knowledge graph construction data source;

[0007] Determining the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source according to user configuration information;

[0008] Determining at least one graph construction sub-task executed by a computing node based on the preset knowledge graph ontology and the mapping relationship.

[0009] According to another aspect of the present application, there is provided a knowledge graph generation task construction apparatus, comprising:

[0010] a data acquisition module configured to acquire a preset knowledge graph ontology and a knowledge graph construction data source;

[0011] a mapping determination module configured to determine a mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source according to user configuration information;

[0012] a task construction module configured to determine at least one graph construction subtask executed by a computing node based on the preset knowledge graph ontology and the mapping relationship.

[0013] According to another aspect of the present application, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory in communication with the at least one processor; wherein

[0016] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the knowledge graph generation task construction method according to any one of the embodiments of the present application.

[0017] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to implement the knowledge graph generation task construction method according to any one of the embodiments of the present application when executed by the processor.

[0018] The technical solution of the embodiments of the present application acquires a preset knowledge graph ontology and a knowledge graph construction data source, determines a mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source according to user configuration information, and determines at least one graph construction subtask executed by a computing node based on the preset knowledge graph ontology and the mapping relationship. The embodiments of the present application divide a single task of constructing a knowledge graph based on a preset knowledge graph ontology into a plurality of graph construction subtasks, the subtask splitting process is simple and flexible, each subtask can be independently executed in series or in parallel, the construction efficiency of the knowledge graph can be improved, the construction process of the knowledge graph has a high fault tolerance rate, and has good practicability.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a knowledge graph generation task construction method provided in Embodiment 1 of the present invention;

[0022] Figure 2 This is a flowchart of a knowledge graph generation task construction method provided in Embodiment 2 of the present invention;

[0023] Figure 3 This is an example diagram of a knowledge graph construction method provided in Embodiment 2 of the present invention;

[0024] Figure 4 This is a flowchart of a single-task knowledge graph construction method according to Embodiment 2 of the present invention;

[0025] Figure 5 This is a flowchart of a batch construction of a knowledge graph according to Embodiment 2 of the present invention;

[0026] Figure 6 This is an example diagram of a knowledge graph ontology provided in Embodiment 2 of the present invention;

[0027] Figure 7 This is a knowledge graph provided according to Embodiment 2 of the present invention. Figure 1 Example diagram;

[0028] Figure 8 This is a knowledge graph provided according to Embodiment 2 of the present invention. Figure 2 Example diagram;

[0029] Figure 9 This is a knowledge graph provided according to Embodiment 2 of the present invention. Figure 3 Example diagram;

[0030] Figure 10 This is an example diagram of the execution of a sub-graph task single task according to Embodiment 2 of the present invention;

[0031] Figure 11 This is an example diagram of distributed parallel execution of subgraph tasks according to Embodiment 2 of the present invention;

[0032] Figure 12 This is a flowchart of a knowledge graph generation task construction method provided in Embodiment 3 of the present invention;

[0033] Figure 13This is an example diagram of a knowledge graph construction method provided in Embodiment 3 of the present invention;

[0034] Figure 14 This is an example diagram of an entity provided according to Embodiment 3 of the present invention;

[0035] Figure 15 This is a schematic diagram of a knowledge graph generation task construction device provided in Embodiment 4 of the present invention;

[0036] Figure 16 This is a schematic diagram of the structure of an electronic device that implements the knowledge graph generation task construction method of the present invention. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0039] Example 1

[0040] Figure 1 This is a flowchart of a knowledge graph generation task construction method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where a knowledge graph is split into sub-tasks for graph construction. This method can be executed by a knowledge graph generation task construction device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown in the figure, the knowledge graph generation task construction method provided in this embodiment includes the following steps:

[0041] S110. Obtain the preset knowledge graph ontology and the knowledge graph construction data source.

[0042] In this embodiment of the invention, the preset knowledge graph ontology can be understood as a semantic data model describing things and the relationships between things in a specific business scenario. The preset knowledge graph ontology may include entities, relationships, and attributes. The knowledge graph construction data source can be understood as the data source used to construct the knowledge graph. The knowledge graph construction data source may include structured data, semi-structured data, and unstructured data related to actual business.

[0043] Specifically, based on actual business needs, corresponding preset knowledge graph ontology and knowledge graph construction data sources can be obtained. The methods for obtaining preset knowledge graph ontology may include, but are not limited to: manual drawing by users according to actual business needs, ontology drawing using ontology construction tools (such as Protege), and direct use of open-source general ontology data models. The methods for obtaining knowledge graph construction data sources may include, but are not limited to: obtaining from business-related open-source databases, obtaining from industry-internal business databases, obtaining from encyclopedic data sources (such as Baidu Encyclopedia, Wikipedia, etc.), and crawling from the Internet using web crawlers.

[0044] S120. Determine the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source based on the user configuration information.

[0045] In this embodiment of the invention, user configuration information can be understood as pre-configured information containing the mapping relationship between a preset knowledge graph ontology and a knowledge graph construction data source. The user configuration information may include the correspondence between entities within the preset knowledge graph ontology and the knowledge graph construction data source, as well as the correspondence between relationships within the preset knowledge graph ontology and the knowledge graph construction data source. The mapping relationship can be understood as the correspondence between the preset knowledge graph ontology and the knowledge graph construction data source. The mapping relationship may include the mapping relationship between entities within the preset knowledge graph ontology and the knowledge graph construction data source, as well as the mapping relationship between relationships within the preset knowledge graph ontology and the knowledge graph construction data source.

[0046] Specifically, the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source can be determined based on the preset user configuration information. The methods for determining the mapping relationship may include, but are not limited to, the following: the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source can be determined based on the configuration items of the custom mapping relationship in the user configuration information; the mapping relationship between the field names corresponding to the attributes of each entity and relation in the preset knowledge graph ontology and the field names in the knowledge graph construction data source can be found in the user configuration information, thereby determining the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source.

[0047] S130. Based on the preset knowledge graph ontology and mapping relationship, determine at least one computing node to perform a graph construction subtask.

[0048] In this embodiment of the invention, a computing node can be understood as a computer and its supporting equipment used to execute the graph construction subtask. The number of computing nodes can be configured according to actual business needs and computing resources, and this invention does not impose any limitations on this. A graph construction subtask can be understood as a subtask used to construct a knowledge graph. The number of graph construction subtasks can be one or more, and the execution method of the graph construction subtasks can include serial execution and parallel execution, etc.

[0049] Specifically, graph construction subtasks can be constructed based on a preset knowledge graph ontology and mapping relationships. These subtasks can be delegated to computing nodes for execution. The construction methods for these subtasks can include, but are not limited to, the following: The preset knowledge graph ontology can be randomly divided into several knowledge graph subgraphs based on computing resources. The corresponding knowledge graph construction data source for each subgraph is determined based on the mapping relationships, and graph construction subtasks are constructed for the knowledge graph construction data sources belonging to the same subgraph. Alternatively, the preset knowledge graph ontology can be evenly divided into several knowledge graph subgraphs based on computing resources and the amount of entity and relation data within the ontology. The corresponding knowledge graph construction data source for each subgraph is determined based on the mapping relationships, and graph construction subtasks are constructed for the knowledge graph construction data sources belonging to the same subgraph. Finally, based on the amount of entity and relation data within the preset knowledge graph ontology and the mapping relationships between the preset knowledge graph ontology and the knowledge graph construction data sources, several entity construction subtasks and several relation construction subtasks can be constructed separately, and all entity construction subtasks and relation construction subtasks can be combined as a single graph construction subtask. Furthermore, computing nodes can choose to execute the graph construction subtask in a serial and / or parallel manner.

[0050] The technical solution of this invention obtains a preset knowledge graph ontology and a knowledge graph construction data source, determines the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source based on user configuration information, and determines at least one graph construction subtask to be executed by a computing node based on the preset knowledge graph ontology and the mapping relationship. This invention breaks down the knowledge graph construction task into multiple graph construction subtasks based on the preset knowledge graph ontology and the mapping relationship, and assigns them to at least one computing node for execution. The subtask splitting process is simpler and more flexible, improving the efficiency and fault tolerance of knowledge graph construction, and has good practicality.

[0051] Example 2

[0052] Figure 2 This is a flowchart of a knowledge graph generation task construction method provided in Embodiment 2 of the present invention. It is further optimized and extended based on the above embodiments and can be combined with various optional technical solutions in the above embodiments. For example... Figure 2 As shown in the figure, the knowledge graph generation task construction method provided in this embodiment includes the following steps:

[0053] S210. Collect local ontology drawing data from the preset knowledge graph.

[0054] In this embodiment of the invention, ontology drawing data can be understood as data used to draw knowledge graph ontology, and ontology drawing data may include several entities and the relationships between entities, etc.

[0055] Specifically, ontology drawing data can be collected locally from a pre-defined knowledge graph. The methods for obtaining ontology drawing data may include, but are not limited to: manual drawing by users according to business needs, ontology drawing using ontology building tools (such as Protege tools), or direct use of open-source general ontology data models.

[0056] S220. Select at least one type of knowledge graph to construct a data source based on the user's selection information.

[0057] In this embodiment of the invention, the user selection information can be understood as information used to select the data source for building the knowledge graph. The user selection information may include information such as the database name, database address, and Java Database Connectivity (JDBC) of the data source for building the knowledge graph.

[0058] Specifically, the required knowledge graph construction data source can be selected based on pre-configured user selection information. The selection methods for the data source may include, but are not limited to: users can manually select one or more knowledge graph construction data sources in a visual interface for selecting knowledge graph construction data sources, based on user selection information such as database name, database address, and JDBC information. Furthermore, the knowledge graph construction data source may include attribute information such as data scale, data type, and data storage method; users can also automatically select one or more knowledge graph construction data sources using database tools based on user selection information and actual business needs.

[0059] S230. Extract the entity data source relationship pairs from the user configuration information.

[0060] In this embodiment of the invention, an entity can be understood as something that is distinguishable and exists independently. For example, an entity may include specific people, events, and things, or it may include abstract concepts or relationships. Entities are the most basic elements in a knowledge graph. An entity data source relationship pair can be understood as a set of relationships composed of mapping relationships between entities and data sources. The entity data source relationship pair can be determined by the mapping relationship between entities within a preset knowledge graph ontology and the mapping relationship between the knowledge graph construction data source, as well as the mapping relationship between relationships within the preset knowledge graph ontology and the knowledge graph construction data source.

[0061] Specifically, entity data source relationship pairs can be extracted from preset user configuration information. These entity data source relationship pairs can be determined by, but are not limited to, the following: entity data - mapping relationship - data source, and relationship data - mapping relationship - data source. The mapping relationship here can include, but is not limited to, database address mapping and field mapping.

[0062] S240. Based on the entity data source relationship, establish mapping relationships between the entities and relationships within the preset knowledge graph body and the knowledge graph construction data source respectively.

[0063] Specifically, based on the extracted entity data source relationship pairs, mapping relationships can be established between the knowledge graph construction data source and the entities and relationships within the preset knowledge graph ontology. The methods for establishing mapping relationships between entities and relationships can include, but are not limited to: entity data—mapping relationship—data source, relationship data—mapping relationship—data source, etc., in the entity data source relationship. For example, mapping relationships can be established between each entity and relationship within the ontology and the knowledge graph construction data source based on the mapping address from entity data to data source and the mapping address from relationship data to data source; mapping relationships can also be established between the field names corresponding to the attributes of each entity and the attributes of the relationship within the ontology and the field names in the knowledge graph construction data source.

[0064] S250. Divide the preset knowledge graph ontology into at least two knowledge graph subgraphs according to the estimated data of entities and relations within the preset knowledge graph ontology, wherein the estimated data of entities and relations within each knowledge graph subgraph is the same.

[0065] In this embodiment of the invention, the estimated amount of entity and relation data can be understood as an estimated value used to characterize the amount of entity data and relation data.

[0066] Specifically, the preset knowledge graph ontology can be divided into several knowledge graph subgraphs according to the estimated data volume of entities and relations within the preset knowledge graph ontology. Each knowledge graph subgraph can be understood as a miniature knowledge graph. The estimated data volume of entities and relations within each knowledge graph subgraph is the same. The estimated data volume of entities and relations can be determined by the data volume of the knowledge graph construction data source and / or the data storage type, etc.

[0067] S260. Based on the mapping relationship, determine the knowledge graphs corresponding to the entities and relationships within the subgraphs of the knowledge graph and construct the data source.

[0068] Specifically, entities and relations within a knowledge graph subgraph can be mapped according to mapping relationships, which may include, but are not limited to: mapping relationships between entities within a pre-defined knowledge graph ontology and data sources for knowledge graph construction, and mapping relationships between relations within a pre-defined knowledge graph ontology and data sources for knowledge graph construction. This allows for the determination of the data sources for knowledge graph construction corresponding to each entity and each relation.

[0069] S270, For knowledge graphs belonging to the same knowledge graph subgraph, construct data source and construct graph construction subtask.

[0070] Specifically, corresponding graph construction subtasks can be constructed based on the knowledge graph construction data sources belonging to the same knowledge graph subgraph, with each knowledge graph subgraph corresponding to one graph construction subtask.

[0071] S280. Assign each graph construction subtask to each computing node.

[0072] Specifically, each completed graph construction subtask can be assigned to a computing node for execution. The allocation method for graph construction subtasks may include, but is not limited to: if computing resources are insufficient, each graph construction subtask can be assigned to a computing node for sequential execution; if computing resources are sufficient, each graph construction subtask can be assigned to multiple computing nodes for parallel execution, thereby improving the efficiency of knowledge graph construction.

[0073] The technical solution of this invention involves collecting local ontology rendering data of a preset knowledge graph, selecting at least one type of knowledge graph construction data source according to user selection information, extracting entity data source relationship pairs from user configuration information, establishing mapping relationships between entities and relationships within the preset knowledge graph ontology and knowledge graph construction data sources according to entity data source relationships, dividing the preset knowledge graph ontology into at least two knowledge graph subgraphs according to the estimated data volume of entities and relationships within the preset knowledge graph ontology, wherein the estimated data volume of entities and relationships within each knowledge graph subgraph is the same, determining the knowledge graph construction data sources corresponding to entities and relationships within each knowledge graph subgraph according to the mapping relationship, constructing graph construction subtasks for knowledge graph construction data sources belonging to the same knowledge graph subgraph, and allocating each graph construction subtask to each computing node. This invention divides a preset knowledge graph ontology into several knowledge graph subgraphs according to the estimated amount of entity and relation data within the preset knowledge graph ontology, and constructs corresponding graph construction subtasks based on the knowledge graph subgraphs. The construction method of the subtasks has good flexibility, and each subtask can be executed independently serially or in parallel, which can improve the construction efficiency of the knowledge graph and reduce the fault tolerance cost of knowledge graph construction.

[0074] Furthermore, based on the above embodiments of the invention, it may also include:

[0075] Control computing nodes to construct subtasks based on the knowledge graph to extract knowledge graph data;

[0076] Store the knowledge graph data as a knowledge graph according to the preset knowledge graph ontology.

[0077] In this embodiment of the invention, knowledge graph data can be understood as data extracted from the knowledge graph data source based on mapping relationships.

[0078] Specifically, each computing node can be controlled to construct subtasks according to the constructed graph and extract corresponding data from the knowledge graph construction data source based on the mapping relationship. The data extraction process can include, but is not limited to, entity extraction, relation extraction, and attribute extraction. Furthermore, pattern matching methods, statistical machine learning methods, and deep learning methods can be used to implement data extraction. The extracted knowledge graph data can be stored in a preset database according to the preset knowledge graph ontology. The preset database can include, but is not limited to, Neo4j database, JanusGraph database, and HugeGraph database.

[0079] Furthermore, based on the above embodiments of the invention, before storing the knowledge graph data as a knowledge graph according to a preset knowledge graph ontology, the method further includes: reducing the knowledge graph data to structured data based on a preset database.

[0080] In this embodiment of the invention, the preset database can be understood as a pre-configured database for processing and storing knowledge graph data. The preset database may include MongoDB database and HarperDB database, etc.

[0081] Specifically, when the data source for knowledge graph construction is unstructured data and semi-structured data, unstructured data can include, but is not limited to: emails, WORD, PPT, EXCEL, and PDF documents, while semi-structured data can include, but is not limited to: log files, XML documents, and JSON documents. A preset database can be used to reduce the data source for knowledge graph construction and store it as structured data. The data reduction process can include, but is not limited to: field creation, collection naming, disabling reserved words, and setting the number of indexes. The preset database can include, but is not limited to: MongoDB database and HarperDB database.

[0082] Furthermore, based on the above embodiments, Embodiment 2 of the present invention provides a method for constructing a knowledge graph. Figure 3 This is an example diagram of a knowledge graph construction method provided in Embodiment 2 of the present invention, wherein the method specifically includes the following steps:

[0083] S310. Select the data source for building the knowledge graph.

[0084] S320. Draw the ontology of the knowledge graph and configure the mapping relationship between the ontology and the data source.

[0085] S330. The ontology is split into subgraphs based on relations, where each subgraph corresponds to a subtask.

[0086] S340. Start running the build task. The system will automatically schedule subtasks, and each subtask can be managed independently.

[0087] S350: The subtask extracts data from the corresponding data source and stores it in the MongoDB database according to the previously configured mapping relationship. Then, it creates a schema in the graph database and finally writes the intermediate data from the MongoDB database into the graph database to generate a subgraph.

[0088] Specifically, once all subtasks have run successfully, the results of all subgraphs will form a complete knowledge graph.

[0089] Most existing knowledge graph construction methods adopt a single-task approach, such as... Figure 4 As shown, its construction idea is to regard the ontology as the smallest unit for knowledge graph construction. This type of method cannot operate on a small scale of subgraphs and has a high fault tolerance cost. The embodiments of this invention adopt the following... Figure 5The batch construction method shown can divide a complete knowledge graph into multiple subgraphs, each of which is constructed separately. Each subgraph includes entity and relation data. After the subgraph is completed, various reasoning tasks can be performed based on it. At the same time, if a subtask fails, it can be rerun separately, which greatly reduces the fault tolerance cost. In addition, the subgraph construction task can be executed in parallel or sequentially depending on the platform's computing resources.

[0090] The knowledge graph construction method provided in this invention can be applied to situations where knowledge graphs are constructed under the premise of limited computing resources. Figure 6 The ontology for the knowledge graph to be constructed. Figure 7 , Figure 8 and Figure 9 They are respectively to Figure 6 The ontology is split into three subgraphs based on relationships. These three subgraphs can be run as three separate subtasks. It's important to note that when splitting a subgraph, if it contains relationships, it's crucial to ensure that the start and end entity data for that relationship has already been written in a previous task or in this subtask. This is because creating each relationship requires ensuring that the start and end entity data are written. To improve splitting efficiency, this embodiment of the invention splits the knowledge graph into subgraphs based on relationships. Furthermore, before partial splitting, the data volume of the final generated knowledge graph needs to be estimated to maintain a balanced data volume across each subgraph.

[0091] Figure 10 This represents an execution order for subgraph tasks within a single task. It's important to note that since each subgraph task is configured based on relationships, scenarios involving missing entity data during edge writing don't need to be considered; that is, the execution order of each subgraph task can be disrupted. Because subgraphs are configured based on relationships, multiple subgraphs may use the same entity. To avoid duplicate entity data extraction, during the sequential execution of subgraph construction tasks in a single task, before writing data to each entity, it's checked whether the entity has already been written. If data already exists, the extraction of that entity's data is skipped. For example, based on the principle of data balance... Figure 9 sub Figure 3 The graph only has one text entity. The data for the three entities—chapter, document, and customer—has already been written to the graph database during the first two subgraph tasks. Therefore, in the subgraph... Figure 3 When the task runs, it will only write text entity data and other relational data from the subgraph to the graph data. Similarly, Figure 8 sub Figure 2 The entity document and label are already inFigure 7 sub Figure 1 The task was written to the graph database during execution, and the sub-database... Figure 2 When the task runs, it will only write entity and relationship data other than these two entities.

[0092] Figure 11 This describes the process of distributed parallel execution of subgraph construction tasks. In some embodiments, this relationship-based subgraph decomposition, if using parallel tasks to build a knowledge graph, requires treating each subgraph as a separate, independent task. Each subgraph construction task extracts entity data independently, without considering duplicate extractions. If a subgraph construction task fails, that subgraph construction task can be run separately to supplement the data, without needing to rerun the entire knowledge graph construction task.

[0093] Example 3

[0094] Figure 12 This is a flowchart of a knowledge graph generation task construction method provided in Embodiment 3 of the present invention. It is further optimized and extended based on the above embodiments and can be combined with various optional technical solutions in the above embodiments. For example... Figure 12 As shown in the figure, the knowledge graph generation task construction method provided in this embodiment includes the following steps:

[0095] S410: Collect local ontology drawing data from the preset knowledge graph.

[0096] S420. Select at least one type of knowledge graph to construct a data source based on the user's selection information.

[0097] S430. Extract entity data source relationship pairs from user configuration information.

[0098] S440. Based on the entity data source relationship, establish mapping relationships between the entities and relationships within the preset knowledge graph body and the knowledge graph construction data source respectively.

[0099] S450: Extract entities and relationships from the preset knowledge graph ontology.

[0100] Specifically, several entities and relationships can be extracted from a pre-defined knowledge graph ontology. The extraction methods may include, but are not limited to, image recognition methods based on Optical Character Recognition (OCR), image recognition methods based on traditional machine learning, and widely used image recognition methods based on deep learning neural networks. The specific extraction methods are not limited in the implementation of this invention.

[0101] S460. Divide the entity into at least one entity subclass according to the estimated data volume of the corresponding entity.

[0102] In this embodiment of the invention, an entity subclass can be understood as a group of entities obtained by dividing the entities in the ontology. An entity subclass may contain one or more entities.

[0103] Specifically, entities in the ontology can be divided into several entity subclasses according to the estimated data volume of each entity, where an entity subclass can contain one or more entities.

[0104] S470. Construct a data source by determining the knowledge graph corresponding to the entity within the entity subclass according to the mapping relationship.

[0105] Specifically, entities within an entity subclass can be mapped to corresponding knowledge graphs to construct data sources.

[0106] S480. Construct an entity extraction subtask for knowledge graphs belonging to the same entity subclass by building a data source.

[0107] Specifically, several entity extraction subtasks can be constructed based on the knowledge graphs belonging to the same entity subclasses to build data sources, where each entity extraction subtask corresponds to an entity subclass.

[0108] S490. Divide the relationship into at least one subclass according to the estimated amount of data in the corresponding relationship.

[0109] In this embodiment of the invention, a relation subclass can be understood as a group of relations obtained by dividing the relations in the ontology. A relation subclass can contain one or more relations.

[0110] Specifically, the relations in the ontology can be divided into several relation subclasses according to the estimated data volume of each relation, where a relation subclass can contain one or more relations.

[0111] S4100. Construct a data source by determining the knowledge graph corresponding to the relations within the relation subclasses according to the mapping relationship.

[0112] Specifically, the relationships within a relation subclass can be mapped to determine the corresponding knowledge graph and construct the data source.

[0113] S4110. Construct a relationship extraction subtask for knowledge graphs belonging to the same relation subclasses by building a data source.

[0114] Specifically, several relation extraction subtasks can be constructed based on the knowledge graphs belonging to the same relation subclasses to build a data source, where each relation extraction subtask corresponds to a relation subclass.

[0115] S4120. Assign the entity extraction subtask and relation extraction subtask as graph construction subtasks to the computing nodes.

[0116] Specifically, the constructed entity extraction subtasks and relation extraction subtasks can be used as graph construction subtasks, and these subtasks can be assigned to various computing nodes for execution. Furthermore, the execution methods of the computing nodes can include, but are not limited to: sequential execution of the graph construction subtasks in a predetermined order, and parallel execution of the graph construction subtasks. In some embodiments, the constructed entity extraction subtasks can first be executed in parallel on a distributed platform, and then the constructed relation extraction subtasks can be executed in parallel based on the already obtained entity data.

[0117] The technical solution of this invention involves collecting local ontology rendering data of a preset knowledge graph, selecting at least one type of knowledge graph construction data source according to user selection information, extracting entity data source relationship pairs from user configuration information, establishing mapping relationships between entities and relationships within the preset knowledge graph ontology and knowledge graph construction data sources according to entity data source relationships, extracting entities and relationships from the preset knowledge graph ontology, dividing entities into at least one entity subclass according to the estimated data volume of the corresponding entities, determining the knowledge graph construction data source corresponding to entities within the entity subclass according to the mapping relationship, constructing entity extraction subtasks for knowledge graph construction data sources belonging to the same entity subclass, dividing relationships into at least one relationship subclass according to the estimated data volume of the corresponding relationships, determining the knowledge graph construction data source corresponding to relationships within the relationship subclass according to the mapping relationship, constructing relationship extraction subtasks for knowledge graph construction data sources belonging to the same relationship subclass, and allocating each entity extraction subtask and relationship extraction subtask as graph construction subtasks to computing nodes. This invention classifies entities into at least one entity subclass based on the estimated amount of entity data, and divides relations into at least one relation subclass based on the estimated amount of relation data. Based on the mapping relationship and the knowledge graph, the data source is constructed to determine the entity subclass and relation subclass as the corresponding entity extraction subtask and relation extraction subtask. Each subtask can be executed serially or in parallel according to computing resources, which has high flexibility in the graph construction method. At the same time, it can support rerunning and recovering data after the subtask fails, which greatly reduces the fault tolerance cost of knowledge graph construction and has high graph construction efficiency.

[0118] Furthermore, based on the above embodiments, Embodiment 3 of the present invention provides a method for constructing a knowledge graph. Figure 13 This is an example diagram of a knowledge graph construction method provided in Embodiment 3 of the present invention, wherein the method specifically includes the following steps:

[0119] S510. Select the data source for building the knowledge graph.

[0120] S520. Draw the ontology of the knowledge graph and configure the mapping relationship between the ontology and the data source.

[0121] S530. Based on entities and relationships, the ontology is split into subgraphs. First, an entity data subgraph construction task is constructed, and then a relationship data subgraph construction task is constructed.

[0122] S540. Start running the build task. The system will automatically schedule subtasks, and each subtask can be managed independently.

[0123] S550: The subtask extracts data from the corresponding data source and stores it in the MongoDB database according to the previously configured mapping relationship. Then, it creates a schema in the graph database and finally writes the intermediate data from the MongoDB database into the graph database to generate a subgraph.

[0124] The knowledge graph construction method provided in this invention is applicable to scenarios where sufficient computing resources are available for knowledge graph construction. This invention writes entity and relation data into the knowledge graph in two batches, with tasks in each batch executed in parallel. Furthermore, it avoids the issue of repeatedly extracting entity data during parallel execution as described in the previous embodiments. This invention is suitable for rapidly constructing ultra-large-scale knowledge graphs using the computing power of big data distributed computing platforms.

[0125] Figure 14 What is being shown is Figure 6 The entities in the ontology are the target entity data for the first batch of construction. Based on the platform's computing power, the nine entities can be divided into several subgraph construction tasks, and multiple subgraph construction tasks can run in parallel on a distributed platform, such as... Figure 11 It can also be executed sequentially in a single task, such as Figure 10 .

[0126] Figure 6 The relational data in the first batch is the target data for the second batch of construction. Similar to the entity data extraction in the first batch, the 17 relational data entries can be divided into several parallel tasks based on the platform's computing power. Multiple subgraph construction tasks can run in parallel on a distributed platform, such as... Figure 11 Of course, tasks can also be executed sequentially, such as... Figure 10 The subgraph decomposition results of the relationships can be referenced. Figure 7 , Figure 8 , Figure 9 .

[0127] This invention also supports running the subgraph construction task separately to supplement data in the event of a subgraph construction task failure, without having to rerun the entire knowledge graph construction task. However, it should be noted that since the relationship data extraction is empty due to entity construction failure, the corresponding entity subgraph task in the first batch must be executed successfully before the relationship extraction subgraph construction task in the second batch can be run.

[0128] Example 4

[0129] Figure 15 This is a schematic diagram of a knowledge graph generation task construction device provided in Embodiment 4 of the present invention. Figure 15 As shown, the device includes:

[0130] The data acquisition module 41 is used to acquire the preset knowledge graph ontology and the knowledge graph construction data source.

[0131] The mapping determination module 42 is used to determine the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source based on the user configuration information.

[0132] Task construction module 43 is used to determine at least one graph construction subtask to be executed by a computing node based on a preset knowledge graph ontology and mapping relationship.

[0133] The technical solution of this invention involves a data acquisition module acquiring a preset knowledge graph ontology and a knowledge graph construction data source, a mapping determination module determining the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source based on user configuration information, and a task construction module determining a graph construction subtask to be executed by at least one computing node based on the preset knowledge graph ontology and the mapping relationship. This invention breaks down the knowledge graph construction task into multiple graph construction subtasks, which are then executed by at least one computing node. The subtask splitting process is simple and flexible, and each subtask can be executed independently, serially or in parallel, improving the efficiency of knowledge graph construction. Furthermore, the knowledge graph construction process has a high fault tolerance rate and is more practical.

[0134] Furthermore, based on the above embodiments of the invention, the data acquisition module 41 includes:

[0135] The ontology data acquisition unit is used to collect ontology rendering data locally from the preset knowledge graph.

[0136] The data source selection unit is used to select at least one type of knowledge graph to construct a data source based on the user's selection information.

[0137] Furthermore, based on the above embodiments of the invention, the mapping determination module 42 includes:

[0138] The relationship pair extraction unit is used to extract entity data source relationship pairs from user configuration information.

[0139] The mapping relationship establishment unit is used to establish mapping relationships between entities and relationships within the preset knowledge graph body and the knowledge graph construction data source, according to the entity data source relationship.

[0140] Furthermore, based on the above embodiments of the invention, the task construction module 43 includes:

[0141] An ontology partitioning unit is used to divide a preset knowledge graph ontology into at least two knowledge graph subgraphs according to the estimated data of entities and relations within the preset knowledge graph ontology, wherein the estimated data of entities and relations within each knowledge graph subgraph is the same.

[0142] The first data source determination unit is used to determine the knowledge graph construction data source corresponding to the entities and relationships in the knowledge graph subgraphs according to the mapping relationship.

[0143] The subtask construction unit is used to construct data sources and graph construction subtasks for knowledge graphs belonging to the same knowledge graph subgraph.

[0144] The first subtask allocation unit is used to allocate each graph construction subtask to each computing node.

[0145] Furthermore, based on the above embodiments of the invention, the task construction module 43 may also include:

[0146] The entity and relation extraction unit is used to extract entities and relations from a predefined knowledge graph ontology.

[0147] The entity subclass determination unit is used to divide an entity into at least one entity subclass according to the estimated data amount of the corresponding entity.

[0148] The second data source determination unit is used to determine the knowledge graph corresponding to the entity within the entity subclass according to the mapping relationship to construct the data source.

[0149] The entity subtask construction unit is used to construct entity extraction subtasks for knowledge graphs that belong to the same entity subclasses, based on the data source they are building.

[0150] A relation subclass determination unit is used to divide the relation into at least one relation subclass according to the estimated amount of data corresponding to the relation.

[0151] The third data source determination unit is used to determine the knowledge graph corresponding to the relationship within the subclass of the relationship according to the mapping relationship to construct the data source.

[0152] The relation subtask construction unit is used to construct relation extraction subtasks for knowledge graphs that belong to the same relation subclass, based on the data source they build.

[0153] The second subtask allocation unit is used to allocate the entity extraction subtasks and relation extraction subtasks as graph construction subtasks to the computing nodes.

[0154] Furthermore, based on the above embodiments of the invention, it may also include:

[0155] The data extraction module controls the computing nodes to extract knowledge graph data by constructing subtasks according to the graph.

[0156] The data storage module is used to store knowledge graph data as a knowledge graph according to a preset knowledge graph ontology.

[0157] Furthermore, based on the above embodiments of the invention, before the data storage module, it may also include:

[0158] The data reduction module is used to reduce knowledge graph data into structured data based on a preset database.

[0159] The knowledge graph generation task construction apparatus provided in the embodiments of the present invention can execute the knowledge graph generation task construction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0160] Example 5

[0161] Figure 16 A schematic diagram of an electronic device 50 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0162] like Figure 16 As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52 and a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The memory stores computer programs executable by the at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the ROM 52 or loaded into the RAM 53 from storage unit 58. The RAM 53 may also store various programs and data required for the operation of the electronic device 50. The processor 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0163] Multiple components in electronic device 50 are connected to I / O interface 55, including: input unit 56, such as keyboard, mouse, etc.; output unit 57, such as various types of monitors, speakers, etc.; storage unit 58, such as disk, optical disk, etc.; and communication unit 59, such as network card, modem, wireless transceiver, etc. Communication unit 59 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0164] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as knowledge graph generation task construction methods.

[0165] In some embodiments, the knowledge graph generation task construction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 50 via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the knowledge graph generation task construction method described above may be performed. Alternatively, in other embodiments, processor 51 may be configured to execute the knowledge graph generation task construction method by any other suitable means (e.g., by means of firmware).

[0166] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0169] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0170] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0171] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0172] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0173] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for constructing a knowledge graph generation task, characterized in that, The method includes: Obtain the preset knowledge graph ontology and the data source for knowledge graph construction; The mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source is determined based on the user configuration information; Based on the preset knowledge graph ontology and the mapping relationship, at least one graph construction subtask to be executed by a computing node is determined; The step of determining the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source based on user configuration information includes: Extract the entity data source relationship pairs from the user configuration information; According to the entity data source relationship, establish the mapping relationship between the entities and relationships within the preset knowledge graph and the knowledge graph construction data source respectively; The entity data source relationship pairs include: entity data—mapping relationship—data source, and relationship data—mapping relationship—data source. The mapping relationship includes: database address mapping and field mapping. The graph construction subtask, which determines at least one computing node to be executed based on the preset knowledge graph ontology and the mapping relationship, includes: The preset knowledge graph ontology is divided into at least two knowledge graph subgraphs according to the estimated data of entities and relations within the preset knowledge graph ontology, wherein the estimated data of entities and relations are the same in each of the knowledge graph subgraphs. According to the mapping relationship, determine the knowledge graph construction data source corresponding to the entity and the relationship in the knowledge graph subgraph; For knowledge graph construction data sources belonging to the same knowledge graph subgraph, construct the graph construction subtask; Each of the aforementioned graph construction subtasks is assigned to each of the aforementioned computing nodes.

2. The method according to claim 1, characterized in that, The process of obtaining the preset knowledge graph ontology and the knowledge graph construction data source includes: Collect local ontology rendering data from the preset knowledge graph; Based on the user's selection information, at least one type of knowledge graph is selected to construct a data source.

3. The method according to claim 1, characterized in that, The graph construction subtask, which determines at least one computing node to execute based on the preset knowledge graph ontology and the mapping relationship, includes: Extract the entities and relationships from the preset knowledge graph ontology; The entity is divided into at least one entity subclass according to the estimated data volume corresponding to the entity; The knowledge graph construction data source corresponding to the entity within the entity subclass is determined according to the mapping relationship; For the knowledge graphs belonging to the same subclass of entities, construct data sources and construct entity extraction subtasks; The relationship is divided into at least one subclass based on the estimated amount of data corresponding to the relationship. According to the mapping relationship, the knowledge graph corresponding to the relationship within the relationship subclass is used to construct the data source; For knowledge graphs belonging to the same subclass of relation, construct data sources and construct relation extraction subtasks; The entity extraction subtasks and the relation extraction subtasks are assigned to the computing nodes as the graph construction subtasks.

4. The method according to claim 1, characterized in that, Also includes: Control the computing nodes to construct subtasks to extract knowledge graph data according to the graph; The knowledge graph data is stored as a knowledge graph according to the preset knowledge graph ontology.

5. The method according to claim 4, characterized in that, Before storing the knowledge graph data as a knowledge graph according to the preset knowledge graph ontology, the method further includes: reducing the knowledge graph data to structured data based on a preset database.

6. A knowledge graph generation task construction device, characterized in that, The device includes: The data acquisition module is used to acquire the preset knowledge graph ontology and the data source for knowledge graph construction; The mapping determination module is used to determine the mapping relationship between the preset knowledge graph ontology and the knowledge graph construction data source based on user configuration information; The task construction module is used to determine at least one graph construction subtask to be executed by a computing node based on the preset knowledge graph ontology and the mapping relationship; The mapping determination module includes: The relationship pair extraction unit is used to extract entity data source relationship pairs from user configuration information; The mapping relationship establishment unit is used to establish mapping relationships between entities and relationships within the preset knowledge graph body and the knowledge graph construction data source according to the entity data source relationship; The entity data source relationship pairs include: entity data—mapping relationship—data source, and relationship data—mapping relationship—data source. The mapping relationship includes: database address mapping and field mapping. The task construction module includes: The ontology partitioning unit is used to divide the preset knowledge graph ontology into at least two knowledge graph subgraphs according to the estimated data of entities and relations within the preset knowledge graph ontology, wherein the estimated data of entities and relations within each knowledge graph subgraph is the same. The first data source determination unit is used to determine the knowledge graph construction data source corresponding to the entities and relationships in the knowledge graph subgraphs according to the mapping relationship. Subtask construction unit, used to construct data source and graph construction subtasks for knowledge graphs belonging to the same knowledge graph subgraph; The first subtask allocation unit is used to allocate each graph construction subtask to each computing node.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the knowledge graph generation task construction method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the knowledge graph generation task construction method according to any one of claims 1-5.

Images