Data construction task planning method based on knowledge graph

Abstract

The invention relates to the technical field of data construction task planning, and discloses a data construction task planning method based on a knowledge graph, and the method comprises five steps of knowledge graph construction, task decomposition, task planning, task execution and task monitoring. The method comprises the following steps: firstly, establishing a knowledge graph containing entities, attributes and relationships by collecting, arranging and analyzing data in related fields; then, according to information in the knowledge graph, the data construction task is decomposed into a plurality of subtasks, and a task planning scheme is generated through a task planning algorithm; next, scheduling and executing each subtask according to the task planning scheme, and monitoring the execution state of the task in real time; according to the method, the knowledge graph technology and the task planning algorithm are combined, automatic planning and optimization of the data construction task are achieved, the data construction efficiency and accuracy are improved, the labor cost and the time cost are reduced, and powerful support is provided for data management and utilization in all fields.

Description

Technical Field

[0001] The present invention relates to the technical field of data construction task planning, and specifically to a data construction task planning method based on a knowledge graph. Background Art

[0002] With the rapid development of information technology, data has become a vital resource across various fields. Efficiently managing and utilizing this data has become a hot topic in current research and application. Knowledge graphs, as an important data management technology, clearly illustrate relationships between data by representing them graphically, providing powerful support for data analysis and decision-making.

[0003] However, building a knowledge graph presents numerous challenges. First, data sources are diverse and complex, and effectively integrating and cleaning this data is a pressing issue. Second, building a knowledge graph requires extensive domain knowledge and expert experience. Integrating this knowledge into the graph while ensuring its accuracy and completeness is also a challenge. Furthermore, as data constantly updates and changes, how to update and maintain the knowledge graph in real time to ensure its timeliness and reliability is also a consideration.

[0004] To address these issues, existing data construction task planning methods typically rely on manual approaches, where domain experts develop data construction tasks and planning plans based on their experience and needs. However, this approach suffers from shortcomings such as low efficiency, strong subjectivity, and difficulty adapting to data changes. Therefore, leveraging computer technology to achieve automated planning and optimization of data construction tasks has become a hot research topic.

[0005] The purpose of this invention is to provide a data construction task planning method based on a knowledge graph to address the problems existing in the prior art. By constructing a knowledge graph, performing task decomposition, task planning, task execution, and task monitoring, this method achieves automated planning and optimization of data construction tasks, improving data construction efficiency and accuracy, and providing strong support for data management and utilization in various fields. Summary of the Invention

[0006] (1) Technical problems solved

[0007] In response to the shortcomings of the existing technology, the present invention provides a data construction task planning method based on knowledge graph, which solves the problems raised in the above background technology.

[0008] (2) Technical solution

[0009] To achieve the above objectives, the present invention provides the following technical solutions: a data construction task planning method based on knowledge graph, the method comprising the following steps:

[0010] Step 1: Use the knowledge graph construction module and knowledge graph analysis module to build and analyze the knowledge graph, and evaluate the priority of each subtask based on the entity importance, relationship strength, and business needs in the domain knowledge graph;

[0011] Step 2: Decompose and model tasks through the task decomposition and modeling module, and integrate priority information into the task dependency graph to implement modeling;

[0012] Step 3: Perform task planning and scheduling through the task planning and scheduling module to form a data construction task planning plan based on the knowledge graph;

[0013] Step 4: Use the task monitoring and adjustment module to perform task execution and monitoring, monitor the execution status and progress of subtasks in real time, and use the knowledge graph to dynamically adjust and optimize according to actual conditions;

[0014] Step 5: Through the user interaction module, the task execution results are displayed in real time in the form of graphics and reports, allowing users to configure the parameters of the data construction task and display the task execution results in the form of graphics and reports.

[0015] The specific method of constructing and analyzing the knowledge graph is as follows:

[0016] Step 100: Constructing a domain knowledge graph: For a specific domain, collect multi-source heterogeneous data, including structured data, semi-structured data, and unstructured data. Through data extraction, entity recognition, relationship extraction, attribute recognition, and knowledge fusion technologies, construct a domain knowledge graph covering the core concepts, entities, attributes, and relationships in the domain, and store it in a graph database. The entity recognition algorithm is as follows:

[0017] P(y|x)=\frac{1}{Z(x)}\exp(\sum_{i,k}\lambda_kf_k(y_{i-1},y_i,x,i))

[0018] Where P(y|x) is the probability of state sequence y given the observation sequence x, Z(x) is the normalization factor, and i,k are the weights of the feature function f_k;

[0019] The relation extraction algorithm is as follows:

[0020] Score(y)=\sum_{k}\lambda_kf_k(y,x)

[0021] Where Score(y) is the score of the parsing result y, k is the feature function, and f_k is the corresponding weight;

[0022] Step 101: Knowledge graph analysis: Analyze the domain knowledge graph constructed in step 100 to identify key entities, core relationships, and key paths and subgraphs related to data construction tasks.

[0023] The steps of task decomposition and modeling are as follows:

[0024] Step 102: Identify task dependencies: Based on the key entities, core relationships, and critical paths identified in step 101, analyze the dependencies between subtasks in the data construction task and construct a task dependency graph.

[0025] Step 103, task priority evaluation: Evaluate the priority of each subtask based on the entity importance, relationship strength and business needs in the domain knowledge graph, and integrate the priority information into the task dependency graph.

[0026] The task planning and scheduling in step 3 are as follows:

[0027] Step 104: Task sorting and scheduling: Based on the task dependency graph constructed in step 102 and the subtask priorities determined in step 103, a topological sorting algorithm is used to sort the subtasks, generate a task execution sequence that satisfies the dependency and priority constraints, and formulate a task scheduling plan that takes resource constraints into consideration.

[0028] Step 105: Generate a task planning solution: Integrate the task execution sequence and the scheduling solution generated in step 104 to form a data-based task planning solution based on the knowledge graph, which is presented in a visual manner.

[0029] The task execution and monitoring in step 4 are as follows:

[0030] Step 106, task execution: executing each subtask according to the task planning scheme generated in step 105;

[0031] Step 107: Task monitoring and adjustment: During the task execution process, the execution status and progress of subtasks are monitored in real time, and dynamic adjustments and optimizations are made based on the actual situation using the knowledge graph.

[0032] Task planning in step three includes: analyzing the dependencies between subtasks and determining the execution order of subtasks; developing a reasonable task planning scheme based on the priorities and resource requirements of subtasks; and presenting the task planning scheme in a visual way to facilitate executors' understanding and execution.

[0033] The task execution in step 4 includes: executing each subtask in sequence according to the task planning scheme; during the execution process, monitoring the execution status of the subtasks in real time and making adjustments based on actual conditions; after completing all subtasks, evaluating and summarizing the entire data construction task.

[0034] Preferably, the knowledge graph construction module includes a data acquisition submodule, a data preprocessing submodule, an entity recognition submodule, a relationship extraction submodule, an attribute recognition submodule, a knowledge fusion submodule and a graph database storage submodule;

[0035] The data acquisition submodule is responsible for collecting data from databases, files, API interfaces and web pages;

[0036] The data preprocessing submodule is used to perform cleaning, deduplication and formatting preprocessing operations on the collected raw data;

[0037] The entity recognition submodule is used to recognize names of people, places and institutions in the text;

[0038] The relationship extraction submodule is used to identify the person relationships and event relationships between entities;

[0039] The attribute recognition submodule is used to recognize the age and gender attributes of a person;

[0040] The knowledge fusion submodule is used to fuse the extracted entities, relationships and attributes to solve entity disambiguation and knowledge conflict problems;

[0041] The graph database storage submodule is used to store the constructed knowledge graph in the graph database.

[0042] Preferably, the knowledge graph analysis module includes a key entity recognition submodule, a core relationship recognition submodule, a subgraph recognition submodule and a knowledge graph visualization submodule;

[0043] The key entity identification submodule identifies key entities in the knowledge graph based on a graph algorithm;

[0044] The core relationship identification submodule is used to identify the core relationship connecting key entities;

[0045] The graph identification submodule is used to identify key paths and subgraphs related to data construction tasks;

[0046] The knowledge graph visualization submodule is used to display the knowledge graph and analysis results in a graphical manner to facilitate user understanding.

[0047] Preferably, the task decomposition and modeling module includes a task decomposition submodule, a task dependency relationship construction submodule, a task priority evaluation submodule and a task dependency relationship graph construction submodule;

[0048] The task decomposition submodule decomposes the data construction task into executable subtasks based on the knowledge graph analysis results;

[0049] The task dependency building submodule builds a dependency graph between subtasks based on the entity relationships and business processes in the knowledge graph;

[0050] The task priority evaluation submodule evaluates the priority of subtasks based on the entity importance, relationship strength and business needs in the knowledge graph;

[0051] The task dependency graph construction submodule is used to integrate task dependency and priority information into the task dependency graph.

[0052] Preferably, the task planning and scheduling module includes a task sequencing submodule, a resource allocation submodule, a task scheduling submodule and a task execution engine submodule;

[0053] The task sorting submodule uses a graph theory algorithm to sort the task dependency graph and generate a feasible task execution sequence;

[0054] The resource allocation submodule allocates computing resources and storage resources to each subtask based on task requirements and resource availability;

[0055] The task scheduling submodule generates a task scheduling plan based on the task execution sequence and resource allocation;

[0056] The task execution engine submodule is responsible for executing task scheduling plans, starting and monitoring the execution of subtasks.

[0057] Preferably, the task monitoring and adjustment module includes a task status monitoring submodule, a knowledge graph updating submodule, a task planning optimization submodule and an exception handling submodule;

[0058] The task status monitoring submodule is used to monitor the execution status, progress and resource usage of subtasks in real time;

[0059] The knowledge graph updating submodule updates and improves the knowledge graph based on the new data generated during the task execution;

[0060] The task planning optimization submodule dynamically adjusts and optimizes the task planning scheme based on the updated results of the knowledge graph and task execution feedback;

[0061] The exception handling submodule is used to identify and handle exceptions during task execution.

[0062] Preferably, the user interaction module includes a task configuration interface submodule, a knowledge graph browsing interface submodule, a task monitoring interface submodule and a result display interface submodule;

[0063] The task configuration interface submodule is used to allow users to configure the parameters of the data construction task;

[0064] The knowledge graph browsing interface submodule is used to allow users to browse and query the knowledge graph;

[0065] The task monitoring interface submodule is used to allow users to view the task execution status and progress;

[0066] The result display interface submodule displays the task execution results in graphical and report forms.

[0067] (3) Beneficial effects

[0068] Compared with the existing technology, this invention provides a data construction task planning method based on knowledge graph, which has the following beneficial effects:

[0069] 1. The present invention combines knowledge graph technology with task planning algorithms to achieve automated decomposition and planning of data construction tasks, reduce manual intervention, improve the efficiency and accuracy of task planning, and can automatically generate task planning plans based on the information in the knowledge graph and preset rules, reducing the influence of subjective factors. At the same time, by constructing a knowledge graph, the relationship between data is clearly expressed in a graphical manner, making task decomposition and planning more efficient. By utilizing the graph database storage and management method of the knowledge graph, data can be quickly queried and updated, which accelerates the task planning process and shortens the data construction cycle.

[0070] 2. The present invention utilizes the rich domain knowledge and entity relationship information in the knowledge graph to more accurately identify the key links and dependencies in the data construction task, avoiding omissions and errors that may occur in traditional methods, and improving the quality of data construction. At the same time, it adopts a modular design approach to decompose the data construction task into multiple subtasks, making the system have good scalability and can easily add new data sources, new processing modules and new business rules to adapt to the ever-changing data environment and business needs.

[0071] 3. Through the task monitoring module, the present invention can monitor and adjust the execution of data construction tasks in real time, promptly discover and solve problems in the execution process, improve the maintainability of the system, ensure the smooth progress of data construction, and through automated and intelligent task planning, further reduce the labor cost and time cost of data construction, improve resource utilization, and thus bring significant economic benefits to enterprises and institutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a schematic diagram of the overall system architecture of the present invention. DETAILED DESCRIPTION

[0073] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0074] A data construction task planning method based on knowledge graph, the method comprises the following steps:

[0075] Step 1: Use the knowledge graph construction module and knowledge graph analysis module to build and analyze the knowledge graph, and evaluate the priority of each subtask based on the entity importance, relationship strength, and business needs in the domain knowledge graph;

[0076] Step 2: Decompose and model tasks through the task decomposition and modeling module, and integrate priority information into the task dependency graph to implement modeling;

[0077] Step 3: Perform task planning and scheduling through the task planning and scheduling module to form a data construction task planning plan based on the knowledge graph;

[0078] Step 4: Use the task monitoring and adjustment module to perform task execution and monitoring, monitor the execution status and progress of subtasks in real time, and use the knowledge graph to dynamically adjust and optimize according to actual conditions;

[0079] Step 5: Through the user interaction module, the task execution results are displayed in real time in the form of graphics and reports, allowing users to configure the parameters of the data construction task and display the task execution results in the form of graphics and reports;

[0080] The specific methods for knowledge graph construction and analysis are as follows:

[0081] Step 100: Constructing a domain knowledge graph: For a specific domain, collect multi-source heterogeneous data, including structured data, semi-structured data, and unstructured data. Through data extraction, entity recognition, relationship extraction, attribute recognition, and knowledge fusion technologies, construct a domain knowledge graph covering the core concepts, entities, attributes, and relationships in the domain, and store it in a graph database. The entity recognition algorithm is as follows:

[0082] P(y|x)=\frac{1}{Z(x)}\exp(\sum_{i,k}\lambda_kf_k(y_{i-1},y_i,x,i))

[0083] Where P(y|x) is the probability of state sequence y given the observation sequence x, Z(x) is the normalization factor, and i,k are the weights of the feature function f_k;

[0084] The relation extraction algorithm is as follows:

[0085] Score(y)=\sum_{k}\lambda_kf_k(y,x)

[0086] Where Score(y) is the score of the parsing result y, k is the feature function, and f_k is the corresponding weight;

[0087] Step 101, knowledge graph analysis: Analyze the domain knowledge graph constructed in step 100 to identify key entities, core relationships, and key paths and subgraphs related to data construction tasks.

[0088] The steps of task decomposition and modeling are as follows:

[0089] Step 102: Identify task dependencies: Based on the key entities, core relationships, and critical paths identified in step 101, analyze the dependencies between subtasks in the data construction task and construct a task dependency graph.

[0090] Step 103, task priority evaluation: Evaluate the priority of each subtask based on the entity importance, relationship strength and business needs in the domain knowledge graph, and integrate the priority information into the task dependency graph.

[0091] The task planning and scheduling in step 3 are as follows:

[0092] Step 104: Task sorting and scheduling: Based on the task dependency graph constructed in step 102 and the subtask priorities determined in step 103, a topological sorting algorithm is used to sort the subtasks, generate a task execution sequence that satisfies the dependency and priority constraints, and formulate a task scheduling plan that takes resource constraints into consideration.

[0093] Step 105: Generate a task planning solution: Integrate the task execution sequence and the scheduling solution generated in step 104 to form a data-based task planning solution based on the knowledge graph, which is presented in a visual manner.

[0094] The task execution and monitoring in step 4 are as follows:

[0095] Step 106, task execution: executing each subtask according to the task planning scheme generated in step 105;

[0096] Step 107: Task monitoring and adjustment: During the task execution process, the execution status and progress of subtasks are monitored in real time, and dynamic adjustments and optimizations are made based on the actual situation using the knowledge graph.

[0097] Task planning in step three includes: analyzing the dependencies between subtasks and determining the execution order of subtasks; developing a reasonable task planning scheme based on the priorities and resource requirements of subtasks; and presenting the task planning scheme in a visual way to facilitate executors' understanding and execution.

[0098] The task execution in step 4 includes: executing each subtask in sequence according to the task planning scheme; during the execution process, monitoring the execution status of the subtasks in real time and making adjustments based on actual conditions; after completing all subtasks, evaluating and summarizing the entire data construction task.

[0099] The knowledge graph construction module includes data acquisition submodule, data preprocessing submodule, entity recognition submodule, relationship extraction submodule, attribute recognition submodule, knowledge fusion submodule and graph database storage submodule;

[0100] The data collection submodule is responsible for collecting data from databases, files, API interfaces and web pages;

[0101] The data preprocessing submodule is used to perform cleaning, deduplication and formatting preprocessing operations on the collected raw data;

[0102] The entity recognition submodule is used to identify names of people, places, and organizations in text;

[0103] The relationship extraction submodule is used to identify the person and event relationships between entities;

[0104] The attribute recognition submodule is used to identify the age and gender attributes of a person;

[0105] The knowledge fusion submodule is used to fuse the extracted entities, relationships and attributes to solve entity disambiguation and knowledge conflict problems;

[0106] The graph database storage submodule is used to store the constructed knowledge graph in the graph database;

[0107] The knowledge graph analysis module includes a key entity recognition submodule, a core relationship recognition submodule, a subgraph recognition submodule, and a knowledge graph visualization submodule;

[0108] The key entity recognition submodule identifies key entities in the knowledge graph based on graph algorithms;

[0109] The core relationship identification submodule is used to identify the core relationships connecting key entities;

[0110] The graph recognition submodule is used to identify the key paths and subgraphs related to the data construction task;

[0111] The knowledge graph visualization submodule is used to display the knowledge graph and analysis results in a graphical way to facilitate user understanding;

[0112] The task decomposition and modeling module includes a task decomposition submodule, a task dependency relationship construction submodule, a task priority evaluation submodule, and a task dependency relationship graph construction submodule;

[0113] The task decomposition submodule decomposes the data construction task into executable subtasks based on the knowledge graph analysis results;

[0114] The task dependency building submodule builds a dependency graph between subtasks based on the entity relationships and business processes in the knowledge graph;

[0115] The task priority assessment submodule evaluates the priority of subtasks based on the entity importance, relationship strength, and business needs in the knowledge graph;

[0116] The task dependency graph construction submodule is used to integrate task dependency and priority information into the task dependency graph;

[0117] The task planning and scheduling module includes a task sorting submodule, a resource allocation submodule, a task scheduling submodule, and a task execution engine submodule;

[0118] The task sorting submodule uses graph theory algorithms to sort the task dependency graph and generate a feasible task execution sequence;

[0119] The resource allocation submodule allocates computing resources and storage resources to each subtask based on task requirements and resource availability;

[0120] The task scheduling submodule generates a task scheduling plan based on the task execution sequence and resource allocation;

[0121] The task execution engine submodule is responsible for executing task scheduling plans, starting and monitoring the execution of subtasks;

[0122] The task monitoring and adjustment module includes a task status monitoring submodule, a knowledge graph update submodule, a task planning optimization submodule, and an exception handling submodule;

[0123] The task status monitoring submodule is used to monitor the execution status, progress and resource usage of subtasks in real time;

[0124] The knowledge graph update submodule updates and improves the knowledge graph based on the new data generated during task execution;

[0125] The task planning optimization submodule dynamically adjusts and optimizes the task planning scheme based on the updated results of the knowledge graph and task execution feedback;

[0126] The exception handling submodule is used to identify and handle exceptions during task execution;

[0127] The user interaction module includes a task configuration interface submodule, a knowledge graph browsing interface submodule, a task monitoring interface submodule, and a result display interface submodule;

[0128] The task configuration interface submodule is used to allow users to configure the parameters of the data construction task;

[0129] The knowledge graph browsing interface submodule is used to allow users to browse and query the knowledge graph;

[0130] The task monitoring interface submodule is used to allow users to view the task execution status and progress;

[0131] The result display interface submodule displays the task execution results in graphical and report forms.

[0132] In order to better understand the present invention, it will be described in detail below with reference to specific embodiments.

[0133] Example 1:

[0134] In the financial field, build a knowledge graph data construction task planning system for risk control.

[0135] Step 1: Build a knowledge graph:

[0136] Collect relevant data from the financial sector, including but not limited to corporate information, personal credit records, transaction data, and market data. Preprocess this data, including data cleaning and conversion. Then, using natural language processing techniques and machine learning algorithms, extract entities, attributes, and relationships from the data to construct a knowledge graph for the financial sector. For example, entities might include companies, individuals, and products; attributes might include a company's registered capital, an individual's age, and a product's price; and relationships might include investment relationships between companies and transaction relationships between individuals.

[0137] Step 2: Task decomposition:

[0138] Based on the constructed financial knowledge graph, the data construction task can be broken down into multiple subtasks. For example, the task can be broken down into subtasks such as data collection, data cleaning, data conversion, data storage, and data analysis. Each subtask corresponds to one or more entities, attributes, and relationships in the knowledge graph.

[0139] Step 3: Task planning:

[0140] A task planning algorithm is used to generate a task plan based on the dependencies and priorities between subtasks. For example, data collection tasks must be executed before data cleaning tasks, and data analysis tasks must be executed only after data storage tasks are completed. The task plan can be represented using a directed acyclic graph (DAG), where nodes represent subtasks and edges represent the dependencies between subtasks.

[0141] Step 4: Task execution:

[0142] Schedule the execution of each subtask according to the task plan. Distributed computing frameworks such as Hadoop and Spark can be used to improve the parallelism and efficiency of task execution. During task execution, the execution status of the task needs to be monitored in real time to promptly identify and resolve any problems during execution.

[0143] Step 5: Task monitoring:

[0144] The task monitoring module provides real-time monitoring of the execution of data construction tasks. This includes subtask execution status, execution time, resource usage, and more. If a subtask fails or takes too long to execute, adjustments can be made promptly, such as re-executing the subtask or adjusting the task plan.

[0145] Example 2:

[0146] In the medical field, build a knowledge graph data construction task planning system for disease diagnosis.

[0147] Step 1: Build a knowledge graph:

[0148] Collect relevant medical data, including but not limited to patient information, medical records, medical literature, and drug information. Preprocess this data, including data cleaning and conversion. Then, using medical ontologies and knowledge bases, combined with natural language processing techniques and machine learning algorithms, extract entities, attributes, and relationships from the data to construct a medical knowledge graph. For example, entities might include diseases, symptoms, and medications; attributes might include disease incidence, symptom duration, and medication price; and relationships might include causal relationships between diseases and symptoms, or therapeutic relationships between medications and diseases.

[0149] Step 2: Task decomposition:

[0150] Based on the constructed medical knowledge graph, the data construction task can be broken down into multiple subtasks. For example, the task can be broken down into subtasks such as data collection, data cleaning, data conversion, data storage, and data analysis. Each subtask corresponds to one or more entities, attributes, and relationships in the knowledge graph.

[0151] Step 3: Task planning:

[0152] A task planning algorithm is used to generate a task plan based on the dependencies and priorities between subtasks. For example, data collection tasks must be executed before data cleaning tasks, and data analysis tasks must be executed only after data storage tasks are completed. The task plan can be represented using a directed acyclic graph (DAG), where nodes represent subtasks and edges represent the dependencies between subtasks.

[0153] Step 4: Task execution:

[0154] Schedule the execution of each subtask according to the task plan. Distributed computing frameworks such as Hadoop and Spark can be used to improve the parallelism and efficiency of task execution. During task execution, the execution status of the task needs to be monitored in real time to promptly identify and resolve any problems during execution.

[0155] Step 5: Task monitoring:

[0156] The task monitoring module provides real-time monitoring of the execution of data construction tasks. This includes subtask execution status, execution time, resource usage, and more. If a subtask fails or takes too long to execute, adjustments can be made promptly, such as re-executing the subtask or adjusting the task plan.

[0157] The above embodiments specifically illustrate the present invention, but the present invention is not limited to these embodiments. For those skilled in the art, various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0158] In summary, the present invention provides an efficient, accurate and adaptable data construction task planning method based on knowledge graph, which can effectively solve the problems existing in the existing technology and provide strong technical support for data management and application in the big data era.

[0159] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities and operations. Moreover, the terms "include", "comprises" and any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, and also includes elements inherent to such process, method, article or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0160] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A data construction task planning method based on knowledge graph, characterized by: The method comprises the following steps: Step 1: Use the knowledge graph construction module and knowledge graph analysis module to build and analyze the knowledge graph, and evaluate the priority of each subtask based on the entity importance, relationship strength, and business needs in the domain knowledge graph; Step 2: Decompose and model tasks through the task decomposition and modeling module, and integrate priority information into the task dependency graph to implement modeling; Step 3: Perform task planning and scheduling through the task planning and scheduling module to form a data construction task planning plan based on the knowledge graph; Step 4: Use the task monitoring and adjustment module to perform task execution and monitoring, monitor the execution status and progress of subtasks in real time, and use the knowledge graph to dynamically adjust and optimize according to actual conditions; Step 5: Through the user interaction module, the task execution results are displayed in real time in the form of graphics and reports, allowing users to configure the parameters of the data construction task and display the task execution results in the form of graphics and reports.

2. A data construction task planning method based on knowledge graph according to claim 1, characterized in that: The specific method of constructing and analyzing the knowledge graph is as follows: Step 100: Constructing a domain knowledge graph: For a specific domain, collect multi-source heterogeneous data, including structured data, semi-structured data, and unstructured data. Through data extraction, entity recognition, relationship extraction, attribute recognition, and knowledge fusion technologies, construct a domain knowledge graph covering the core concepts, entities, attributes, and relationships in the domain, and store it in a graph database. The entity recognition algorithm is as follows: P(y|x)=\frac{1}{Z(x)}\exp(\sum_{i,k}\lambda_kf_k(y_{i-1},y_i,x,i)) Where P(y|x) is the probability of state sequence y given the observation sequence x, Z(x) is the normalization factor, and i,k is the weight of the feature function f_k; The relation extraction algorithm is as follows: Score(y)=\sum_{k}\lambda_kf_k(y,x) Where Score(y) is the score of the parsing result y, k is the feature function, and f_k is the corresponding weight; Step 101, knowledge graph analysis: Analyze the domain knowledge graph constructed in step 100 to identify key entities, core relationships, and key paths and subgraphs related to data construction tasks.

3. A data construction task planning method based on knowledge graph according to claim 2, characterized in that: The steps of task decomposition and modeling are as follows: Step 102: Identify task dependencies: Based on the key entities, core relationships, and critical paths identified in step 101, analyze the dependencies between subtasks in the data construction task and construct a task dependency graph. Step 103, task priority evaluation: Evaluate the priority of each subtask based on the entity importance, relationship strength and business needs in the domain knowledge graph, and integrate the priority information into the task dependency graph.

4. The method for planning data construction tasks based on knowledge graph according to claim 1, characterized in that: The task planning and scheduling in step 3 are as follows: Step 104: Task sorting and scheduling: Based on the task dependency graph constructed in step 102 and the subtask priorities determined in step 103, a topological sorting algorithm is used to sort the subtasks, generate a task execution sequence that satisfies the dependency and priority constraints, and formulate a task scheduling plan that takes resource constraints into consideration. Step 105: Generate a task planning solution: Integrate the task execution sequence and the scheduling solution generated in step 104 to form a data-based task planning solution based on the knowledge graph, which is presented in a visual manner. The task execution and monitoring in step 4 are as follows: Step 106, task execution: executing each subtask according to the task planning scheme generated in step 105; Step 107: Task monitoring and adjustment: During the task execution process, the execution status and progress of subtasks are monitored in real time, and dynamic adjustments and optimizations are made based on the actual situation using the knowledge graph. Task planning in step three includes: analyzing the dependencies between subtasks and determining the execution order of subtasks; developing a reasonable task planning scheme based on the priorities and resource requirements of subtasks; and presenting the task planning scheme in a visual way to facilitate executors' understanding and execution. The task execution in step 4 includes: executing each subtask in sequence according to the task planning scheme; during the execution process, monitoring the execution status of the subtasks in real time and making adjustments based on actual conditions; after completing all subtasks, evaluating and summarizing the entire data construction task.

5. The method for planning data construction tasks based on knowledge graph according to claim 1, characterized in that: The knowledge graph construction module includes a data acquisition submodule, a data preprocessing submodule, an entity recognition submodule, a relationship extraction submodule, an attribute recognition submodule, a knowledge fusion submodule and a graph database storage submodule; The data acquisition submodule is responsible for collecting data from databases, files, API interfaces and web pages; The data preprocessing submodule is used to perform cleaning, deduplication and formatting preprocessing operations on the collected raw data; The entity recognition submodule is used to recognize names of people, places and institutions in the text; The relationship extraction submodule is used to identify the person relationships and event relationships between entities; The attribute recognition submodule is used to recognize the age and gender attributes of a person; The knowledge fusion submodule is used to fuse the extracted entities, relationships and attributes to solve entity disambiguation and knowledge conflict problems; The graph database storage submodule is used to store the constructed knowledge graph in the graph database.

6. The method for planning data construction tasks based on knowledge graph according to claim 1, characterized in that: The knowledge graph analysis module includes a key entity recognition submodule, a core relationship recognition submodule, a subgraph recognition submodule and a knowledge graph visualization submodule; The key entity identification submodule identifies key entities in the knowledge graph based on a graph algorithm; The core relationship identification submodule is used to identify the core relationship connecting key entities; The graph identification submodule is used to identify key paths and subgraphs related to data construction tasks; The knowledge graph visualization submodule is used to display the knowledge graph and analysis results in a graphical manner to facilitate user understanding.

7. The method for planning data construction tasks based on knowledge graph according to claim 1, characterized in that: The task decomposition and modeling module includes a task decomposition submodule, a task dependency relationship construction submodule, a task priority evaluation submodule and a task dependency relationship graph construction submodule; The task decomposition submodule decomposes the data construction task into executable subtasks based on the knowledge graph analysis results; The task dependency building submodule builds a dependency graph between subtasks based on the entity relationships and business processes in the knowledge graph; The task priority evaluation submodule evaluates the priority of subtasks based on the entity importance, relationship strength and business needs in the knowledge graph; The task dependency graph construction submodule is used to integrate task dependency and priority information into the task dependency graph.

8. The method for planning data construction tasks based on knowledge graph according to claim 1, characterized in that: The task planning and scheduling module includes a task sorting submodule, a resource allocation submodule, a task scheduling submodule and a task execution engine submodule; The task sorting submodule uses a graph theory algorithm to sort the task dependency graph and generate a feasible task execution sequence; The resource allocation submodule allocates computing resources and storage resources to each subtask based on task requirements and resource availability; The task scheduling submodule generates a task scheduling plan based on the task execution sequence and resource allocation; The task execution engine submodule is responsible for executing task scheduling plans, starting and monitoring the execution of subtasks.

9. The method for planning data construction tasks based on knowledge graph according to claim 1, characterized in that: The task monitoring and adjustment module includes a task status monitoring submodule, a knowledge graph updating submodule, a task planning optimization submodule and an exception handling submodule; The task status monitoring submodule is used to monitor the execution status, progress and resource usage of subtasks in real time; The knowledge graph updating submodule updates and improves the knowledge graph based on the new data generated during the task execution; The task planning optimization submodule dynamically adjusts and optimizes the task planning scheme based on the updated results of the knowledge graph and task execution feedback; The exception handling submodule is used to identify and handle exceptions during task execution.

10. The method for planning data construction tasks based on knowledge graph according to claim 1, characterized in that: The user interaction module includes a task configuration interface submodule, a knowledge graph browsing interface submodule, a task monitoring interface submodule and a result display interface submodule; The task configuration interface submodule is used to allow users to configure the parameters of the data construction task; The knowledge graph browsing interface submodule is used to allow users to browse and query the knowledge graph; The task monitoring interface submodule is used to allow users to view the task execution status and progress; The result display interface submodule displays the task execution results in graphical and report forms.

Images