A data intelligent processing method, device and equipment, and a storage medium

Abstract

The application relates to the computer field and provides a data intelligent processing method, device and equipment and a storage medium. Workflow information obtained is analyzed to obtain task detail information and task dependency information; a graph database statement is generated according to the task detail information and the task dependency information, and the graph database statement is written into a preset graph database; a user query statement constructed based on a query node is used to query the preset graph database to obtain a dependent node of the query node and a node dependency relationship between the query node and the dependent node. The application can quickly obtain the dependency of the query node, and improves the query efficiency between the node and the dependent node.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data intelligent processing method, apparatus, device, and storage medium. Background Technology

[0002] Currently, there are various workflow task scheduling platforms on the market, and there are many similar products. Taking Apache DolphinScheduler as an example, it is a distributed, decentralized, and easily scalable visual DAG (Directed Acyclic Graph) workflow task scheduling platform. This scheduling platform assembles tasks in a DAG streaming manner to solve the complex dependencies in the data development process.

[0003] However, the front-end page that comes with the Apache DolphinScheduler platform can only show the dependencies of a task in a single workflow by clicking on the single workflow page. This makes it impossible to quickly and easily view the dependencies of a task in multiple workflows, let alone quickly and easily find out how many tasks depend on a certain task. Summary of the Invention

[0004] In view of the above, it is necessary to provide a data intelligent processing method, apparatus, device and storage medium that can solve the technical problems of not being able to quickly and conveniently view the dependencies of a task in multiple workflows, and not being able to quickly and conveniently know how many tasks depend on a certain task.

[0005] On the one hand, this invention proposes a data intelligent processing method, which includes:

[0006] Parse the obtained workflow information to obtain task details and task dependency information;

[0007] Generate graph database statements based on task details and task dependencies, and write the graph database statements into the preset graph database;

[0008] Using user query statements constructed based on query nodes, a pre-defined graph database is queried to obtain the dependent nodes of the query nodes and the node dependency relationships between the query nodes and their dependent nodes.

[0009] According to a preferred embodiment of the present invention, the method further includes:

[0010] Workflow information is obtained based on the workflow form and the project form associated with the preset workflow.

[0011] According to a preferred embodiment of the present invention, obtaining workflow information based on a workflow form with a preset workflow and a project form associated with the workflow form includes:

[0012] Identify the project information corresponding to the preset workflow from the workflow form;

[0013] Obtain project forms based on project information;

[0014] Extract all information from workflow forms and project forms as workflow information.

[0015] According to a preferred embodiment of the present invention, when the task type of the work task included in the workflow information is a dependency type, parsing the obtained workflow information yields task details information and task dependency information, including:

[0016] Based on the preset dependency fields, the workflow identification code corresponding to the dependent workflow of the work task is parsed from the workflow information.

[0017] Based on the workflow identification code, locate the storage path of the workflow-dependent part from the project form;

[0018] Based on the storage path, retrieve all task information in the dependent workflow, and extract task information from the workflow information as task detail information;

[0019] The relationship between work tasks and dependent workflows is extracted from workflow information and used as task dependency information.

[0020] According to a preferred embodiment of the present invention, when the task type of the work task included in the workflow information is a non-dependent type, parsing the obtained workflow information yields task details information and task dependency information, including:

[0021] Based on preset prerequisite fields, the prerequisite tasks of the work tasks are parsed from the workflow information;

[0022] Obtain task information for work tasks and their predecessor tasks from workflow information as task details information;

[0023] The prerequisite relationships between work tasks and their predecessor tasks are used as task dependency information.

[0024] According to a preferred embodiment of the present invention, generating graph database statements based on task details and task dependency information specifically includes:

[0025] If the node corresponding to the task details does not exist in the preset graph database, generate a node creation statement based on the task details;

[0026] Generate statements based on task dependency information to create relationships;

[0027] Merge the node creation statements and relationship creation statements to obtain the graph database statements.

[0028] According to a preferred embodiment of the present invention, a user query statement constructed based on query nodes is used to query a preset graph database to obtain the dependent nodes of the query nodes and the node dependency relationship between the query nodes and the dependent nodes, specifically including:

[0029] Generate user query statements based on query nodes;

[0030] Retrieve the visualization page containing the workflow graph corresponding to the graph database statement from the preset graph database;

[0031] Run the user's query to obtain the dependent nodes and their dependencies;

[0032] Based on a visual page, dependent nodes and their dependencies are identified and displayed.

[0033] On the other hand, the present invention also proposes a data intelligence processing device, which includes:

[0034] The parsing unit is used to parse the acquired workflow information to obtain task details and task dependency information.

[0035] The generation unit is used to generate graph database statements based on task details and task dependency information, and write the graph database statements into a preset graph database.

[0036] The query unit is used to query a preset graph database using user query statements constructed based on query nodes, and to obtain the dependent nodes of the query nodes and the node dependency relationship between the query nodes and the dependent nodes.

[0037] On the other hand, the present invention also proposes an electronic device, the electronic device comprising:

[0038] Memory, which stores computer-readable instructions; and

[0039] A processor executes computer-readable instructions stored in memory to implement intelligent data processing methods.

[0040] On the other hand, the present invention also proposes a computer-readable storage medium storing computer-readable instructions, which are executed by a processor in an electronic device to realize a data intelligent processing method.

[0041] As can be seen from the above technical solutions, this application converts the preset workflow in the workflow task scheduling platform into graph database statements and writes them into the preset graph database. Then, it obtains the dependent nodes and node dependencies through the preset graph database containing the graph database statements. Since it is not necessary to manually click on the workflow tasks on the page, and at the same time, it can obtain the dependency status of the query node in multiple workflows at one time, the query efficiency of dependent nodes and node dependencies is improved. Attached Figure Description

[0042] Figure 1 This is a flowchart of a preferred embodiment of the data intelligent processing method of the present invention.

[0043] Figure 2 This is a visual representation of the workflow diagram in this invention.

[0044] Figure 3 This is a functional block diagram of a preferred embodiment of the data intelligent processing device of the present invention.

[0045] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the data intelligent processing method of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] like Figure 1 The diagram shown is a flowchart of a preferred embodiment of the data intelligent processing method of the present invention. The order of the steps in this flowchart can be changed, and some steps can be omitted, depending on different requirements.

[0048] Data intelligent processing methods are applied to one or more electronic devices. An electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0049] Electronic devices can be any electronic product that allows human-computer interaction, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0050] Electronic devices may include network devices and / or user devices. Among them, network devices include, but are not limited to, single network electronic devices, groups of electronic devices consisting of multiple network electronic devices, or cloud computing-based clouds consisting of a large number of hosts or network electronic devices.

[0051] The networks in which electronic devices are located include, but are not limited to: the Internet, wide area networks (WANs), metropolitan area networks (MANs), local area networks (LANs), and virtual private networks (VPNs).

[0052] 101. Parse the obtained workflow information to obtain task details and task dependency information.

[0053] In at least one embodiment of the present invention, taking the Apache DolphinScheduler workflow task scheduling platform as an example, the workflow information can be related information of the workflow table in the Apache DolphinScheduler workflow task scheduling platform. For example, the workflow information may include, but is not limited to, information such as workflow ID, workflow name, and project name to which the workflow belongs.

[0054] Task details refer to information related to work tasks in the workflow. Task details may include, but are not limited to, the task name, etc.

[0055] Task dependency information includes, but is not limited to: the dependency between a task and a workflow, and the dependency between a task and another task.

[0056] In at least one embodiment of the present invention, before parsing the acquired workflow information to obtain task details information and task dependency information, the method further includes:

[0057] Workflow information is obtained based on the workflow form and the project form associated with the preset workflow.

[0058] The preset workflow can be any workflow in the Apache DolphinScheduler workflow task scheduling platform.

[0059] Workflow forms refer to all forms associated with a predefined workflow.

[0060] A project form is a form corresponding to a project identifier stored in the workflow form. The project form records information related to the project corresponding to the preset workflow.

[0061] Workflow forms and associated project forms provide comprehensive access to workflow information.

[0062] Specifically, the electronic device obtains workflow information based on a preset workflow form and associated project forms, including:

[0063] Identify the project information corresponding to the preset workflow from the workflow form;

[0064] Obtain project forms based on project information;

[0065] Extract all information from workflow forms and project forms as workflow information.

[0066] The project information can be any project identifier, such as the project name or the project ID.

[0067] Project information can accurately locate project forms, thereby improving the accuracy of workflow information extraction.

[0068] In at least one embodiment of the present invention, when the task type of the work task included in the workflow information is a dependency type, the electronic device parses the obtained workflow information to obtain task details information and task dependency information, including:

[0069] Based on the preset dependency fields, the workflow identification code corresponding to the dependent workflow of the work task is parsed from the workflow information.

[0070] Based on the workflow identification code, locate the storage path of the workflow-dependent part from the project form;

[0071] Based on the storage path, retrieve all task information in the dependent workflow, and extract task information from the workflow information as task detail information;

[0072] The relationship between work tasks and dependent workflows is extracted from workflow information and used as task dependency information.

[0073] Among them, dependency type refers to the type of work task that depends on the workflow.

[0074] The default dependency field can be the field definitionId, which represents the workflow identifier of the task being depended upon.

[0075] The storage path refers to the path where information about all tasks in the dependent workflow is stored.

[0076] By pre-setting dependency fields, workflow identification codes can be quickly parsed from workflow information. Then, the storage path can be accurately located using the workflow identification code. This allows for the rapid retrieval of all task information in the dependent workflow, improving the efficiency of obtaining task details. Furthermore, since task dependency information can be directly parsed from workflow information, the parsing efficiency of task dependency information is improved.

[0077] Specifically, the electronic device obtains the task information of the work task from the workflow information based on the field depTasks as the task details information, where the field depTasks represents the task name of the work task.

[0078] In at least one embodiment of the present invention, when the task type of the work task included in the workflow information is a non-dependent type, the electronic device parses the obtained workflow information to obtain task details information and task dependency information, including:

[0079] Based on preset prerequisite fields, the prerequisite tasks of the work tasks are parsed from the workflow information;

[0080] Obtain task information for work tasks and their predecessor tasks from workflow information as task details information;

[0081] The prerequisite relationships between work tasks and their predecessor tasks are used as task dependency information.

[0082] Among them, non-dependent types include types such as SHELL.

[0083] Preset prerequisite fields are used to indicate the prerequisite tasks for a task. Preset prerequisite fields are usually represented by the field preTasks.

[0084] By pre-setting prerequisite fields, the prerequisite tasks can be directly parsed out. Then, by combining the work task and the prerequisite tasks, comprehensive task details and task dependency information can be obtained.

[0085] 102. Generate graph database statements based on task details and task dependency information, and write the graph database statements into the preset graph database.

[0086] In at least one embodiment of the present invention, graph database statements are used to create nodes and node relationships.

[0087] The default graph database refers to the platform database of the Apache DolphinScheduler workflow task scheduling platform. The default graph database can typically be any type of graph database, including but not limited to Neo4j, which is an embedded, disk-based Java persistence engine with full transactional features.

[0088] In at least one embodiment of the present invention, the electronic device generates graph database statements based on task details information and task dependency information, specifically including:

[0089] If the node corresponding to the task details does not exist in the preset graph database, generate a node creation statement based on the task details;

[0090] Generate statements based on task dependency information to create relationships;

[0091] Merge the node creation statements and relationship creation statements to obtain the graph database statements.

[0092] The node creation statement is used to create nodes in the Apache DolphinScheduler workflow task scheduling platform. For example, the node creation statement can be MERGE(task-Name:DsTask{name:taskName,projectId:projectId,processId:processId,attribute1:value1,...,attributen:valuen}), where the MERGE keyword is the keyword for creating nodes.

[0093] Relationship creation statements are used to create node relationships in the Apache DolphinScheduler workflow task scheduling platform. For example, a relationship creation statement could be MATCH(depend_taskName:DsTask{name:depend_taskName,projectId:projectId,processId:processId,attribute1:value1,...,attribute...).

[0094] n:valuen}),(taskName:DsTask{name:taskName,projectId:projectId,processId:pro cessId,attribute 1:value1,...,attribute n:valuen})

[0095] MERGE(depend_taskName)-[taskR:TASK_RELATIONSHIP]->(task-Name), where depend_taskName is the prerequisite task for the job task. When there are multiple prerequisite tasks, they can exist in the relation creation statement as an array, and taskName is the job task.

[0096] Through the above implementation method, even when there is no node corresponding to the task details in the preset graph database, the node and node relationship can be created in the Apache DolphinScheduler workflow task scheduling platform by using graph database statements.

[0097] In at least one embodiment of the present invention, if a node corresponding to task details exists in a preset graph database, the attribute value and storage relationship of the node in the preset graph database are obtained, the attribute value is compared with the task details, and the storage relationship is compared with the task dependency information. The task details that are not present in the attribute value are determined as target information, and the task dependency information that is not present in the storage relationship is determined as target relationship. The graph database statement of the node is generated based on the target information and target relationship.

[0098] By comparing attribute values ​​with task details, the information integrity of nodes on the Apache Dolphin-Scheduler workflow task scheduling platform can be improved.

[0099] 103. Using user query statements constructed based on query nodes, query the preset graph database to obtain the dependent nodes of the query nodes and the node dependency relationship between the query nodes and the dependent nodes.

[0100] In at least one embodiment of the present invention, a query node refers to the node corresponding to the work task that requires a dependency query.

[0101] User queries are used to indicate the dependencies of a query node. For example, a user query could be MATCH(a)-[taskR:TASK_RELATIONSHIP]->(b).

[0102] A dependent node is a node that has a dependency relationship with a query node. Dependent nodes include the preceding nodes of the query node and all process nodes in the dependent workflow of the query node.

[0103] Node dependencies are used to indicate the process execution relationship between query nodes and dependent nodes.

[0104] In at least one embodiment of the present invention, the electronic device uses a user query statement constructed based on query nodes to query a preset graph database, and obtains the dependent nodes of the query nodes and the node dependency relationship between the query nodes and the dependent nodes, specifically including:

[0105] Generate user query statements based on query nodes;

[0106] Retrieve the visualization page containing the workflow graph corresponding to the graph database statement from the preset graph database;

[0107] Run the user's query to obtain the dependent nodes and their dependencies;

[0108] Based on a visual page, dependent nodes and their dependencies are identified and displayed.

[0109] The workflow graph includes nodes created by graph database statements and their relationships, such as... Figure 2 As shown, Figure 2 This is a visual representation of the workflow diagram in this invention. Figure 2 The workflow graph includes nodes A, B, C, D, and E. Figure 2 The arrows in the diagram represent the dependencies between nodes. For example, if a user query is generated based on node B, then the dependent node obtained by running the user query is node C, and the resulting node dependency relationship is the relationship between node B and node C.

[0110] By running user queries in a pre-defined graph database, the dependencies of the query nodes can be obtained in one go, thereby improving the efficiency of obtaining dependent nodes and node dependencies. Furthermore, through the visualization page of the workflow graph corresponding to the graph database query, the dependent nodes and node dependencies can be directly identified and displayed, improving the intuitiveness of the display of the dependencies of the query nodes.

[0111] As can be seen from the above technical solutions, this application converts the preset workflow in the workflow task scheduling platform into graph database statements and writes them into the preset graph database. Then, it obtains the dependent nodes and node dependencies through the preset graph database containing the graph database statements. Since it is not necessary to manually click on the workflow tasks on the page, and at the same time, it can obtain the dependency status of the query node in multiple workflows at one time, the query efficiency of dependent nodes and node dependencies is improved.

[0112] like Figure 3The diagram shown is a functional block diagram of a preferred embodiment of the data intelligent processing device of the present invention. The data intelligent processing device 11 includes a parsing unit 110, a generation unit 111, a query unit 112, and an acquisition unit 113. The module / unit referred to in this invention refers to a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, and are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0113] The parsing unit 110 parses the obtained workflow information to obtain task details and task dependency information.

[0114] In at least one embodiment of the present invention, the workflow information may be related information of the workflow table in the Apache Dolphin-Scheduler workflow task scheduling platform. For example, the workflow information may include, but is not limited to, information such as workflow ID, workflow name, and project name to which the workflow belongs.

[0115] Task details refer to information related to work tasks in the workflow. Task details may include, but are not limited to, the task name, etc.

[0116] Task dependency information includes, but is not limited to: the dependency between a task and a workflow, and the dependency between a task and another task.

[0117] In at least one embodiment of the present invention, before parsing the acquired workflow information to obtain task details information and task dependency information, the acquisition unit 113 acquires workflow information based on the workflow form of the preset workflow and the project form associated with the workflow form.

[0118] The preset workflow can be any workflow in the Apache DolphinScheduler workflow task scheduling platform.

[0119] Workflow forms refer to all forms associated with a predefined workflow.

[0120] A project form is a form corresponding to a project identifier stored in the workflow form. The project form records information related to the project corresponding to the preset workflow.

[0121] Workflow forms and associated project forms provide comprehensive access to workflow information.

[0122] Specifically, the acquisition unit 113 acquires workflow information based on the preset workflow's workflow form and the project form associated with the workflow form, including:

[0123] Identify the project information corresponding to the preset workflow from the workflow form;

[0124] Obtain project forms based on project information;

[0125] Extract all information from workflow forms and project forms as workflow information.

[0126] The project information can be any project identifier, such as the project name or the project ID.

[0127] Project information can accurately locate project forms, thereby improving the accuracy of workflow information extraction.

[0128] In at least one embodiment of the present invention, when the task type of the work task included in the workflow information is a dependency type, the parsing unit 110 parses the acquired workflow information to obtain task details information and task dependency information, including:

[0129] Based on the preset dependency fields, the workflow identification code corresponding to the dependent workflow of the work task is parsed from the workflow information.

[0130] Based on the workflow identification code, locate the storage path of the workflow-dependent part from the project form;

[0131] Based on the storage path, retrieve all task information in the dependent workflow, and extract task information from the workflow information as task detail information;

[0132] The relationship between work tasks and dependent workflows is extracted from workflow information and used as task dependency information.

[0133] Among them, dependency type refers to the type of work task that depends on the workflow.

[0134] The default dependency field can be the field definitionId, which represents the workflow identifier of the task being depended upon.

[0135] The storage path refers to the path where information about all tasks in the dependent workflow is stored.

[0136] By pre-setting dependency fields, workflow identification codes can be quickly parsed from workflow information. Then, the storage path can be accurately located using the workflow identification code. This allows for the rapid retrieval of all task information in the dependent workflow, improving the efficiency of obtaining task details. Furthermore, since task dependency information can be directly parsed from workflow information, the parsing efficiency of task dependency information is improved.

[0137] Specifically, the parsing unit 110 obtains the task information of the work task from the workflow information based on the field depTasks as the task details information, where the field depTasks represents the task name of the work task.

[0138] In at least one embodiment of the present invention, when the task type of the work task included in the workflow information is a non-dependent type, the parsing unit 110 parses the obtained workflow information to obtain task details information and task dependency information, including:

[0139] Based on preset prerequisite fields, the prerequisite tasks of the work tasks are parsed from the workflow information;

[0140] Obtain task information for work tasks and their predecessor tasks from workflow information as task details information;

[0141] The prerequisite relationships between work tasks and their predecessor tasks are used as task dependency information.

[0142] Among them, non-dependent types include types such as SHELL.

[0143] Preset prerequisite fields are used to indicate the prerequisite tasks for a task. Preset prerequisite fields are usually represented by the field preTasks.

[0144] By pre-setting prerequisite fields, the prerequisite tasks can be directly parsed out. Then, by combining the work task and the prerequisite tasks, comprehensive task details and task dependency information can be obtained.

[0145] The generation unit 111 generates graph database statements based on task details and task dependency information, and writes the graph database statements into a preset graph database.

[0146] In at least one embodiment of the present invention, graph database statements are used to create nodes and node relationships.

[0147] The default graph database refers to the platform database of the Apache DolphinScheduler workflow task scheduling platform. The default graph database is typically a Neo4j database, which is an embedded, disk-based Java persistence engine with full transactional features.

[0148] In at least one embodiment of the present invention, the generation unit 111 generates graph database statements based on task details information and task dependency information, specifically including:

[0149] If the node corresponding to the task details does not exist in the preset graph database, generate a node creation statement based on the task details;

[0150] Generate statements based on task dependency information to create relationships;

[0151] Merge the node creation statements and relationship creation statements to obtain the graph database statements.

[0152] The node creation statement is used to create nodes in the Apache DolphinScheduler workflow task scheduling platform. For example, the node creation statement can be MERGE(task-Name:DsTask{name:taskName,projectId:projectId,processId:processId,attribute1:value1,...,attributen:valuen}), where the MERGE keyword is the keyword for creating nodes.

[0153] Relationship creation statements are used to create node relationships in the Apache DolphinScheduler workflow task scheduling platform. For example, a relationship creation statement could be MATCH(depend_taskName:DsTask{name:depend_taskName,projectId:projectId,processId:processId,attribute1:value1,...,attribute...).

[0154] n:valuen}),(taskName:DsTask{name:taskName,projectId:projectId,processId:pro cessId,attribute 1:value1,...,attribute n:valuen})

[0155] MERGE(depend_taskName)-[taskR:TASK_RELATIONSHIP]->(task-Name), where depend_taskName is the prerequisite task for the job task. When there are multiple prerequisite tasks, they can exist in the relation creation statement as an array, and taskName is the job task.

[0156] Through the above implementation method, even when there is no node corresponding to the task details in the preset graph database, the node and node relationship can be created in the Apache DolphinScheduler workflow task scheduling platform by using graph database statements.

[0157] In at least one embodiment of the present invention, if a node corresponding to task details exists in the preset graph database, the generation unit 111 obtains the attribute value and storage relationship of the node in the preset graph database, compares the attribute value with the task details, compares the storage relationship with the task dependency information, determines the task details that are not present in the attribute value as the target information, and determines the task dependency information that is not present in the storage relationship as the target relationship, and generates the graph database statement of the node based on the target information and the target relationship.

[0158] By comparing attribute values ​​with task details, the information integrity of nodes on the Apache Dolphin-Scheduler workflow task scheduling platform can be improved.

[0159] The query unit 112 uses the user query statement constructed based on the query node to query the preset graph database and obtain the dependent nodes of the query node and the node dependency relationship between the query node and the dependent nodes.

[0160] In at least one embodiment of the present invention, a query node refers to the node corresponding to the work task that requires a dependency query.

[0161] User queries are used to indicate the dependencies of a query node. For example, a user query could be MATCH(a)-[taskR:TASK_RELATIONSHIP]->(b).

[0162] A dependent node is a node that has a dependency relationship with a query node. Dependent nodes include the preceding nodes of the query node and all process nodes in the dependent workflow of the query node.

[0163] Node dependencies are used to indicate the process execution relationship between query nodes and dependent nodes.

[0164] In at least one embodiment of the present invention, the query unit 112 uses a user query statement constructed based on the query node to query a preset graph database, and obtains the dependent nodes of the query node and the node dependency relationship between the query node and the dependent nodes, specifically including:

[0165] Generate user query statements based on query nodes;

[0166] Retrieve the visualization page containing the workflow graph corresponding to the graph database statement from the preset graph database;

[0167] Run the user's query to obtain the dependent nodes and their dependencies;

[0168] Based on a visual page, dependent nodes and their dependencies are identified and displayed.

[0169] The workflow graph includes nodes created by graph database statements and their relationships, such as... Figure 2 As shown, Figure 2 This is a visual representation of the workflow diagram in this invention. Figure 2 The workflow graph includes nodes A, B, C, D, and E. Figure 2 The arrows in the diagram represent the dependencies between nodes. For example, if a user query is generated based on node B, then the dependent node obtained by running the user query is node C, and the resulting node dependency relationship is the relationship between node B and node C.

[0170] By running user queries in a pre-defined graph database, the dependencies of the query nodes can be obtained in one go, thereby improving the efficiency of obtaining dependent nodes and node dependencies. Furthermore, through the visualization page of the workflow graph corresponding to the graph database query, the dependent nodes and node dependencies can be directly identified and displayed, improving the intuitiveness of the display of the dependencies of the query nodes.

[0171] As can be seen from the above technical solutions, this application converts the preset workflow in the workflow task scheduling platform into graph database statements and writes them into the preset graph database. Then, it obtains the dependent nodes and node dependencies through the preset graph database containing the graph database statements. Since it is not necessary to manually click on the workflow tasks on the page, and at the same time, it can obtain the dependency status of the query node in multiple workflows at one time, the query efficiency of dependent nodes and node dependencies is improved.

[0172] like Figure 4 The diagram shown is a schematic representation of the structure of an electronic device that implements the intelligent data processing method of the present invention.

[0173] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and computer-readable instructions, such as a data intelligence processing program, stored in the memory 12 and executable on the processor 13.

[0174] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 1 may also include input / output devices, network access devices, buses, etc.

[0175] Processor 13 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. Processor 13 is the computational core and control center of electronic device 1, connecting various parts of electronic device 1 through various interfaces and lines, and executing the operating system of electronic device 1, as well as various installed application programs and program code.

[0176] For example, computer-readable instructions can be divided into one or more modules / units, one or more of which are stored in memory 12 and executed by processor 13 to complete the present invention. One or more modules / units can be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer-readable instructions in electronic device 1. For example, the computer-readable instructions can be divided into a parsing unit 110, a generation unit 111, a query unit 112, and a retrieval unit 113.

[0177] The memory 12 can be used to store computer-readable instructions and / or modules. The processor 13 implements various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and by calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. The memory 12 may include non-volatile and volatile memory, such as: hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other storage devices.

[0178] The memory 12 can be the external memory and / or internal memory of the electronic device 1. Furthermore, the memory 12 can be a physical memory, such as a memory stick, a TF card (Trans-flash Card), etc.

[0179] If the modules / units integrated in electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when executed by a processor, the computer-readable instructions can implement the steps of the various method embodiments described above.

[0180] Computer-readable instructions include computer-readable instruction code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer-readable instruction code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), and random access memory (RAM).

[0181] Combination Figure 1 The memory 12 in electronic device 1 stores computer-readable instructions to implement a data intelligent processing method, and the processor 13 can execute the computer-readable instructions to achieve:

[0182] Parse the obtained workflow information to obtain task details and task dependency information;

[0183] Generate graph database statements based on task details and task dependencies, and write the graph database statements into the preset graph database;

[0184] Using user query statements constructed based on query nodes, a pre-defined graph database is queried to obtain the dependent nodes of the query nodes and the node dependency relationships between the query nodes and their dependent nodes.

[0185] Specifically, the specific implementation method of the processor 13 for the above-mentioned computer-readable instructions can be found in [reference]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0186] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0187] A computer-readable storage medium stores computer-readable instructions, which, when executed by processor 13, are used to perform the following steps:

[0188] Parse the obtained workflow information to obtain task details and task dependency information;

[0189] Generate graph database statements based on task details and task dependencies, and write the graph database statements into the preset graph database;

[0190] Using user query statements constructed based on query nodes, a pre-defined graph database is queried to obtain the dependent nodes of the query nodes and the node dependency relationships between the query nodes and their dependent nodes.

[0191] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0192] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0193] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0194] Furthermore, it is clear that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices can also be implemented by a single unit or device through software or hardware. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A data intelligent processing method, characterized in that, The method includes: Parse the obtained workflow information to obtain task details and task dependency information; Generate a graph database statement based on the task details and the task dependency information, and write the graph database statement into a preset graph database. The step of generating the graph database statement based on the task details and the task dependency information includes: if a node corresponding to the task details exists in the preset graph database, obtain the attribute values ​​and storage relationships of the node corresponding to the task details in the preset graph database, compare the attribute values ​​with the task details, and compare the storage relationships with the task dependency information; determine the task details that do not exist in the attribute values ​​as target information, and determine the task dependency information that does not exist in the storage relationships as target relationships; and generate the graph database statement based on the target information and the target relationships. The preset graph database is queried using user query statements constructed based on query nodes to obtain the dependent nodes of the query node and the node dependency relationship between the query node and the dependent nodes.

2. The data intelligent processing method as described in claim 1, characterized in that, The method further includes: The workflow information is obtained based on the workflow form of the preset workflow and the project form associated with the workflow form.

3. The data intelligent processing method as described in claim 2, characterized in that, The workflow form based on the preset workflow and the project form associated with the workflow form, obtaining the workflow information includes: Identify the project information corresponding to the preset workflow from the workflow form; Obtain the project form based on the project information; Extract all information from the workflow form and the project form as the workflow information.

4. The data intelligent processing method as described in claim 2, characterized in that, When the workflow information includes a task type that is a dependency type, the parsed workflow information, which yields task details and task dependency information, includes: Based on the preset dependency fields, the workflow identification code corresponding to the dependent workflow of the work task is parsed from the workflow information; Based on the workflow identification code, the storage path of the dependent workflow is located from the project form; Based on the storage path, obtain all task information in the dependent workflow, and obtain the task information of the work task from the workflow information as the task detail information; The relationship between the work task and the dependent workflow is parsed from the workflow information and used as the task dependency information.

5. The data intelligent processing method as described in claim 1, characterized in that, When the task type of the work task included in the workflow information is a non-dependent type, the parsed workflow information, including task details and task dependency information, includes: Based on preset prerequisite fields, the prerequisite tasks for the work task are parsed from the workflow information; The task information of the work task and the preceding task is obtained from the workflow information as the task details information; The prerequisite relationship between the work task and the preceding task is used as the task dependency information.

6. The data intelligent processing method as described in claim 1, characterized in that, The step of generating graph database statements based on the task details and task dependency information specifically includes: If the preset graph database does not contain a node corresponding to the task details information, a node creation statement is generated based on the task details information; Generate a statement based on the task dependency information to create a relationship; The node creation statement and the relationship creation statement are combined to obtain the graph database statement.

7. The data intelligent processing method as described in claim 1, characterized in that, The step of using a user query statement constructed based on query nodes to query the preset graph database and obtain the dependent nodes of the query node and the node dependency relationship between the query node and the dependent nodes specifically includes: The user query statement is generated based on the query node; Retrieve the visualization page containing the workflow graph corresponding to the graph database statement from the preset graph database; Run the user query to obtain the dependent nodes and their dependencies; Based on the visualization page, the dependent nodes and their dependencies are identified and displayed.

8. A data intelligent processing device, characterized in that, The data intelligent processing device includes: The parsing unit is used to parse the acquired workflow information to obtain task details and task dependency information. A generation unit is configured to generate a graph database statement based on the task details information and the task dependency information, and write the graph database statement into a preset graph database. The step of generating the graph database statement based on the task details information and the task dependency information includes: if a node corresponding to the task details information exists in the preset graph database, obtaining the attribute values ​​and storage relationships of the node corresponding to the task details information in the preset graph database; comparing the attribute values ​​with the task details information; comparing the storage relationships with the task dependency information; determining the task details information not present in the attribute values ​​as target information; determining the task dependency information not present in the storage relationships as target relationships; and generating the graph database statement based on the target information and the target relationships. The query unit is used to query the preset graph database using a user query statement constructed based on the query node, and to obtain the dependent nodes of the query node and the node dependency relationship between the query node and the dependent nodes.

9. An electronic device, characterized in that, The electronic device includes: Memory, which stores computer-readable instructions; and The processor executes computer-readable instructions stored in the memory to implement the data intelligent processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which are executed by a processor in an electronic device to implement the data intelligent processing method as described in any one of claims 1 to 7.

Images