Data processing method and task execution method

Abstract

The invention provides a data processing method and a task execution method, and the data processing method comprises the steps: obtaining a pre-generated task relation model and a workflow constructed based on the task relation model, and determining workflow data based on the workflow; generating a target data source based on the data features of the workflow data; based on the attribute characteristics of the target data source, respectively establishing a binding relationship with the task relationship model and the workflow; and processing the workflow data of the target data source based on the binding relationship. Based on the above data processing method, adaptive processing is performed on the workflow data based on the data features of the workflow data, so that the processing efficiency of the workflow data is improved, and the security of the workflow data is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data processing method and a task execution method. BACKGROUND

[0002] This section is intended to provide background information to facilitate a better understanding of embodiments of the present application recited in the claims. The description herein does not constitute admission of prior art.

[0003] In a traditional application scenario, the management and scheduling of offline tasks mainly rely on the task management tools of ordinary task platforms or servers. Although this approach can meet the needs of simple timing tasks, it exposes many limitations when facing complex business scenarios.

[0004] From the perspective of task dependency, ordinary task platforms and existing task management tools only support single task timing triggering, lack effective management of logical dependencies between tasks, and if data collection is delayed due to network latency, subsequent tasks will still start according to the preset time, which will cause data processing errors.

[0005] In terms of task execution order, the traditional approach cannot handle task chains with complex dependencies. When multiple tasks have multi-level dependencies, the administrator needs to manually sort out the start order of each task and set the timing rules separately, which is not only tedious, but also prone to errors due to human negligence. More importantly, there is a problem of data flow between multiple tasks. When the data flow between any two tasks is too large, it will cause processing pressure on the entire task link and even cause system crashes.

[0006] Therefore, there is an urgent need to provide a data processing method and a task execution method to solve the above technical problems. SUMMARY

[0007] Aspects of the present application provide a data processing method and a task execution method to maintain a complex chain asynchronous workflow data processing flow.

[0008] In an aspect of the present application, a data processing method is provided, which comprises: obtaining a pre-generated task relationship model and a workflow constructed based on the task relationship model, and determining workflow data based on the workflow; generating a target data source based on the data characteristics of the workflow data; establishing a binding relationship between the attribute characteristics of the target data source and the task relationship model and the workflow, respectively; processing the workflow data of the target data source based on the binding relationship.

[0009] Further, the target data source is generated based on the data features of the workflow data, including: obtaining business data of the workflow data; and generating the target data source based on the business data.

[0010] Further, the target data source is generated based on the data features of the workflow data, including: obtaining business data of the workflow data; and generating the target data source based on the business data. Further, the target data source is generated based on the data features of the workflow data, including: obtaining business data of the workflow data; and generating the target data source based on the business data.

[0011] Further, the target data source is generated based on the data features of the workflow data, including: obtaining business data of the workflow data; and generating the target data source based on the business data.

[0012] Further, the target data source is generated based on the data features of the workflow data, including: obtaining business data of the workflow data; and generating the target data source based on the business data.

[0013] Further, the target data source is generated based on the data features of the workflow data, including: obtaining business data of the workflow data; and generating the target data source based on the business data.

[0014] Further, the updating the persistent data source based on the association feature of the task node comprises: when the association feature of the task node is serial execution, sequentially updating target workflow data in the persistent data source based on a serial execution order of the task node; when the association feature of the task node is parallel execution, respectively updating target workflow data in the persistent data source based on a parallel execution order of the task node; wherein the target workflow data is workflow data in the persistent data source associated with an output result of executing the task node.

[0015] Further, the processing the workflow data of the target data source based on the binding relationship comprises: obtaining a lifecycle of the workflow; when the lifecycle of the workflow is ended, erasing a one-time data source having a second binding relationship with the workflow.

[0016] In another aspect of the present application, a task execution method is provided, the method comprising: in response to a task execution request, obtaining a task to be executed and an association of any two tasks to be executed; generating a task relationship model based on the association of the any two tasks to be executed and constructing a workflow corresponding to the task relationship model; executing the task to be executed based on the workflow and the task relationship model; wherein the workflow data generated in the execution of the task to be executed is processed based on the data processing method as described above.

[0017] Further, the generating a task relationship model based on the association of the any two tasks to be executed and constructing a workflow corresponding to the task relationship model comprises: obtaining an identifier of any of the tasks to be executed, an association between any two of the tasks to be executed, and additional information of the tasks to be executed; generating a map array based on the additional information of the tasks to be executed; determining other tasks to be executed having an association with any of the tasks to be executed, and generating a two-dimensional array based on the identifier of the task to be executed and the identifier of the other task to be executed; and generating the task relationship model based on the map array and the two-dimensional array.

[0018] In still another aspect of the present application, a data processing device is provided, the device comprising: an obtaining unit configured to obtain a pre-generated task relationship model and a workflow constructed based on the task relationship model, and determine workflow data based on the workflow; a generating unit configured to generate a target data source based on a data feature of the workflow data; a building unit configured to establish a binding relationship between the target data source and the task relationship model and the workflow based on attribute features of the target data source; a processing unit configured to process workflow data of the target data source based on the binding relationship.

[0019] In another aspect of the present application, a task execution device is provided, which comprises: an obtaining unit configured to obtain a task to be executed and the relevance of any two tasks to be executed in response to a task execution request; a building unit configured to generate a task relationship model based on the relevance of the any two tasks to be executed and build a workflow corresponding to the task relationship model; an execution unit configured to execute the task to be executed based on the workflow and the task relationship model; wherein the workflow data generated in the execution of the task to be executed is processed based on the data processing device as described above.

[0020] In another aspect of the present application, an electronic device is provided, which comprises: at least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0021] In yet another aspect of the present application, a computer readable medium is provided, which stores computer program instructions executable by a processor to implement the method as described above.

[0022] In another aspect of the present application, a computer program product is provided, which comprises a computer program, and the computer program is executable by a processor to implement the method as described above.

[0023] Firstly, the data processing method proposed in the present application obtains a pre-generated task relationship model and a workflow built based on the task relationship model, and determines workflow data based on the workflow; generates a target data source based on data features of the workflow data; establishes a binding relationship between the attribute features of the target data source and the task relationship model and the workflow, and processes the workflow data of the target data source based on the binding relationship. By maintaining asynchronous tasks through workflows, fine-grained control over task execution flow can be achieved, ensuring that tasks are executed in the pre-set order and dependency relationship, improving the controllability and consistency of the overall process. On the one hand, workflows can effectively manage the execution status of tasks, including recording task progress, execution results, and handling abnormal situations, making the task execution process more transparent and facilitating monitoring and operation. In the face of failures, workflow engines usually support mechanisms such as automatic retry, task rollback, and breakpoint recovery, enhancing the fault tolerance and stability of the system. Especially in distributed systems, workflows can help coordinate asynchronous tasks between different nodes and services, ensuring the correct execution order of tasks and avoiding data inconsistency problems caused by network delays or service abnormalities. On the other hand, most workflow systems also provide visual interfaces to facilitate users to view task execution status, analyze log information, and perform necessary tuning operations, thereby improving the maintainability and operation efficiency of the system; Overall, using workflows to manage asynchronous tasks not only improves the flexibility and reliability of task scheduling, but also significantly reduces the development and maintenance costs of complex business processes, making it an important means for enterprises to achieve automated and intelligent task scheduling.

[0024] Secondly, the task execution method proposed in this application, by responding to the task execution request, obtains the to-be-executed tasks and the correlation between any two to-be-executed tasks; generates a task relationship model based on the correlation between the two to-be-executed tasks and constructs a workflow corresponding to the task relationship model; based on the workflow and the task relationship model, the smooth and efficient execution of the to-be-executed tasks is realized; This scheme first obtains the to-be-executed tasks and the correlation between any two tasks after receiving the task execution request, and constructs a task relationship model and a corresponding workflow based on these correlations, thereby realizing the orderly and efficient processing of to-be-executed tasks; On the one hand, it can automatically identify the dependency or coordination relationship between tasks, thereby optimizing the task scheduling order and reducing unnecessary waiting time and resource waste; On the other hand, by constructing a reasonable task relationship model, the system can better cope with complex and variable task scenarios, improve overall execution efficiency and automation level, and ultimately achieve the effects of improving work efficiency, reducing the degree of manual intervention, and enhancing the reliability of task execution. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0026] Other features, objects, and advantages of the application will become more apparent from the following detailed description when read in connection with the following drawings: Figure 1 A flowchart of a data processing method according to an embodiment of the application is shown in FIG. 1; Figure 2 A flowchart of a task execution method according to another embodiment of the application is shown in FIG. 2; Figure 3 A structure diagram of a data processing device according to still another embodiment of the application is shown in FIG. 3; Figure 4 A structure diagram of a task execution device according to yet another embodiment of the application is shown in FIG. 4; Figure 5 A structure diagram of an electronic device suitable for implementing the scheme according to an embodiment of the application is shown in FIG. 5; The same or similar reference signs in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0027] In order to make the objects, technical schemes and advantages of the embodiments of the application clearer, the technical schemes in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are some but not all of the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the application.

[0028] In a typical configuration of the application, the devices of the terminal and the service network each include one or more processors (CPU), input / output interfaces, network interfaces and memories.

[0029] The memory can include non-permanent memory in a computer readable medium, random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of the computer readable medium.

[0030] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for information storage. Information can be computer program instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0031] The embodiment of the present application provides a data processing method, which comprises at least the following processing steps: Step S101, obtaining a task relationship model generated in advance and a workflow constructed based on the task relationship model, and determining workflow data based on the workflow; Step S102, generating a target data source based on the data characteristics of the workflow data; Step S103, establishing a binding relationship between the attribute characteristics of the target data source and the task relationship model and the workflow respectively; Step S104, processing the workflow data of the target data source based on the binding relationship.

[0032] In actual scenarios, the execution subject of the method can be a user device, or a device integrated by a user device and a network device through a network, or an application program running on the above-mentioned device. The user device includes but is not limited to computers, mobile phones, tablet computers, smart watches, wristbands and various terminal devices, and the network device includes but is not limited to network hosts, single network servers, multiple network server sets or computer sets based on cloud computing, which can be used to realize part of the processing function when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, wherein cloud computing is a kind of distributed computing, which is composed of a virtual computer set by a loose coupling computer group.

[0033] Embodiment one

[0034] Figure 1 A data processing method provided by the embodiment of the present application is shown, which comprises at least the following processing steps: In step S101, a pre-generated task relationship model and a workflow constructed based on the task relationship model are acquired, and workflow data is determined based on the workflow; In step S102, a target data source is generated based on data features of the workflow data; In step S103, a binding relationship is established between the target data source and the task relationship model and the workflow based on attribute features of the target data source. In step S104, the workflow data of the target data source is processed based on the binding relationship.

[0035] The data processing method proposed in this embodiment can reasonably process workflow data generated in the execution process of a to-be-executed task, thereby ensuring the flow efficiency of the workflow data and the security of the workflow data.

[0036] In one of the embodiments, the target data source is generated based on data features of the workflow data, including: acquiring business data of the workflow data; and generating the target data source based on the business data.

[0037] Specifically, the workflow data is roughly divided into process data and business data. The process data mainly includes data for controlling the flow of the workflow, including but not limited to task relationship model, to-be-executed task, association between to-be-executed tasks in the task relationship model, and other workflow entity data. The process data can also include the workflow itself, the state of the to-be-executed task, input parameters and output parameters of the to-be-executed task, etc. The business data is data that needs to be executed and processed by the to-be-executed task. The data amount is generally large, and therefore the data needs to be stored in a data source.

[0038] In one of the embodiments, the binding relationship is established between the attribute features of the target data source and the task relationship model and the workflow, including: acquiring the timeliness of the target data source; when the timeliness is high, determining that the target data source is a persistent data source, acquiring a task relationship model corresponding to the persistent data source, and establishing a first binding relationship between the persistent data source and the task relationship model; and when the timeliness is low, determining that the target data source is a one-time data source, acquiring a workflow corresponding to the one-time data source, and establishing a second binding relationship between the one-time data source and the workflow.

[0039] Specifically, the life cycle of the one-time data source in the target data source is a workflow, and the one-time data source ends when the workflow ends. The life cycle of the persistent data source does not change with the life cycle of the workflow.

[0040] In one embodiment, the processing of the workflow data of the target data source based on the binding relationship comprises: obtaining the number of task relationship models that have the first binding relationship with the persistent data source, and when the number of task relationship models exceeds one, generating a first identifier corresponding to each of the task relationship models, and isolating the persistent data source based on the first identifier; obtaining the number of workflows that have the second binding relationship with the one-time data source, and when the number of workflows exceeds one, generating a second identifier corresponding to each of the workflows, and isolating the one-time data source based on the second identifier.

[0041] Specifically, for a task relationship model or a workflow, a task relationship model and a workflow can bind multiple target data sources; similarly, a target data source can also bind multiple task relationship models or multiple workflows.

[0042] When multiple workflows in a task relationship model are executed in parallel, if data isolation is not added, it may cause data interference between multiple workflows, resulting in data disorder; by introducing the first identifier and the second identifier to distinguish the persistent data source and the one-time data source, respectively, an independent data space can be provided for each execution, effectively avoiding such conflicts. In addition, in the case that a to-be-executed task is reused by multiple task relationship models, if there is no good data isolation mechanism, it is also easy to cause cross-task relationship model data pollution problems; and by distinguishing and isolating the persistent data source based on the first identifier, the data isolation between processes can be realized while ensuring the reusability of the to-be-executed task, ensuring the safety and accuracy of task execution.

[0043] In one embodiment, the processing of the workflow data of the target data source based on the binding relationship further comprises: based on each task relationship model, obtaining the number of persistent data sources that have the first binding relationship with the task relationship model; when the number of persistent data sources exceeds one, obtaining the task node of the task relationship model; determining the corresponding persistent data source based on the task node, and establishing a third binding relationship based on the persistent data source and the task node.

[0044] In one embodiment, the processing of the workflow data of the target data source based on the binding relationship comprises: when multiple task nodes of the same task relationship model have the third binding relationship with the same persistent data source, obtaining the association characteristics of the multiple task nodes; updating the persistent data source based on the association characteristics of the task nodes.

[0045] In one of the embodiments, the updating of the persistent data source based on the associated features of the task node includes: when the associated features of the task node are serial execution, the target workflow data in the persistent data source is updated in sequence based on the serial execution order of the task node; when the associated features of the task node are parallel execution, the target workflow data in the persistent data source is updated respectively based on the parallel execution order of the task node; wherein the target workflow data is the workflow data in the persistent data source associated with the output result of executing the task node.

[0046] Specifically, for the to-be-executed tasks of the task relationship model, when the task relationship model or the workflow established based on the task relationship model is bound with multiple target data sources, the association between any two target data sources is adaptively adjusted according to the association between the to-be-executed tasks of the task relationship model to which the target data sources are bound.

[0047] In one of the embodiments, the processing of the workflow data of the target data source based on the binding relationship includes: obtaining the lifecycle of the workflow; when the lifecycle of the workflow is ended, erasing the one-time data source that has the second binding relationship with the workflow.

[0048] In the data processing flow, different association forms of to-be-executed tasks can be used for different scenarios to improve the execution efficiency and the accuracy of data management. For example, when data needs to be processed in parallel, the automatic merge function of the data source can be used to execute three to-be-executed tasks at the same time, thereby speeding up the overall processing speed and ensuring that the data can be correctly converged in the end. In a typical serial scenario, the data generated by an upstream to-be-executed task is used as the input of a downstream to-be-executed task, and the data source is used for serial connection to realize the orderly flow.

[0049] Embodiment Two

[0050] Figure 2 A task execution method provided by an embodiment of the present application is shown, and the method includes: Step S201, in response to a task execution request, obtaining to-be-executed tasks and the association of any two to-be-executed tasks; Step S202, generating a task relationship model based on the association of any two to-be-executed tasks and constructing a workflow corresponding to the task relationship model; Step S203, executing the to-be-executed tasks based on the workflow and the task relationship model; Wherein, the workflow data generated in the execution of the to-be-executed tasks is processed based on the data processing method as described above.

[0051] The task execution method provided in the application is applied to the execution scheduling process of asynchronous tasks with a chain relationship, and a plurality of to-be-executed tasks are reasonably arranged according to the correlation of the to-be-executed tasks and the execution order of any two to-be-executed tasks, to generate a task relationship model; a workflow is constructed based on the task relationship model, and the execution order of the to-be-executed tasks in the task relationship model is scheduled and controlled through the workflow.

[0052] In one of the embodiments, an identifier of any to-be-executed task, the correlation between any two to-be-executed tasks and additional information of the to-be-executed tasks are acquired; a map array is generated based on the additional information of the to-be-executed tasks; for any to-be-executed task, other to-be-executed tasks having a correlation with the to-be-executed task are determined, and a two-dimensional array is generated based on the identifier of the to-be-executed task and the identifier of the other to-be-executed tasks; and the task relationship model is generated based on the map array and the two-dimensional array.

[0053] In one of the embodiments, the task execution method provided in the embodiment is applied to a task execution system to acquire the task execution state in real time; specifically, the task execution system can include a task maintenance page, a service center and a client; the task maintenance page is used for a user to maintain the related information of a task relationship model (DAG), control the start and stop states of the tasks of the task relationship model and query the task execution record, etc.; the service center is used for responding to the task relationship model maintenance information sent by the user to implement the maintenance of the task relationship model, return the task running state, the client cluster information, distribute the tasks, etc., and is the actual execution party of the tasks of the task relationship model; and the client is deployed on the server, the client actually executes the workflow, executes the tasks contained in the task relationship model in sequence according to the task relationship model, and feeds back the workflow data generated in the task execution process to the service center. Through the form of the client + task maintenance page, the execution process of the to-be-executed tasks is visualized, and the development cost and the maintenance cost of the task relationship model are greatly reduced.

[0054] In one of the embodiments, the attribute information of the tasks in the task relationship model includes the name, the label, the type, the language, the code library, the execution command and the associated target data source of the tasks, etc.

[0055] Specifically, assuming that the task relationship model is a directed acyclic graph, the task described in the embodiment corresponds to a node of the directed acyclic graph, and each node corresponds to an executable task. The task code of the task relationship model can be maintained and updated through the task maintenance page of the task execution system; and the corresponding maintenance and update rule can include hosting the code to git, and the client updates the code through the pulling mode.

[0056] In one of the embodiments, the tasks of the task relationship model are executed based on the output parameters sent by the upstream tasks as the input parameters, and the output parameters are obtained based on the input parameters and preset task execution rules and output to the next task or directly output the results. The flow of the input parameters and the output parameters is processed by the data processing method described in Embodiment 1. In short, a small amount of data is directly interacted by the input parameters and the output parameters, that is, directly flows between the tasks; a large amount of data is pulled by the input parameters and the output parameters through the data source.

[0057] Specifically, the way of obtaining data by each task can include a client method or an interface calling method.

[0058] In one of the embodiments, a two-dimensional array is generated based on the identifier of the task to be executed and the identifier of the other task to be executed, and the two-dimensional array is determined as the task relationship model, including: generating a one-dimensional array based on the identifier of the task to be executed; the parameters involved in the task such as the input parameters, the output parameters, the input data source and the output data source and other additional information are stored by a map array; the association relationship between any two tasks is stored by a two-dimensional array.

[0059] The association relationship between any two tasks is stored by a two-dimensional data, including: the first layer represents an edge, and each layer has two elements; for example: [[0, 1][0, 2][1, 3][2, 3]] represents that the downstream tasks of the 0th task are the 1st and 2nd tasks, and the downstream tasks of the 1st and 2nd tasks are the 3rd task.

[0060] In one of the embodiments, based on the workflow and the task relationship model, the task to be executed is executed, including: initializing the state of the task to be executed of the task relationship model; when the workflow starts to execute, the task to be executed is traversed, if all the upstream tasks to be executed are executed, the task to be executed is executed; when there is no executable task to be executed, the task relationship model is stopped to be traversed, and the execution state of the task relationship model is saved.

[0061] Based on the workflow and the task relationship model, the task to be executed is executed, including: generating an execution log corresponding to the task relationship model, the execution log including task to be executed error, workflow error, error handling process, error handling state and error handling result.

[0062] Embodiment three

[0063] Figure 3 A data processing device provided by the embodiments of the present application is shown, and the device at least includes the following units: an acquisition unit, configured to acquire a pre-generated task relationship model and a workflow constructed based on the task relationship model, and determine workflow data based on the workflow; a generation unit, configured to generate a target data source based on a data feature of the workflow data; an establishment unit, configured to establish a binding relationship between an attribute feature of the target data source and the task relationship model and the workflow respectively; a processing unit, configured to process the workflow data of the target data source based on the binding relationship.

[0064] Embodiment Four

[0065] Figure 4 A task execution device is shown, which includes at least the following units: a generation unit, configured to generate a target data source based on a data feature of the workflow data; an establishment unit, configured to establish a binding relationship between an attribute feature of the target data source and the task relationship model and the workflow respectively; a processing unit, configured to process the workflow data of the target data source based on the binding relationship.

[0066] In another aspect of the present application, a task execution device is provided, which includes: an acquisition unit, configured to acquire a task to be executed and the correlation between any two tasks to be executed in response to a task execution request; a construction unit, configured to generate a task relationship model based on the correlation between the any two tasks to be executed and construct a workflow corresponding to the task relationship model; an execution unit, configured to execute the task to be executed based on the workflow and the task relationship model; wherein the workflow data involved in the process of executing the task to be executed is processed based on the data processing device as described above.

[0067] Embodiment Five

[0068] Based on the same inventive concept, the electronic device provided in the embodiments of the present application can correspond to the method for workflow data execution and the method for task execution in the foregoing embodiments, and the problem solving principle thereof is similar to that of the method. The electronic device provided in the embodiments of the present application includes at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method and / or technical solution of the plurality of embodiments of the present application.

[0069] The electronic device can be a user device, or a device integrated by a user device and a network device through a network, or also can be an application program running on the above device, the user device includes but is not limited to computers, mobile phones, tablets, smart watches, bracelets and various terminal devices, and the network device includes but is not limited to network hosts, single network servers, multiple network server sets or cloud computing-based computer sets, which can be used to realize part of the processing function when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, wherein cloud computing is a kind of distributed computing, which is composed of a virtual computer set by a group of loosely coupled computers.

[0070] Figure 5 The structure of an electronic device suitable for implementing the method and / or technical scheme in the embodiments of the present application is shown, the device 500 includes a central processing unit (CPU, Central Processing Unit) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, ReadOnlyMemory) 502 or the program loaded from the storage part 508 to the random access memory (RAM, RandomAccessMemory) 503. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, the ROM 502 and the RAM 503 are connected to each other through the bus 504. The input / output (I / O, Input / Output) interface 505 is also connected to the bus 504.

[0071] The following components are connected to the I / O interface 505: the input part 506 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; the output part 507 including a cathode ray tube (CRT, Cathode Ray Tube), a liquid crystal display (LCD, Liquid Crystal Display), an LED display, an OLED display, etc., and a speaker, etc.; the storage part 508 including one or more computer readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and the communication part 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part 509 performs communication processing via a network such as the Internet.

[0072] In particular, the methods and / or embodiments of this application can be implemented as a computer program product. For example, embodiments disclosed herein include a computer program product comprising a computer program tangibly embodied in a computer readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. When the computer program is executed by a central processing unit (CPU) 501, the aforementioned functions defined in the methods of this application are performed.

[0073] Embodiment Six

[0074] Another embodiment of this application provides a computer readable storage medium having stored thereon computer program instructions which can be executed by a processor to implement the method and / or techniques of any one or more embodiments of the present application.

[0075] In particular, the embodiments can take the form of one or more computer program products. The computer program product can be embodied in any

[0076] The computer readable medium can include, but is not limited to, portable computer diskette, compact flash, RAM, ROM, EEPROM, CD-ROM, optical storage device, magnetic storage device, or any suitable combination of the foregoing. In the various

[0077] The program code can be transmitted as a computer readable signal on a computer readable medium, including, but not limited to, wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0078] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0079] The flow diagrams and block diagrams in the drawings are representative of the architectural, functional, and operational aspects of possible implementations of apparatuses, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0080] Those skilled in the art can clearly understand that, for the convenience and brevity, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0081] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or page components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0082] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0083] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0084] The integrated unit realized in the form of software functional unit can be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, and various program code storage media.

[0085] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

[0086] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the device claim can also be realized by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A data processing method, characterized by, The method comprises: acquiring a pre-generated task relationship model and a workflow constructed based on the task relationship model, and determining workflow data based on the workflow; generating a target data source based on data characteristics of the workflow data; establishing a binding relationship between attribute characteristics of the target data source and the task relationship model and the workflow, respectively; processing the workflow data of the target data source based on the binding relationship.

2. The data processing method according to claim 1, characterized in that, The generation of the target data source based on the data characteristics of the workflow data comprises: acquiring business data of the workflow data; generating the target data source based on the business data.

3. The data processing method according to claim 2, characterized in that, The establishment of the binding relationship between the attribute characteristics of the target data source and the task relationship model and the workflow comprises: acquiring the timeliness of the target data source; when the timeliness is high timeliness, determining that the target data source is a persistent data source, acquiring a task relationship model corresponding to the persistent data source, and establishing a first binding relationship between the persistent data source and the task relationship model; when the timeliness is low timeliness, determining that the target data source is a one-time data source, acquiring a workflow corresponding to the one-time data source, and establishing a second binding relationship between the one-time data source and the workflow.

4. The data processing method according to claim 3, characterized in that, The processing of the workflow data of the target data source based on the binding relationship comprises: acquiring the number of task relationship models that have established the first binding relationship with the persistent data source, when the number of task relationship models exceeds one, generating a first identifier corresponding to each task relationship model, and isolating the persistent data source based on the first identifier; acquiring the number of workflows that have established the second binding relationship with the one-time data source, when the number of workflows exceeds one, generating a second identifier corresponding to each workflow, and isolating the one-time data source based on the second identifier.

5. The data processing method according to claim 4, characterized in that, The processing of the workflow data of the target data source based on the binding relationship further comprises: based on each task relationship model, acquiring the number of persistent data sources that have established the first binding relationship with the task relationship model; when the number of persistent data sources exceeds one, acquiring a task node of the task relationship model; based on the task node, determining the corresponding persistent data source, and establishing a third binding relationship between the persistent data source and the task node.

6. The data processing method according to claim 5, characterized in that, The processing of the workflow data of the target data source based on the binding relationship comprises: when multiple task nodes of the same task relationship model have established the third binding relationship with the same persistent data source, acquiring the association characteristics of the multiple task nodes; updating the persistent data source based on the association characteristics of the task node.

7. The data processing method according to claim 6, characterized in that, The updating of the persistent data source based on the association characteristics of the task node comprises: when the association characteristics of the task node are series execution, then the target workflow data in the persistent data source is updated in sequence based on the series execution order of the task node; When the association feature of the task node is parallel execution, then the target workflow data in the persistent data source is updated based on the parallel execution order of the task node respectively; The target workflow data is workflow data in the persistent data source associated with the output result of executing the task node.

8. The data processing method according to claim 7, characterized in that, The processing of the workflow data of the target data source based on the binding relationship comprises: acquiring the lifecycle of the workflow; when the lifecycle of the workflow is ended, erasing the one-time data source having the second binding relationship with the workflow.

9. A task execution method characterized by, The method comprises: in response to a task execution request, acquiring a to-be-executed task and the association of any two to-be-executed tasks; generating a task relationship model based on the association of the any two to-be-executed tasks and constructing a workflow corresponding to the task relationship model; executing the to-be-executed task based on the workflow and the task relationship model; wherein the workflow data generated in the execution of the to-be-executed task is processed based on the data processing method of any one of claims 1-8.

10. The task execution method according to claim 9, wherein The generating of the task relationship model based on the association of the any two to-be-executed tasks and the constructing of the workflow corresponding to the task relationship model comprises: acquiring an identifier of any to-be-executed task, the association between any two to-be-executed tasks and additional information of the to-be-executed task; generating a map array based on the additional information of the to-be-executed task; determining other to-be-executed tasks having an association with any to-be-executed task, and generating a two-dimensional array based on the identifier of the to-be-executed task and the identifier of the other to-be-executed task respectively; generating the task relationship model based on the map array and the two-dimensional array.

11. A data processing device, characterized by The device comprises: an acquisition unit configured to acquire a pre-generated task relationship model and a workflow constructed based on the task relationship model, and determine workflow data based on the workflow; a generation unit configured to generate a target data source based on the data features of the workflow data; an establishment unit configured to establish a binding relationship between the target data source and the task relationship model and the workflow respectively based on the attribute features of the target data source; a processing unit configured to process the workflow data of the target data source based on the binding relationship.

12. A task execution device characterized by comprising: The device comprises: an acquisition unit configured to acquire a to-be-executed task and the association of any two to-be-executed tasks in response to a task execution request; a construction unit configured to generate a task relationship model based on the association of the any two to-be-executed tasks and construct a workflow corresponding to the task relationship model; an execution unit configured to execute the to-be-executed task based on the workflow and the task relationship model; wherein the workflow data generated in the execution of the to-be-executed task is processed based on the data processing device of claim 11.

13. An electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; characterized in that, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 10.

14. A computer readable medium having stored thereon computer program instructions, wherein, The computer program instructions are executable by a processor to implement the method of any one of claims 1 to 10.

15. A computer program product comprising a computer program, characterized in that, The computer program is implemented by a processor to implement the method of any one of claims 1 to 10.

Images