A data processing method, device, computer, storage medium and program product
By breaking down the orchestration and execution of business data into component-based processing, and using component description information to obtain and execute node business, the maintenance complexity problem of Airflow under non-Python stacks is solved, achieving flexible cross-language adaptation and low maintenance complexity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TENCENT TECHNOLOGY (SHENZHEN) CO LTD
- Filing Date
- 2024-12-06
- Publication Date
- 2026-06-09
AI Technical Summary
Existing orchestration engines such as Airflow primarily use object-oriented programming languages for business process orchestration, resulting in high maintenance complexity in non-Python technology stacks or user-managed scenarios, making them unsuitable for diverse enterprise business processes.
By breaking down the orchestration and execution of business data into component-based processes, the business processing node information is obtained using component description information, and the node business is executed through the target business execution component to generate node execution results, and finally the business processing result is determined.
It improves the flexibility and versatility of business data processing, reduces the maintenance complexity of the orchestration engine, and can be applied to any business scenario without the need to introduce various programming languages.
Smart Images

Figure CN122175172A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer, storage medium and program product. Background Technology
[0002] As businesses expand and become more complex, the business processes they need to complete become more diversified. Relying solely on manual processes is not only inefficient but also costly. Businesses can introduce orchestration engines to automate the processing of their business processes (or business data), thereby improving system efficiency. Various orchestration engines are available on the market, with Airflow being a commonly used one. However, Airflow primarily orchestrates business processes using an object-oriented programming language (Python). All logic in this approach is implemented through code, which means that in some business scenarios, such as when the technology stack uses other programming languages or when users orchestrate business processes themselves, Airflow may not function properly or may require the introduction of other programming languages to re-orchestrate the business processes, thus increasing the maintenance complexity of the orchestration engine. Summary of the Invention
[0003] This application provides a data processing method, apparatus, computer, storage medium, and program product. By modularizing the orchestration engine business execution system, the orchestration processing and execution of business data can be separated, making the processing of business data applicable to any business scenario, thereby improving the flexibility and universality of business data processing.
[0004] One embodiment of this application provides a data processing method, the method comprising:
[0005] According to the business processing logic of the business data, obtain the information of N business processing nodes corresponding to the business data; the information of N business processing nodes is used to represent the processing logic order for the business data; each piece of information of business processing nodes includes component description information that indicates the target business execution component associated with that business processing node; N is a positive integer;
[0006] Based on the target business execution component associated with N business processing node information, the node business indicated by each of the N business processing node information is executed sequentially to obtain the node execution results corresponding to each of the N business processing node information.
[0007] Based on the execution results of N nodes, determine the business processing result for the business data.
[0008] One embodiment of this application provides a data processing apparatus, the apparatus comprising:
[0009] The information acquisition module is used to acquire N business processing node information corresponding to the business data according to the business processing logic of the business data; the N business processing node information is used to represent the processing logic order for the business data; each business processing node information includes component description information that indicates the target business execution component associated with the business processing node information; N is a positive integer;
[0010] The node service execution module is used to execute the node services indicated by the N service processing node information in sequence, based on the target service execution components associated with the N service processing node information, and to obtain the node execution results corresponding to the N service processing node information respectively.
[0011] The results generation module is used to determine the business processing results for business data based on the execution results of N nodes.
[0012] In one possible implementation, the N business processing node information includes business processing node information A. i , where i is a positive integer less than or equal to N; the node service execution module is used to execute the node services indicated by the N service processing node information in sequence based on the target service execution components associated with the N service processing node information, and to obtain the node execution results corresponding to the N service processing node information respectively. The data decomposition module is specifically used to perform the following operations:
[0013] Based on business processing node information A i The component description information is used to determine the business processing node information A from the component service library. i The associated target business execution component;
[0014] Obtain business processing node information A i The associated node has pending execution data. Based on the node's pending execution data, the business processing node information A is called. i The associated target business execution component, through business processing node information A i The associated target business execution component processes the node business indicated by the data to be executed on the node and obtains the component execution information;
[0015] Based on the component execution information, generate business processing node information A. i The execution result of the corresponding node.
[0016] In one possible implementation, the node business execution module is used to generate business processing node information A based on the component execution information. i When the corresponding node executes its result, the node's business execution module is specifically used to perform the following operations:
[0017] If the component execution information does not include business sub-processes, then the component execution information will be identified as business processing node information A. i The execution result of the corresponding node;
[0018] If the component execution information includes business sub-processes, then a sub-process execution object is generated for the business sub-process through the business control system. Based on the sub-process execution object, the sub-process execution result for the business sub-process is obtained. Based on the sub-process execution result, business processing node information A is generated. i The corresponding node execution result; the business control system is used to process information from N business processing nodes.
[0019] In one possible implementation, when the node business execution module obtains the sub-process execution result for a business sub-process based on the sub-process execution object, the node business execution module is specifically used to perform the following operations:
[0020] According to the sub-process execution logic indicated by the sub-process execution object, obtain the information of M sub-business nodes corresponding to the business sub-process from the sub-process execution object; M is a positive integer;
[0021] Based on the component description information in the M sub-business node information, obtain the target business execution components associated with the M sub-business node information respectively. Based on the target business execution components associated with the M sub-business node information respectively, execute the node sub-business indicated by the M sub-business node information respectively, and obtain the sub-process execution results corresponding to the M sub-business node information respectively.
[0022] In one possible implementation, the node business execution module is also used to perform the following operations:
[0023] If the business processing node information A is used i The associated target business execution component obtains information A regarding the business processing node. i If the business execution identifier is used, then based on the business execution identifier, the business processing node information A is invoked. i The query interface of the associated target business execution component is used to obtain business processing node information A. i The associated target business execution component provides execution status and result information for the node's pending data.
[0024] When the obtained execution status information for the data to be executed at a node is execution completed, the execution result information corresponding to the execution completed information is determined as business processing node information A. i The execution result of the corresponding node.
[0025] In one possible implementation, the business processing node information A iIt exists in the node execution sequence; the node business execution module is also used to perform the following operations:
[0026] Obtain the component execution cycle. If no business processing node information A is obtained within the component execution cycle... i The associated target business execution component returns component execution information, or the business processing node information A is obtained. i If the execution information returned by the associated target business execution component indicates an execution failure, then the business processing node information A in the node execution queue will be... i The execution status information is updated to execution failure information;
[0027] Regarding business processing node information A i Create new business processing node information A i If, based on business processing logic, information A that depends on the business processing node is obtained... i Based on the first business processing node information, the new business processing node information A will be generated. i Add to the node execution sequence; New business processing node information A i In the node execution sequence, at any position preceding the first business processing node information; new business processing node information A i The execution status information is "waiting to be executed"; business processing node information A i And new business processing node information A i The node identifiers are the same.
[0028] In one possible implementation, the node business execution module is also used to perform the following operations:
[0029] Based on the business processing logic, determine the business processing node information A. i Information about the second business processing node with data dependencies exists; N business processing node information includes the information about the second business processing node, and for business processing node information A... i The execution of node services depends on the execution results of the node that processes the information of the second service processing node.
[0030] If the execution status information of the second business processing information is detected as "execution completed", then the execution based on business processing node information A will proceed. i The component description information is used to determine the business processing node information A from the component service library. i The process of the associated target business execution component.
[0031] In one possible implementation, the data processing device further includes a component registration module, which is specifically used to perform the following operations:
[0032] In response to a component registration request for the first business execution component, the first component association information of the first business execution component is obtained through the component registration request; the first component association information is used to represent the functions and attributes of the first business execution component;
[0033] The association information of the first component is detected to obtain the component detection results for the first business execution component;
[0034] If the component detection result indicates that the first business execution component is valid, then the association information of the first component will be stored in the component service library.
[0035] In one possible implementation, the component registration module is used to respond to a component registration request for the first business execution component. When obtaining the first component association information of the first business execution component through the component registration request, the component registration module is specifically used to perform the following operations:
[0036] The system retrieves the component name and functional information carried in the component registration request. Based on the component name or functional information, it searches among registered components. If a second business execution component is found among the registered components, the component identifier in the second component association information of the second business execution component is determined as the component identifier of the first business execution component. The component identifier of the first business execution component, the component name, component service address, component version, and component functional information carried in the component registration request are determined as the first component association information of the first business execution component. The component service address is used to call the first business execution component. The second business execution component has the same component name or component functional information as the first business execution component.
[0037] Alternatively, a random identifier is generated for the target component, and the random identifier is determined as the component identifier of the first business execution component. The component identifier of the first business execution component, the component name, component service address, component version, and component function information carried in the component registration request are determined as the first component association information of the first business execution component. The component identifier of the first business execution component is different from the component identifier of any business execution component in the component service library.
[0038] In one possible implementation, the business data is code instruction data; when the information acquisition module obtains the information of N business processing nodes corresponding to the business data according to the business processing logic of the business data, the information acquisition module is specifically used to perform the following operations:
[0039] The code instruction data is parsed to determine the N business nodes indicated by the code instruction data, as well as the processing logic order between the N business nodes;
[0040] Obtain the node identifier, node business, and component description information corresponding to N business nodes from the code instruction data. Encapsulate the node identifier, node business, and component description information corresponding to each business node to obtain N node description information.
[0041] Based on the description information of N nodes and the processing logic order between N business nodes, determine the information of N business processing nodes corresponding to the business data.
[0042] In one possible implementation, the business data is a directed graph of business data, which includes N business nodes. When the information acquisition module retrieves information about the N business processing nodes corresponding to the business data according to the business processing logic, the information acquisition module specifically performs the following operations:
[0043] By traversing the directed graph data of the business, we can sequentially obtain business nodes without parent nodes or business nodes whose parent nodes are already traversed in the directed acyclic graph data, obtain the processing logic order between N business nodes, and obtain the node description information of each business node.
[0044] Based on the node description information of N business nodes and the processing logic order between N business nodes, N business processing node information corresponding to the business data is generated; one business processing node information corresponds to one node description information and one business node; the node description information includes component description information, node identifier, and node business used to indicate the target business execution component associated with the business processing node information.
[0045] One embodiment of this application provides a computer device, including a processor, a memory, and an input / output interface;
[0046] The processor is connected to a memory and an input / output interface, respectively. The input / output interface is used to receive and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device containing the processor executes the method in one aspect of the embodiments of this application.
[0047] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method of one aspect of this application.
[0048] One aspect of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional embodiments of this application. In other words, when the computer instructions are executed by the processor, they implement the methods provided in various optional embodiments of this application.
[0049] Implementing the embodiments of this application will have the following beneficial effects:
[0050] In this embodiment, according to the business processing logic of the business data, N business processing node information corresponding to the business data is obtained; the N business processing node information is used to represent the processing logic order for the business data; each business processing node information includes component description information for indicating the target business execution component associated with the business processing node information; N is a positive integer; based on the target business execution component associated with the N business processing node information, the node business indicated by the N business processing node information is executed sequentially to obtain the node execution results corresponding to the N business processing node information; based on the N node execution results, the business processing result for the business data is determined. Through the above process, the orchestration, processing, and execution of business data are broken down. Executors implementing different functions are componentized (i.e., business execution components). Business data is broken down into steps, resulting in N business processing node information. This allows computer devices to directly call the business execution component with the capability to process each business processing node, executing the corresponding node's business logic. Since business processing node information is not program code, its processing does not involve the programming language used by the program code. This allows for orchestration and execution of business data regardless of whether the business data uses any programming language or other user-manufactured business processes. In other words, it can be applied to any business scenario, thus improving the universality and flexibility of business data processing. Furthermore, the processing of business data does not require the introduction of various programming languages, thereby reducing the maintenance complexity of the orchestration engine. Attached Figure Description
[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0052] Figure 1 This is a network interaction architecture diagram provided in an embodiment of this application;
[0053] Figure 2 This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 1 ;
[0054] Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 ;
[0055] Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 ;
[0056] Figure 5 This is a schematic diagram of a directed graph data traversal provided in an embodiment of this application;
[0057] Figure 6 This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 2 ;
[0058] Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 3 ;
[0059] Figure 8 This is a schematic diagram of a data processing device provided in an embodiment of this application;
[0060] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0062] If this application requires the collection of object data (such as user data), a prompt interface or pop-up window will be displayed before and during the collection process. This prompt interface or pop-up window is used to inform the user that certain data is being collected. The data acquisition steps will only begin after the user confirms the prompt interface or pop-up window; otherwise, the process will end. Furthermore, the acquired user data will be used in reasonable and legal scenarios or for legitimate purposes. Optionally, in scenarios where user data needs to be used but user authorization has not been obtained, authorization can be requested from the user, and the user data can only be used after authorization is granted.
[0063] It is understood that, in the specific embodiments of this application, the user data involved requires user permission or consent when the following embodiments of this application are applied to specific products or technologies, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant regions.
[0064] In the embodiments of this application, please refer to Figure 1 , Figure 1 This is a network interaction architecture diagram provided in an embodiment of this application, such as... Figure 1 As shown in the diagram, the network interaction architecture may include computer device 101 and a cluster of service devices. The cluster of service devices may include service devices 102a, 102b, 102c, ..., 102n. Communication connections may exist between service devices in the cluster; for example, there may be a communication connection between service devices 102a and 102b, or between service devices 102a and 102c. Furthermore, any service device in the cluster may have a communication connection with computer device 101; for example, there may be a communication connection between service device 102a and computer device 101. The communication connection method is not limited; it can be established directly or indirectly through wired communication, wireless communication, or other methods. This application does not impose any restrictions on these methods.
[0065] It should be understood that, such as Figure 1 Each service device in the service device cluster shown can have an application client installed. When the application client runs on each service device, it can interact with the aforementioned... Figure 1The computer devices 101 shown interact with each other, enabling them to receive business data from each business device. The application client can be a financial application, social application, instant messaging application, live streaming application, short video application, video application, music application, shopping application, novel application, payment application, browser, or other application client capable of displaying text, images, audio, and video data. This application client can be a standalone client or an embedded sub-client integrated into another client (e.g., an instant messaging client, social client, video client, etc.), without limitation. Business devices can send business data to the computer device through the application client. The computer device can obtain the target business execution component corresponding to each of the N business processing node information in the business data, and then execute the N business processing node information sequentially through the target business execution component to obtain the node execution results corresponding to each of the N business processing node information. Based on the N node execution results, the business processing result for the business data is determined. Furthermore, the computer device can send the business processing result to the business device.
[0066] It is understood that the process of transforming business data into business processing results using computer equipment can be executed by the orchestration engine client implementing the data processing method of this application. The data processing method of this application is the process mentioned above of obtaining the business processing results of business data by acquiring the target business execution component, executing the corresponding business processing node information, and thus obtaining the business processing results. Optionally, the orchestration engine implementing the data processing method of this application can be directly integrated into the application client of the business equipment, thereby eliminating the need to send business data to the computer equipment for processing. Business data can be processed directly on the business equipment using the orchestration engine to obtain the business processing results.
[0067] Through the above process, the orchestration, processing, and execution of business data are broken down. Executors implementing different functions are componentized (i.e., business execution components). Business data is broken down into steps, resulting in N business processing node information. This allows computer devices to directly call the business execution component with the capability to process each business processing node, executing the corresponding node's business logic. Since business processing node information is not program code, its processing does not involve the programming language used by the program code. This allows for orchestration and execution of business data regardless of whether the business data uses any programming language or other user-manufactured business processes. In other words, it can be applied to any business scenario, thus improving the universality and flexibility of business data processing. Furthermore, the processing of business data does not require the introduction of various programming languages, thereby reducing the maintenance complexity of the orchestration engine.
[0068] For details, please see Figure 2 , Figure 2 This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 1 .like Figure 2 As shown, taking a computer device integrating an orchestration engine that implements the data processing method of this application and executing the data processing method of this application as an example, business data 201 is a business directed graph (DAG) containing four business nodes, namely business node A, business node B, business node C, and business node D. Business nodes B and C depend on business node A, and business node D depends on business node C. The computer device can obtain N business processing node information 202 corresponding to business data 201 according to the business processing logic in business data 201, including business processing node information A, business processing node information B, business processing node information C, and business processing node information D. Each business processing node information includes component description information indicating the target business execution component associated with that business processing node information.
[0069] The computer device can determine the target service execution component 203 associated with each of the N service processing node information 202 based on the component description information of the target service execution component associated with each of the N service processing node information 202. For example... Figure 2 As shown in the target business execution component 203, business processing node information A is associated with target business execution component a, business processing node information B is associated with target business execution component b, business processing node information C is associated with target business execution component c, and business processing node information D is associated with target business execution component d.
[0070] Furthermore, the computer device can sequentially execute the node services indicated by each of the N service processing node information 202 based on the target service execution component 203 associated with the N service processing node information 202, thereby obtaining the node execution results 204 corresponding to each of the N service processing node information 202. For example, target service execution component a executes the node service in service processing node information A, obtaining node execution result 1; target service execution component b executes the node service in service processing node information B, obtaining node execution result 2; target service execution component c executes the node service in service processing node information C, obtaining node execution result 3; and target service execution component d executes the node service in service processing node information D, obtaining node execution result 4.
[0071] It is understandable that, since business nodes B and C depend on business node A, the generation of node execution result 2 and node execution result 3 can both depend on node execution result 1. Business node D depends on business node C, so the generation of node execution result 4 depends on node execution result 1. Furthermore, when the terminal device executes business processing node information B and business processing node information C respectively through target business execution component b and target business execution component c, it needs to determine whether the terminal device has completed the execution of business processing node information A. That is, the terminal device can only begin executing business processing node information B and business processing node information C after obtaining node execution result 1. And since there is no dependency between business nodes B and C, the terminal device can execute business processing node information B and business processing node information C asynchronously. That is, the terminal device can directly begin executing business processing node information C before completing the execution of business processing node information B and obtaining node execution result 2, or before starting the execution of business processing node information B.
[0072] After obtaining the node execution results 203 corresponding to N business processing node information 202, the computer equipment can determine the business processing result for business data 201 based on node execution result 1, node execution result 2, node execution result 3, and node execution result 4. It can be understood that node execution result 2 and node execution result 3 depend on node execution result 1 for generation, and node execution result 4 depends on node execution result 3 for generation. Therefore, the business processing result corresponding to business data 201 can actually be understood as being generated by node execution result 2 and node execution result 4.
[0073] Through the above process, the orchestration, processing, and execution of business data are broken down. Executors implementing different functions are componentized (i.e., business execution components). Business data is broken down into steps, resulting in N business processing node information. This allows computer devices to directly call the business execution component with the capability to process each business processing node, executing the corresponding node's business logic. Since business processing node information is not program code, its processing does not involve the programming language used by the program code. This allows for orchestration and execution of business data regardless of whether the business data uses any programming language or other user-manufactured business processes. In other words, it can be applied to any business scenario, thus improving the universality and flexibility of business data processing. Furthermore, the processing of business data does not require the introduction of various programming languages, thereby reducing the maintenance complexity of the orchestration engine.
[0074] It is understood that the computer equipment or business equipment mentioned in the embodiments of this application includes, but is not limited to, terminal equipment or servers. In other words, computer equipment can be a server or a terminal device, or a system composed of a server and a terminal device. The terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablets, desktop computers, laptops, handheld computers, in-vehicle devices, augmented reality / virtual reality (AR / VR) devices, head-mounted displays, smart TVs, wearable devices, smart speakers, digital cameras, webcams, and other mobile internet devices (MIDs) with network access capabilities, or terminal devices in scenarios such as trains, ships, and flights. Figure 1 As shown, the terminal device can be a desktop computer (as shown in service device 102b), a laptop computer (as shown in service device 102n), a mobile phone (as shown in service device 102a), or a tablet computer (as shown in service device 102c), etc. Figure 1 Only a portion of the equipment is listed. The servers mentioned above can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-to-everything (V2X) communication, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0075] Optionally, the data involved in the embodiments of this application may be stored in a computer device, or may be stored based on cloud storage technology or a blockchain network, without limitation.
[0076] Further, please see Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 This data processing method can be executed by a computer device. The following description will use the execution of this data processing method by a computer device as an example, where the computer device integrates an orchestration engine that implements this data processing method. This data processing method may include at least the following steps S301-S303:
[0077] Step S301: According to the business processing logic of the business data, obtain the information of N business processing nodes corresponding to the business data; the information of N business processing nodes is used to represent the processing logic order for the business data; each information of business processing nodes includes component description information used to indicate the target business execution component associated with the business processing node information; N is a positive integer.
[0078] In this embodiment, the computer device can acquire business data sent by the user. This business data can be user-coded code instructions or directed graph data (e.g., a Merkle-directed acyclic graph, DAG). The orchestration engine integrated in the computer device can provide a visual interface for orchestrating business data. Users can orchestrate and configure node execution information using nodes and directed lines to obtain the directed graph data. Alternatively, the orchestration engine may have a built-in business process template library. Users can submit a business requirement, and the computer device can retrieve the corresponding business process template from the library and identify it as the business data. Optionally, if the business process template does not perfectly fulfill the business requirement, the computer device can modify it to obtain the final business data. Alternatively, after acquiring the business process template, the computer device can display it to the user, who can then modify it to obtain the final business data.
[0079] Furthermore, the computer device can use an orchestration engine to obtain information about N business processing nodes corresponding to the business data, according to the business processing logic. Specifically, the computer device can determine from the business data the N business nodes included in the business data, the node description information corresponding to the N business nodes, and the processing logic order between the N business nodes. The node description information includes component description information, node identifier, and node business, indicating the target business execution component associated with the business processing node information. The component description information generally includes, but is not limited to, the component identifier, component name, and component version of the target business execution component. The computer device can use this component description information to find the target execution component associated with the business processing node information, and thus execute the node business in the business processing node information through the target execution component. The computer device can generate information about N business processing nodes corresponding to the business data based on the node description information of the N business nodes and the logical processing order between the N business nodes. Each piece of information about business processing nodes includes component description information indicating the target business execution component associated with the business processing node information; N is a positive integer.
[0080] For example, if the business data is "Node A → Node B → Node C", and the node description information in Node A is "Node A, Loading Data, Component Description Information A", where the component description information A may include, but is not limited to, "Component A Identifier, Component A Name, Component A Version", then the business processing node information corresponding to Node A can be used to indicate the execution of node business (loading data) through component A.
[0081] Step S302: Based on the target business execution component associated with the N business processing node information, execute the node business indicated by the N business processing node information in sequence to obtain the node execution results corresponding to the N business processing node information.
[0082] In this embodiment, a computer device can determine the target service execution component associated with each of the N service processing node information from the component service library using the component description information in the N service processing node information. Specifically, taking one service processing node information as an example, the computer device can use the component description information in this service processing node information, such as "13122, Component A, 1.20", where "13122" indicates the component identifier (kind field) of the target service execution component associated with this service processing node information, "Plug-in A" indicates the component name of the target service execution component associated with this service processing node information, and "1.20" indicates the component version of the target service execution component associated with this service processing node information. The computer device can then determine from the component service library that the target service execution component to be used is Plug-in A version 1.20 under the kind field "13122".
[0083] Furthermore, the computer device can acquire the node execution data associated with each of the N business processing node information, and call the target business execution components associated with each of the N business processing node information based on the N node execution data. For example, business processing node information 1a is associated with node execution data 1b and target business execution component 1c; business processing node information 2a is associated with node execution data 2b and target business execution component 2c; ...; business processing node information Na is associated with node execution data Nb and target business execution component Nc. Then the computer device can call target business execution component 1c based on node execution data 1b; it can call target business execution component 2c based on node execution data 2b; and it can call target business execution component Nc based on node execution data Nb.
[0084] Furthermore, the computer device can sequentially process the node services indicated by the N business processing node information through the target business execution components associated with the N business processing node information, obtaining the component execution information corresponding to each of the N business processing node information. It can then generate the node execution results corresponding to each of the N business processing node information based on the N component execution information. For example, the node execution result corresponding to business processing node information 1a can be generated using the component execution information corresponding to business processing node information 1a; the node execution result corresponding to business processing node information 2a can be generated using the component execution information corresponding to business processing node information 2a; and so on, until the node execution results corresponding to each of the N business processing node information are obtained. It is understandable that the computer device can store the node execution results corresponding to each of the N business processing node information in a database. For example, the computer device can store the node execution result 1 corresponding to business processing node information 1 in the database. When the computer device executes business processing node information 2 that depends on node execution result 1, it can retrieve node execution result 1 from the database.
[0085] Step S303: Based on the execution results of N nodes, determine the business processing result for the business data.
[0086] In this embodiment, the computer device can determine the business processing result for business data based on the execution results of N nodes. Specifically, if the business data is a unidirectional business process, for example, the business data is "node A → node B → node C", and the execution result of node A is used as input data for node B to obtain the execution result of node B, then the computer device can determine the execution result of node C as the business processing result of the business data.
[0087] Optionally, if the business data is a non-unidirectional business process, for example, where K out of N node execution results serve as input data for business nodes that are dependent on these K node execution results, and are used to generate the node execution results of these dependent business nodes, then the computer device can generate a business processing result for the business data based on the NK node execution results that have not been further processed or executed by the other business nodes out of the N node execution results. Here, K is a positive integer less than N, and the execution of the node business in the business processing node information corresponding to the business node that is dependent on the business node depends on the node execution result of the corresponding business node. For example, if the business data includes N business nodes A, B, C, and D, where the node execution result of node A serves as input data for nodes B and C to generate the node execution results of nodes B and C respectively, and the node execution result of node B serves as input data for node D to generate the node execution result of node D, then the business processing result of the business data can be generated by the node execution results of nodes C and D.
[0088] Through the above process, the orchestration, processing, and execution of business data are broken down. Executors implementing different functions are componentized (i.e., business execution components). Business data is broken down into steps, resulting in N business processing node information. This allows computer devices to directly call the business execution component with the capability to process each business processing node, executing the corresponding node's business logic. Since business processing node information is not program code, its processing does not involve the programming language used by the program code. This allows for orchestration and execution of business data regardless of whether the business data uses any programming language or other user-manufactured business processes. In other words, it can be applied to any business scenario, thus improving the universality and flexibility of business data processing. Furthermore, the processing of business data does not require the introduction of various programming languages, thereby reducing the maintenance complexity of the orchestration engine.
[0089] In simple terms, a computer device may include a component service library and a business control system. The component service library maintains business execution components, or maintains component association information for business execution components, as described in step S302. The business control system maintains business data, orchestrates and processes the business data, breaks it down into steps to obtain multiple business processing node information, and schedules and processes each business processing node, as described in steps S301 to S303. The component service library and the business control system together form an orchestration engine. The business execution components can be considered as plug-ins, integrated into the orchestration engine as plug-ins. A plug-in (also known as an addin, addon, or add-on) is a program written according to a specific application programming interface (API). Any computer device that needs to orchestrate business data can deploy the orchestration engine described in this application to achieve the orchestration, processing, and execution of the business data. This application employs an orchestration engine that separates the orchestration processing and execution of business data, enabling it to orchestrate, process, and execute business data in any business scenario, thereby improving the flexibility and versatility of business data processing.
[0090] Further, please see Figure 4 , Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 This data processing method can be executed by a computer device, which can be, for example,... Figure 1 Any terminal device in the terminal device cluster shown. The following description assumes that this data processing method is executed by a computer device, and that the computer device integrates an orchestration engine that implements this data processing method. This data processing method may include at least the following steps S401-S409:
[0091] Step S401: Register the business execution component.
[0092] In this embodiment, the computer device can respond to a component registration request for the first business execution component and obtain the first component association information of the first business execution component through the component registration request. The component registration request can be initiated by a developer when building the orchestration engine of this data processing method, or by a user when using the orchestration engine. The first component association information represents the function and attributes of the first business execution component, where the attributes of the first business execution component refer to basic parameter information such as the component name, component version, and component service address.
[0093] Specifically, the computer device can obtain the component name and component function information carried in the component registration request. Based on the component name or component function information, it searches among registered components, that is, it searches the component service library for business execution components with component names or component function information in the component association information. If a second business execution component is found among the registered components, the component identifier in the second component association information of the second business execution component is determined as the component identifier of the first business execution component. The second business execution component has the same component name or component function information as the first business execution component. That is, when there is component association information for a second business execution component similar to the first business execution component in the component service library, the component identifier of the first business execution component can be set to be the same as the component identifier of the second business execution component. The first and second business execution components can be understood as business execution components of the same family. "Business execution components of the same family" indicates that there are multiple business execution components with the same component name or component function information, and the computer device configures the component identifiers of multiple registered business execution components to be the same. A registered component can be considered as a business execution component that has already been registered to the component service library.
[0094] For example, the first component association information of the first business execution component includes the component name (e.g., component a), component version (e.g., 1.30), component function information (execution data cleaning), and component service address. The component association information 1 of the business execution component in the component service library contains the component identifier 12342, the component name is component a, and the component function information is data cleaning; the component association information 2 contains the component identifier 12342, the component name is component b, and the component function information is data cleaning; the component association information 3 contains the component identifier 14542, the component name is component c, and the component function information is data loading. Therefore, the second business execution component is the business execution component corresponding to component association information 1 and component association information 2. The computer device can determine the component identifier 12342 of the second business execution component as the component identifier of the first business execution component.
[0095] It is understandable that the component functionality information of the first business execution component carried in the component registration request is optional. That is, many of the functions provided by the first business execution component are not mandatory for all first business execution components to implement. For example, the component functionality information includes some core functions and optional functions. Core functions may include the implementation of the Run interface and the Query interface, error handling, resource management, status reporting, etc. Optional functions can be determined according to the design and purpose of the first business execution component.
[0096] Furthermore, the computer device can obtain the component name, component service address, component version, and component function information carried in the component registration request from the component registration request, and determine the component identifier of the first business execution component, the component name, component service address, component version, and component function information in the component registration request as the first component association information of the first business execution component.
[0097] Optionally, the computer device can also generate a random identifier for the target component, and determine the random identifier as the component identifier of the first business execution component. The component identifier of the first business execution component, the component name carried in the component registration request, the component service address, the component version, and the component functional information are determined as the first component association information of the first business execution component. The component identifier of the first business execution component must be different from the component identifier of any other business execution component corresponding to any component association information in the component service library. For example, if the component identifier consists of a string of numbers, the computer device can generate a random number. When this random number is different from the component identifier in any other component association information in the component service library, the computer device can determine this random number as the component identifier of the first business execution component.
[0098] Furthermore, the computer device can store the first component association information in a component service library. Alternatively, the computer device can detect the first component association information to obtain a component detection result for the first business execution component. That is, the computer device can detect the validity of the first business execution component based on the first component association information. If the component detection result indicates that the first business execution component is valid, then the first component association information is stored in the component service library. Optionally, the computer device can store the first component association information in the component service library. In this case, the computer device can maintain the component association information in the component service library, resulting in a smaller amount of data to maintain and saving resources. Alternatively, the first component association information can be associated with the first business execution component and stored in the component service library. In this case, when calling the business execution component, the component can be called directly from the component service library, thereby improving the efficiency of business data processing. The processing of the component service library can be performed as needed.
[0099] The first component association information in the component service library includes component association information corresponding to business execution components registered by developers and business execution components registered by users. It can be understood that the first component association information in the component server library includes component association information corresponding to the target business execution component, for example, Figure 3The component association information corresponding to the target business execution components mentioned in the embodiments refers to the component association information of each target business execution component in the business data. This information is obtained from the component service library after the first business execution component completes registration and stores its corresponding first component association information in the component service library. When a computer device needs to use a certain business execution component, it can call that business execution component through the component service address in the component association information in the component service library.
[0100] It is understandable that the process of registering business execution components described above could be as follows: The orchestration engine developer initiates a component registration request during engine development to register some default business execution components in the component service library and store the component association information corresponding to each business execution component. Alternatively, it could be the process where the orchestration engine user (i.e., the business party sending business data) initiates a component registration request to register the required business execution components when the business execution components corresponding to the component association information in the component service library cannot meet the execution requirements of the business data.
[0101] Step S402: Obtain information on N business processing nodes.
[0102] In this embodiment, the computer device can obtain N business processing node information corresponding to the business data according to the business processing logic of the business data. The N business processing node information represent the processing logic order for the business data; each business processing node information includes component description information indicating the target business execution component associated with that business processing node information; N is a positive integer.
[0103] Specifically, if the business data is code instruction data, the computer device can parse the code instruction data to determine the N business nodes indicated by the code instruction data, and the logical processing order between the N business nodes. That is, the computer device can identify the N business nodes indicated by the business data from the code execution data. These N business nodes can be functions, methods, or code blocks, representing various steps or operations in the business data. Furthermore, the computer device can parse the logical processing order between these business nodes, that is, the related relationships or dependencies between these business nodes, including determining which business nodes are executed sequentially, which business nodes are executed based on conditional judgments, and which business nodes are executed under the triggering of specific events, etc.
[0104] Optionally, the code instruction data can be business orchestration code segments, such as program code orchestrated by Python, for example, "Node1 = TaskNode("Node1", task1); Node2 = TaskNode("Node2", task2)". In this case, the computer device can directly obtain the node keywords in the code instruction data, such as "Node" in Python. Based on the code statements associated with the node keywords, it can determine the N business nodes indicated by the code instruction data, such as Node1 and Node2 in the above example. It can also obtain the dependency keywords in the code instruction data, such as "set_downstream", "set_upstream", and "add_dependency" in Python. Based on the code statements associated with the dependency keywords, it can determine the processing logic order between the N business nodes. For example, "Node2.add_dependency(Node1)" can determine the processing logic order between Node1 and Node2, that is, Node2 depends on Node2, and Node2 is executed after Node1.
[0105] Alternatively, the code instruction data can be a regular code segment, which refers to a non-orchestrated code segment. In this case, the computer device can split the regular code segment into N code sub-segments, and construct N business nodes based on these N code sub-segments. Based on the code call relationships of the N code sub-segments within the regular code segment, the processing logic order among the N business nodes can be determined. This approach enables the orchestration and processing of business data in various business scenarios, even when the technology stack uses any programming language, thereby improving the flexibility and versatility of business data processing.
[0106] Computer equipment can obtain node identifiers, node business information, and component descriptions corresponding to N business nodes from code instruction data. It can then encapsulate these information to obtain N node descriptions. For example, a code block defines business node A, which also includes its node identifier, node business information, and component descriptions. The computer equipment can extract these information from the code block and encapsulate it into the node description information for business node A.
[0107] Furthermore, the computer equipment can determine the N business processing node information corresponding to the business data based on the N node description information and the processing logic order between the N business nodes. It can be understood that one business processing node information corresponds to one business node and its node description information. And there can be directed lines among the N business processing node information, which are used to indicate the logical processing order between the N business processing node information.
[0108] Optionally, if the business data is a directed graph of business data, which includes N business nodes, the computer device can traverse the directed graph of business data to sequentially obtain business nodes without parent nodes or business nodes whose parent nodes are already traversed, thereby obtaining the processing logic order between the N business nodes and obtaining the node description information of each business node.
[0109] A computer device can generate N business processing node information corresponding to business data based on the node description information of N business nodes and the processing logic order between the N business nodes. Each business processing node information corresponds to one node description information and one business node; the node description information includes component description information indicating the target business execution component associated with that business processing node information, a node identifier, and the node's business.
[0110] Please see also Figure 5 , Figure 5 This is a schematic diagram of a directed graph data traversal provided in an embodiment of this application, such as... Figure 5As shown, the computer device can traverse the directed graph data corresponding to business data 501, sequentially obtaining business nodes without parent nodes or business nodes whose parent nodes are already traversed. During traversal, the computer device first obtains business node A without a parent node, resulting in business node sequence 502; upon further traversal to business node B1 (whose parent node is business node A, and is now already traversed), it obtains business node sequence 503 (including "business node A → business node B1"); it obtains business node B2 (whose parent node is already traversed), resulting in business node sequence 504 (including "business node A → business node B1 → business node B2"); and it obtains business node B3 (whose parent node is already traversed), resulting in business node sequence 505 (including "business node A → business node B1 → business node B2 → business node B3"). The computer device then traverses to the business node... When C1 (the parent node of business node C1 is business node B2, which is in a traversed state), obtain business node sequence 506 (including "business node A → business node B1 → business node B2 → business node B3 → business node C1") by obtaining business node C1 (the parent node is in a traversed state); obtain business node sequence 507 (including "business node A → business node B1 → business node B2 → business node B3 → business node C1 → business node C2") by obtaining business node D (the parent node is in a traversed state) by obtaining business node sequence 508 (including "business node A → business node B1 → business node B2 → business node B3 → business node C1 → business node D").
[0111] It should be understood that after associating the node description information with each business node in the business node sequence 508, the business processing node information corresponding to each business node can be obtained. That is, N business nodes, the node description information corresponding to each of the N business nodes, and the processing logic order between the N business nodes can constitute the N business processing node information corresponding to the business data.
[0112] Step S403: Detect the execution status information of the second business processing information.
[0113] In this embodiment, the computer device can sequentially execute the node services indicated by the N service processing node information based on the target service execution component associated with the N service processing node information, thereby obtaining the node execution results corresponding to the N service processing node information. For the specific execution process, please refer to the detailed description in step S302, which will not be repeated here. The N service processing node information includes service processing node information A. i, where i is a positive integer less than or equal to N; taking the business processing node information A as an example. i For example, computer equipment can determine the relationship between business processing node information A and business processing logic. i There are second business processing node information with data dependencies. Among them, N business processing node information includes the second business processing node information, and for business processing node information A... i The execution of node services depends on the execution results of the second service processing node information.
[0114] For example, among the N service processing node information corresponding to the service node sequence 508 in step 402 above, when service processing node information A... i When the business processing node information is for business node C2, the execution result of business node B3 is applied to business node C2, and is related to business processing node information A. i The second business processing node information that has a data dependency relationship is the business processing node information corresponding to business node B3.
[0115] The computer equipment can detect the execution status information of the second business processing information. If the execution status information of the second business processing information is detected as completed, then the execution based on the business processing node information A is performed. i The component description information is used to determine the business processing node information A from the component service library. i The process of the associated target business execution component is the process of jumping to execution step S404. If the execution status information of the second business processing information is not detected as execution completion information, the process waits until the execution status information of the second business processing information is detected as execution completion information, at which point the computer device can begin executing the business processing node information A. i .
[0116] Step S404, execute business processing node information A i .
[0117] In this embodiment of the application, the computer device can be based on the service processing node information A i The component description information is used to determine the business processing node information A from the component service library. i The associated target business execution component, specifically, includes component identifier, component name, and component version information. The computer device can search in the component service library based on the component identifier, component name, and component version, and determine the first business execution component corresponding to the first component association information found as business processing node information A. i The associated target business execution component.
[0118] It should be understood that computer equipment determines business processing node information A i After associating with the target business execution component, it is possible to check whether the component function information of the target business execution component is consistent, and the business processing node information A. i The required function to be invoked, i.e., whether the component function information of the target business execution component can support the processing of the business processing node information A. i The detection process can be as follows: using the target business execution component to execute business processing node information A. i If the target service execution component returns an error message, it is considered that the component function information of the target service execution component does not support processing the service processing node information A. i Error messages can indicate that a component is not supported. It's important to note that the target business execution component associated with a leaf business node in the business data must implement the RunNode interface. Only when the computer device processes the leaf business nodes in the business data through the orchestration engine can it call the RunNode interface to execute the business processing node information corresponding to the leaf business node. For example, ... Figure 5 The leaf business nodes of the business data 501 shown can be business nodes B1, C1 and D in the business data 501.
[0119] Computer equipment can obtain business processing node information A i The associated node has pending execution data. Based on the node's pending execution data, the business processing node information A is called. i The associated target business execution component, through business processing node information A i The associated target business execution component processes the node business indicated by the data to be executed at the processing node. This data may include the node business and the data required to execute it. The data required to execute the node business may originate from the business processing node information A. i The information from the second business processing node can also originate from a database, or from both the second business processing node information and the database, depending on the actual situation in the business data. It is understandable that when the computer device processes the business at the node indicated by the data to be executed by the target business execution component, it can obtain the return result from the target business execution component.
[0120] Step S405: Detect the return result of the target business execution component.
[0121] In this embodiment, the computer device can detect the return result of the target service execution component. If the detected return result is component execution information, then step S406 is executed; if the computer device detects the return result through service processing node information A...i The associated target business execution component obtains information A regarding the business processing node. i If the computer device detects that the returned result is a business execution identifier, then step S407 is executed; the computer device can obtain the component execution cycle. If the business processing node information A is not obtained within the component execution cycle... i The associated target business execution component returns component execution information, or the business processing node information A is obtained. i If the component execution information returned by the associated target business execution component indicates execution failure information, that is, if the returned result is component execution information and the component execution information is empty, or the component execution information indicates execution failure information, then step S408 is executed.
[0122] Step S406: Based on the component execution information, generate the node execution result corresponding to the business execution node information.
[0123] In this embodiment, the computer device can generate the node execution result corresponding to the business execution node information based on the component execution information. Specifically, the computer device can identify the content included in the component execution information. If the component execution information does not include a business sub-process, then the component execution information is determined as business processing node information A. i The corresponding node execution result. That is, if the field in the component execution information that indicates the business sub-process is empty, it is determined that the target business execution component did not return a business sub-process, and the returned component execution information is the target business execution component's information for business processing node A. i The execution result of the node.
[0124] Optionally, if the component execution information includes business sub-processes, the computer device can generate sub-process execution objects for the business sub-processes through the business control system, obtain the sub-process execution results for the business sub-processes based on the sub-process execution objects, and generate business processing node information A based on the sub-process execution results. i The corresponding node execution results. The business control system, which is the orchestration system or orchestration layer of the orchestration engine corresponding to this data processing method, can handle information from N business processing nodes. This orchestration engine also includes a business execution system (i.e., the execution layer), used to indicate the execution process involved in the target business execution component. The sub-process execution object is a specific execution instance created based on a business sub-process. The sub-process execution object can be used to record the business sub-nodes included in the business sub-process, the sub-business node information corresponding to the business sub-nodes, the sub-process execution logic of the business sub-process, and the execution status information of the business sub-process, etc.
[0125] The specific implementation process of a computer device obtaining the execution result of a sub-process for a business sub-process based on a sub-process execution object can be as follows: The computer device can obtain M sub-business node information corresponding to the business sub-process from the sub-process execution object according to the sub-process execution logic indicated by the sub-process execution object; M is a positive integer. Further, based on the component description information in the M sub-business node information, the target business execution components associated with each of the M sub-business node information can be obtained. Based on the target business execution components associated with each of the M sub-business node information, the node sub-businesses indicated by each of the M sub-business node information are executed to obtain the sub-process execution results corresponding to each of the M sub-business node information. It can be understood that the target business execution components associated with each of the M sub-business node information can include business processing node information A. i The associated target business execution component. For example, business processing node information A. i The associated target business execution component X returns information A regarding the business processing node. i The business sub-process includes three sub-business node information, which can be associated with target business execution component X, target business execution component Y, and target business execution component Z, respectively.
[0126] Step S407: Based on the business execution identifier, obtain the node execution result corresponding to the business processing node information Ai.
[0127] In this embodiment, the computer device can obtain the result query period. When the time interval between obtaining the service execution identifier and the system time reaches the result query period, it can call the service processing node information A based on the service execution identifier. i The query interface (Query interface) of the associated target business execution component is used to obtain business processing node information A. i The associated target business execution component retrieves the execution status and result information for the node's pending data. Specifically, the computer device periodically calls the Query interface to retrieve the execution status and result information from the target business execution component. When the retrieved execution status information for the node's pending data indicates completion, the computer device can identify the corresponding execution result information as business processing node information A. i The execution result of the corresponding node.
[0128] Understandably, when a computer device calls the Query interface, it can also pass additional conditional parameters, meaning that the Query interface can only retrieve execution result information when the execution status information is "execution completed". If the execution status information retrieved through the Query interface is not "execution completed", it can return an empty value or no information.
[0129] Step S408: Add the business processing node information A from the node execution queue. i The execution status information is updated to execution failure information.
[0130] In this embodiment of the application, the service processing node information A i Existing in the node execution sequence, computer devices can store the business processing node information A in the node execution queue. i The execution status information is updated to execution failure information. Furthermore, regarding the business processing node information A... i Create new business processing node information A i If, based on business processing logic, information A that depends on the business processing node is obtained... i Based on the first business processing node information, the new business processing node information A will be generated. i Add it to the node execution sequence. The node execution sequence can be a queue.
[0131] It should be understood that the new business processing node information A i In the node execution sequence, at any position preceding the first business processing node information; new business processing node information A i The execution status information is "waiting to be executed"; business processing node information A i And new business processing node information A i The node identifiers are the same. That is, the target business execution component targets the business processing node information A. i In the event of execution failure, the computer device can transmit the business processing node information A. i Add it back to the node execution sequence and wait for the business processing node information A. i The process is re-executed, i.e., jumps to execution step S404 and executes the new business processing node information A. i Business processing node information A i And business processing node information A i Apart from the differences in basic information such as execution status and execution time, all other information is exactly the same.
[0132] Step S409: Based on the execution results of N nodes, determine the business processing result for the business data.
[0133] In this embodiment of the application, the specific implementation process of step S409 can refer to the following: Figure 3 The specific process described in step S303 shown is not repeated here.
[0134] Please see also Figure 6 , Figure 6 This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 2 This is used to describe the processing architecture in the orchestration engine corresponding to this data processing method. For example... Figure 6 As shown, the orchestration engine corresponding to this data processing method includes a business control system (orchestration layer) and a component service library. The various business execution components corresponding to the component service addresses in the component service library constitute the business execution system (execution layer) of the orchestration engine. Figure 6 The component services in the diagram only show a few example business execution components used to provide component services, including PipelinePlugin, XXX components, and Atomic Plugin.
[0135] The orchestration engine is integrated into the computer device. When the computer device obtains business data through the Application Programming Interface Server (Api Server) in the business control system, it can trigger the corresponding operation indicated by the business data through the business execution controller (DagRun Controller). That is, a business execution object (DAGRun object) is created for the business data, and the business execution object is added to the business execution sequence (DagRun Queue).
[0136] A business execution object represents an execution instance of business data. It typically includes the definition of the business data (the relationship between business nodes and directed edges or lines), the execution status information of the business data (such as execution status, success status, failure status, etc.), the execution status information of each business node in the business data, and the input and output data of the business data. For example, consider a business data set with three business nodes (business node A, business node B, and business node C), where business node B depends on business node A, and business node C depends on business node B. A business execution object might contain the following information: the logical definition of the business data: node A → node B → node C; the execution status information of business node A: execution completed; the execution status information of business node B: execution in progress; the execution status information of business node C: awaiting execution; and the execution status information of the business data itself: execution in progress.
[0137] Furthermore, the coordinator (DagRun Reconciler) in the business execution controller can obtain information on N business processing nodes corresponding to the business data according to the business processing logic of the business data. That is, based on the logical definition of the business data in the business execution object corresponding to the business data, it can decompose the business data into N executable business nodes. For example, the business execution object in the above example can obtain information on three business processing nodes: the business processing node information corresponding to business node A, the business processing node information corresponding to business node B, and the business processing node information corresponding to business node C. The coordinator in the business execution controller can detect the tasks (business execution objects) in the DagRun Queue and ensure that they execute as expected, including creating and managing NodeRun objects in the DAGRun.
[0138] DagRun Reconciler can create corresponding NodeRun objects for N business processing nodes based on the business execution objects in the business execution sequence, and add the N NodeRun objects to the NodeRun Queue. A NodeRun object represents an execution instance of a business node and generally includes the business processing node information (including node identifier, component description information, node business, and node name); the node's execution status information (such as execution in progress, execution success, and execution failure); the start and end times of the business node; the input and output data of the business node; and the execution logs and error information of the business node. For example, the NodeRun object for business node A can be represented as "NodeRun A: Node identifier: A; Component description information a; Node business a; Node execution status information: Execution completed; Start time: 2024-11-27 10:00:00; End time: 2024-11-27 10:05:00; Output data: Dataset X".
[0139] The NodeRun Controller can obtain node execution objects from the node execution sequence. The NodeRun Reconciler within the NodeRun Controller can determine the target business execution component indicated by the component description information in the business processing node information of that node execution object. Furthermore, for that target business execution component, it obtains the component command-line interface (Cil). This Cil is used to interact with the target business execution component via command-line instructions to execute specific tasks or commands. For example, if the target business execution component is the "data_processor" component, the NodeRun Controller can construct a command-line instruction, such as "data_processor--input data.csv--output processed_data.csv". By executing this command-line instruction, the NodeRun Controller can execute the component's running method, thereby interacting with the "data_processor" component and using the component services provided by it. Specifically, when executing the component's running method, it can obtain the component service address from the component service library, which is the component description information of the target business execution component associated with the business processing node information, and then call the corresponding business execution component, i.e., the "data_processor" component in this case.
[0140] by Figure 5 Taking the business processing node information corresponding to business node sequence 508 as an example, the business execution controller can add the node execution object corresponding to business node A to the node execution sequence, waiting for the node execution controller to execute it. When the node execution controller completes the execution of business node A through the target business execution component associated with business node A, that is, when the execution status information included in the node execution object corresponding to business node A in the node execution sequence is execution success information, the business execution controller can add the node execution object corresponding to business node B1 to the node execution sequence, waiting for the node execution controller to execute the node execution object corresponding to business node B1. It can be understood that since the execution status information of business node A, which business nodes B2 and B3 depend on, is execution success information, even if the node execution controller has not yet executed the node execution object corresponding to business node B1, that is, has not yet obtained the node execution result corresponding to business node B1, the business execution controller can still add the node execution objects corresponding to business nodes B2 and B3 respectively to the node execution sequence. In other words, the node execution controller can asynchronously execute multiple node execution objects that have no dependencies on each other, as long as the execution status information of the node execution objects that the executed node execution object depends on is successful.
[0141] Through the above process, the orchestration and execution of business data are broken down. Executors implementing different functions are componentized (i.e., business execution components). Business data is broken down into steps, resulting in N business processing node information. This allows computer devices to directly call the business execution component with the function to process each business processing node, executing the corresponding node business. Since business processing node information is not program code, its processing does not involve the programming language used by the program code. This allows for orchestration and execution of business data regardless of whether the business data uses any programming language or other user-manufactured business processes. In other words, it can be applied to any business scenario, thus improving the universality and flexibility of business data processing. Furthermore, the processing of business data does not require the introduction of various programming languages, reducing the maintenance complexity of the orchestration engine. Moreover, there is no need to predefine business sub-processes in the business data; the target business execution component can automatically create and return business sub-processes for processing, thereby meeting flexible and varied business needs and enhancing the practicality of the orchestration engine. Furthermore, users can register business execution components themselves and store the component association information of the business execution components in the component service library for use. By registering business execution components that implement different business functions, the business functions of the orchestration engine can be enriched.
[0142] Further, please see Figure 7 , Figure 7 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 3 This data processing method can be executed by a computer device, which can be, for example,... Figure 1 Any terminal device in the terminal device cluster shown. The following explanation will use the execution of this data processing method by a computer device as an example, where the computer device integrates an orchestration engine that implements this data processing method. This data processing method describes the process of dynamically creating and executing business sub-processes, and may include at least the following steps S701-S710:
[0143] Step S701: Obtain the node execution object.
[0144] In this embodiment of the application, the computer device can obtain a node execution object (NodeRun object) from the node execution sequence through a node execution controller. For example, the node execution object can be as follows: Figure 4 The service processing node information A mentioned in the embodiment i The corresponding node execution object, hereinafter referred to as business processing node information A. iThe following explanation will be based on the corresponding node execution object.
[0145] Step S702: Obtain relevant information about the execution object of the execution node.
[0146] In this embodiment of the application, when the service processing node information A i When the execution results or required data of the nodes are stored in the database, the node execution controller can obtain relevant information about the execution object of the execution node from the database, which is the business processing node information A. i The execution result of the business processing node information it depends on, or the execution result of business processing node information A. i The required data.
[0147] Step S703: Determine the target business execution component.
[0148] In this embodiment of the application, the computer device can be based on the service processing node information A i The component description information in the corresponding node execution object is used to determine the business processing node information A from the component service library. i The associated target business execution component.
[0149] Step S704: Process node services.
[0150] In this embodiment of the application, the computer device can be based on a node execution controller and process node information A through a service processing node. i The associated target business execution component processes business processing node information A. i The indicated node service.
[0151] Step S705: Return component execution information.
[0152] In this embodiment of the application, the node execution controller obtains information from service processing node A. i Within the associated target business execution component, obtain the component execution information returned by that component. If the component execution information includes a business sub-process, then continue with step S706.
[0153] Step S706: Store the business sub-process and the sub-process execution object in the database.
[0154] In this embodiment of the application, the computer device can generate a sub-process execution object for a business sub-process and store the business sub-process and the sub-process execution object in a database.
[0155] Step S707: Add the sub-process execution object to the business execution sequence.
[0156] In this embodiment, the computer device can add sub-process execution objects to the business execution sequence. Further, according to the sub-process execution logic indicated by the sub-process execution object, it can obtain M sub-business node information corresponding to the business sub-process from the sub-process execution object; M is a positive integer. The computer device can create sub-node execution objects for each of the M sub-business node information and add the M sub-node execution objects to the node execution queue. It waits for the node execution controller to obtain the target business execution components associated with each of the M sub-business node information based on the component description information in the sub-business node information of the M sub-node execution objects. Based on the target business execution components associated with each of the M sub-business node information, it executes the node sub-businesses indicated by each of the M sub-business node information, thereby obtaining the sub-process execution results corresponding to each of the M sub-business node information.
[0157] Step S708: Update the execution status information to "currently executing" status information.
[0158] In this embodiment of the application, after the computer device adds M child node execution objects to the node execution queue, the business processing node information A can be... i The corresponding execution status information is updated to "in execution" status information, that is, while waiting for the node execution controller to execute the M sub-node execution objects corresponding to the business sub-process, the business processing node information A... i The current state is "Executing". Optionally, when the node execution controller completes the execution of M sub-nodes corresponding to the business sub-process, it obtains the sub-process execution result for the business sub-process, and further generates business processing node information A based on the sub-process execution result. i When the corresponding node executes the result, the computer device can transmit the business processing node information A. i The execution status information is updated to execution completed information.
[0159] Step S709: Add the sub-process execution object corresponding to the predefined business sub-process to the business execution sequence.
[0160] In this embodiment of the application, if the service processing node information A i The corresponding business nodes have predefined business sub-processes associated with them in the business data. This means that the business data acquired by the computer device already contains these sub-processes. Since the target business execution component does not need to return the sub-processes, the computer device can add the predefined sub-process execution object to the business execution sequence when executing the corresponding sub-process execution object. It can be understood that when executing this sub-process execution object, the computer device can obtain the execution status information of the sub-process execution object through the node execution coordinator to determine whether the sub-process execution object has been completed.
[0161] Step S710: If a business node fails to execute, add it to the node execution sequence for re-execution.
[0162] In this embodiment of the application, when the target service execution component targets service processing node information A i If execution fails, or if component execution information is not returned within the component execution cycle, the business processing node information A can be determined. i If the corresponding business node fails to execute, the computer device can send the business processing node information A from the node execution queue. i The execution status information is updated to execution failure information, and this can be applied to business processing node information A. i Create new business processing node information A i If, based on business processing logic, information A that depends on the business processing node is obtained... i Based on the first business processing node information, the new business processing node information A will be generated. i Add it to the node execution sequence. Among them, the new business processing node information A... i In the node execution sequence, at any position preceding the first business processing node information; and the new business processing node information A i The execution status information is "waiting to be executed"; business processing node information A i And new business processing node information A i The node identifiers are the same.
[0163] Through the above process, computer devices can directly invoke business execution components with the capability to process each business processing node to execute the corresponding node business. This allows for application to any business scenario, thereby improving the universality and flexibility of business data processing. Furthermore, the processing of business data does not require the introduction of various programming languages, thus reducing the maintenance complexity of the orchestration engine. It also provides processing procedures for business data that do not require pre-defining business sub-processes within the business data; these sub-processes can be automatically created and returned by the target business execution component for processing. This saves on business data orchestration costs, enables flexible and diverse business requirements, and enhances the practicality of the orchestration engine.
[0164] Further, please see Figure 8 , Figure 8 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. The data processing apparatus 800 can be a computer program (including program code, etc.) running on a computer device; for example, the data processing apparatus 800 can be an application software. The data processing apparatus 800 can be used to execute corresponding steps in the method provided in the embodiments of this application. Figure 8 As shown, the data processing device 800 can be used for Figure 3 , Figure 4 and Figure 7 Specifically, the data processing device 800 in the corresponding embodiment may include: a component registration module 11, an information acquisition module 12, a node business execution module 13, and a result generation module 14.
[0165] Information acquisition module 12 is used to acquire N business processing node information corresponding to the business data according to the business processing logic of the business data; the N business processing node information is used to represent the processing logic order for the business data; each business processing node information includes component description information that indicates the target business execution component associated with the business processing node information; N is a positive integer;
[0166] The node service execution module 13 is used to execute the node services indicated by the N service processing node information in sequence based on the target service execution component associated with the N service processing node information, and obtain the node execution results corresponding to the N service processing node information respectively.
[0167] The result generation module 14 is used to determine the business processing result for the business data based on the execution results of N nodes.
[0168] In one possible implementation, the N business processing node information includes business processing node information A. i , where i is a positive integer less than or equal to N; the node service execution module 13 is used to execute the node services indicated by the N service processing node information in sequence based on the target service execution component associated with the N service processing node information, and to obtain the node execution results corresponding to the N service processing node information respectively. The data decomposition module is specifically used to perform the following operations:
[0169] Based on business processing node information A i The component description information is used to determine the business processing node information A from the component service library. i The associated target business execution component;
[0170] Obtain business processing node information A i The associated node has pending execution data. Based on the node's pending execution data, the business processing node information A is called. i The associated target business execution component, through business processing node information A i The associated target business execution component processes the node business indicated by the data to be executed on the node and obtains the component execution information;
[0171] Based on the component execution information, generate business processing node information A. iThe execution result of the corresponding node.
[0172] In one possible implementation, the node business execution module 13 is used to generate business processing node information A based on the component execution information. i When the corresponding node executes its result, the node business execution module 13 is specifically used to perform the following operations:
[0173] If the component execution information does not include business sub-processes, then the component execution information will be identified as business processing node information A. i The execution result of the corresponding node;
[0174] If the component execution information includes business sub-processes, then a sub-process execution object is generated for the business sub-process through the business control system. Based on the sub-process execution object, the sub-process execution result for the business sub-process is obtained. Based on the sub-process execution result, business processing node information A is generated. i The corresponding node execution result; the business control system is used to process information from N business processing nodes.
[0175] In one possible implementation, when the node business execution module 13 obtains the sub-process execution result for a business sub-process based on the sub-process execution object, the node business execution module 13 is specifically used to perform the following operations:
[0176] According to the sub-process execution logic indicated by the sub-process execution object, obtain the information of M sub-business nodes corresponding to the business sub-process from the sub-process execution object; M is a positive integer;
[0177] Based on the component description information in the M sub-business node information, obtain the target business execution components associated with the M sub-business node information respectively. Based on the target business execution components associated with the M sub-business node information respectively, execute the node sub-business indicated by the M sub-business node information respectively, and obtain the sub-process execution results corresponding to the M sub-business node information respectively.
[0178] In one possible implementation, the node service execution module 13 is also used to perform the following operations:
[0179] If the business processing node information A is used i The associated target business execution component obtains information A regarding the business processing node. i If the business execution identifier is used, then based on the business execution identifier, the business processing node information A is invoked. i The query interface of the associated target business execution component is used to obtain business processing node information A. i The associated target business execution component provides execution status and result information for the node's pending data.
[0180] When the obtained execution status information for the data to be executed at a node is execution completed, the execution result information corresponding to the execution completed information is determined as business processing node information A. i The execution result of the corresponding node.
[0181] In one possible implementation, the business processing node information A i It exists in the node execution sequence; the node business execution module 13 is also used to perform the following operations:
[0182] Obtain the component execution cycle. If no business processing node information A is obtained within the component execution cycle... i The associated target business execution component returns component execution information, or the business processing node information A is obtained. i If the execution information returned by the associated target business execution component indicates an execution failure, then the business processing node information A in the node execution queue will be... i The execution status information is updated to execution failure information;
[0183] Regarding business processing node information A i Create new business processing node information A i If, based on business processing logic, information A that depends on the business processing node is obtained... i Based on the first business processing node information, the new business processing node information A will be generated. i Add to the node execution sequence; New business processing node information A i In the node execution sequence, at any position preceding the first business processing node information; new business processing node information A i The execution status information is "waiting to be executed"; business processing node information A i And new business processing node information A i The node identifiers are the same.
[0184] In one possible implementation, the node service execution module 13 is also used to perform the following operations:
[0185] Based on the business processing logic, determine the business processing node information A. i Information about the second business processing node with data dependencies exists; N business processing node information includes the information about the second business processing node, and for business processing node information A... i The execution of node services depends on the execution results of the node that processes the information of the second service processing node.
[0186] If the execution status information of the second business processing information is detected as "execution completed", then the execution based on business processing node information A will proceed. iThe component description information is used to determine the business processing node information A from the component service library. i The process of the associated target business execution component.
[0187] In one possible implementation, the data processing device 800 further includes a component registration module 11, which is specifically used to perform the following operations:
[0188] In response to a component registration request for the first business execution component, the first component association information of the first business execution component is obtained through the component registration request; the first component association information is used to represent the functions and attributes of the first business execution component;
[0189] The association information of the first component is detected to obtain the component detection results for the first business execution component;
[0190] If the component detection result indicates that the first business execution component is valid, then the association information of the first component will be stored in the component service library.
[0191] In one possible implementation, the component registration module 11 is used to respond to a component registration request for the first business execution component. When obtaining the first component association information of the first business execution component through the component registration request, the component registration module 11 is specifically used to perform the following operations:
[0192] The system retrieves the component name and functional information carried in the component registration request. Based on the component name or functional information, it searches among registered components. If a second business execution component is found among the registered components, the component identifier in the second component association information of the second business execution component is determined as the component identifier of the first business execution component. The component identifier of the first business execution component, the component name, component service address, component version, and component functional information carried in the component registration request are determined as the first component association information of the first business execution component. The component service address is used to call the first business execution component. The second business execution component has the same component name or component functional information as the first business execution component.
[0193] Alternatively, a random identifier is generated for the target component, and the random identifier is determined as the component identifier of the first business execution component. The component identifier of the first business execution component, the component name, component service address, component version, and component function information carried in the component registration request are determined as the first component association information of the first business execution component. The component identifier of the first business execution component is different from the component identifier of any business execution component in the component service library.
[0194] In one possible implementation, the business data is code instruction data; when the information acquisition module 12 acquires the information of N business processing nodes corresponding to the business data according to the business processing logic of the business data, the information acquisition module 12 is specifically used to perform the following operations:
[0195] The code instruction data is parsed to determine the N business nodes indicated by the code instruction data, as well as the processing logic order between the N business nodes;
[0196] Obtain the node identifier, node business, and component description information corresponding to N business nodes from the code instruction data. Encapsulate the node identifier, node business, and component description information corresponding to each business node to obtain N node description information.
[0197] Based on the description information of N nodes and the processing logic order between N business nodes, determine the information of N business processing nodes corresponding to the business data.
[0198] In one possible implementation, the business data is a directed graph of business data, which includes N business nodes. When the information acquisition module 12 acquires the information of the N business processing nodes corresponding to the business data according to the business processing logic of the business data, the information acquisition module 12 is specifically used to perform the following operations:
[0199] By traversing the directed graph data of the business, we can sequentially obtain business nodes without parent nodes or business nodes whose parent nodes are already traversed in the directed acyclic graph data, obtain the processing logic order between N business nodes, and obtain the node description information of each business node.
[0200] Based on the node description information of N business nodes and the processing logic order between N business nodes, N business processing node information corresponding to the business data is generated; one business processing node information corresponds to one node description information and one business node; the node description information includes component description information, node identifier, and node business used to indicate the target business execution component associated with the business processing node information.
[0201] Through the above process, the orchestration and execution of business data are broken down. Executors implementing different functions are componentized (i.e., business execution components). Business data is broken down into steps, resulting in N business processing node information. This allows computer devices to directly call the business execution component with the function to process each business processing node, executing the corresponding node business. Since business processing node information is not program code, its processing does not involve the programming language used by the program code. This allows for orchestration and execution of business data regardless of whether the business data uses any programming language or other user-manufactured business processes. In other words, it can be applied to any business scenario, thus improving the universality and flexibility of business data processing. Furthermore, the processing of business data does not require the introduction of various programming languages, reducing the maintenance complexity of the orchestration engine. Moreover, there is no need to predefine business sub-processes in the business data; the target business execution component can automatically create and return business sub-processes for processing, thereby meeting flexible and varied business needs and enhancing the practicality of the orchestration engine. Furthermore, users can register business execution components themselves and store the component association information of the business execution components in the component service library for use. By registering business execution components that implement different business functions, the business functions of the orchestration engine can be enriched.
[0202] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 9 As shown, the computer device 900 in this embodiment may include a processor 901, a network interface 904, and a memory 905. Furthermore, the computer device 900 may also include a user interface 903 and at least one communication bus 902. The communication bus 902 is used to enable communication between these components. The user interface 903 may include a display screen and a keyboard; optionally, the user interface 903 may also include a standard wired interface or a wireless interface. The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 905 may be a high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 905 may also be at least one storage device located remotely from the processor 901. Figure 9 As shown, the memory 905, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.
[0203] Network interface 904 provides network communication elements; user interface 903 is mainly used to provide input for users; and processor 901 can be used to call device control applications stored in memory 905 to perform the following operations:
[0204] According to the business processing logic of the business data, obtain the information of N business processing nodes corresponding to the business data; the information of N business processing nodes is used to represent the processing logic order for the business data; each piece of information of business processing nodes includes component description information that indicates the target business execution component associated with that business processing node; N is a positive integer;
[0205] Based on the target business execution component associated with N business processing node information, the node business indicated by each of the N business processing node information is executed sequentially to obtain the node execution results corresponding to each of the N business processing node information.
[0206] Based on the execution results of N nodes, determine the business processing result for the business data.
[0207] Furthermore, it should be noted that embodiments of this application also provide a computer-readable storage medium storing a computer program adapted to be loaded and executed by the processor. Figure 3 , Figure 4 and Figure 7 For details on the methods provided in each step, please refer to the document. Figure 3 , Figure 4 and Figure 7 The implementation methods provided for each step are not repeated here. Furthermore, the beneficial effects of using the same method are also not repeated. For technical details not disclosed in the computer-readable storage medium embodiments involved in this application, please refer to the description of the method embodiments of this application. As an example, a computer program may be deployed to execute on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network.
[0208] The computer-readable storage medium can be the apparatus provided in any of the foregoing embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0209] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform... Figure 3 , Figure 4 and Figure 7 The methods provided are among the various optional methods available in the code, so they will not be elaborated upon here.
[0210] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.
[0211] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.
[0212] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.
[0213] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, causing the instructions stored in the computer-readable storage medium to produce an article of manufacture including the instruction means, or to be transmitted via a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The instruction means is implemented in the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.
[0214] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.
[0215] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.
[0216] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. A data processing method, characterized in that, The method includes: According to the business processing logic of the business data, obtain N business processing node information corresponding to the business data; the N business processing node information is used to represent the processing logic order for the business data; each business processing node information includes component description information indicating the target business execution component associated with the business processing node information; N is a positive integer; Based on the target service execution component associated with the N service processing node information, the node services indicated by the N service processing node information are executed sequentially to obtain the node execution results corresponding to the N service processing node information. Based on the execution results of N nodes, determine the business processing result for the business data.
2. The method according to claim 1, characterized in that, The N service processing node information includes service processing node information A. i , where i is a positive integer less than or equal to N; the target service execution component associated with the N service processing node information sequentially executes the node services indicated by the N service processing node information respectively, and obtains the node execution results corresponding to the N service processing node information respectively, including: Based on the aforementioned business processing node information A i The component description information is used to determine the business processing node information A from the component service library. i The associated target business execution component; Obtain the business processing node information A i The associated node has pending execution data. Based on the node's pending execution data, the business processing node information A is invoked. i The associated target business execution component, through the business processing node information A i The associated target service execution component processes the node service indicated by the node's data to be executed, and obtains component execution information; Based on the component execution information, the business processing node information A is generated. i The execution result of the corresponding node.
3. The method according to claim 2, characterized in that, The business processing node information A is generated based on the component execution information. i The execution results of the corresponding nodes include: If the component execution information does not include a business sub-process, then the component execution information is determined as the business processing node information A. i The execution result of the corresponding node; If the component execution information includes the business sub-process, then a sub-process execution object for the business sub-process is generated through the business control system. Based on the sub-process execution object, the sub-process execution result for the business sub-process is obtained. Based on the sub-process execution result, the business processing node information A is generated. i The corresponding node execution result; the business control system is used to process the information of the N business processing nodes.
4. The method according to claim 3, characterized in that, The step of obtaining the sub-process execution result for the business sub-process based on the sub-process execution object includes: According to the sub-process execution logic indicated by the sub-process execution object, obtain the information of M sub-business nodes corresponding to the business sub-process from the sub-process execution object; M is a positive integer; Based on the component description information in the M sub-business node information, obtain the target business execution components associated with the M sub-business node information respectively, and execute the node sub-businesses indicated by the M sub-business node information respectively based on the target business execution components associated with the M sub-business node information respectively, to obtain the sub-process execution results corresponding to the M sub-business node information respectively.
5. The method according to claim 2, characterized in that, The method further includes: If the business processing node information A is used i The associated target business execution component obtains information A for the business processing node. i If the business execution identifier is obtained, then the business processing node information A is invoked based on the business execution identifier. i The query interface of the associated target business execution component is used to obtain the business processing node information A. i The associated target business execution component provides execution status information and execution result information for the data to be executed on the node; When the obtained execution status information for the data to be executed on the node is execution completion information, the execution result information corresponding to the execution completion information is determined as the business processing node information A. i The execution result of the corresponding node.
6. The method according to claim 2, characterized in that, The business processing node information A i Exists in the node execution sequence; the method further includes: Obtain the component execution cycle. If the business processing node information A is not obtained within the component execution cycle,... i The component execution information returned by the associated target business execution component, or the business processing node information A obtained. i If the associated target service execution component returns a component execution information indication of execution failure, then the service processing node information A in the node execution queue will be removed. i The execution status information is updated to execution failure information; Regarding the business processing node information A i Create new business processing node information A i If, based on the aforementioned business processing logic, information A dependent on the business processing node is obtained... i If the first service processing node information is obtained, then based on the first service processing node information, the new service processing node information A will be... i Added to the node execution sequence; the new business processing node information A i In the node execution sequence, the new service processing node information A is located at any position before the first service processing node information. i The execution status information is "waiting to execute"; the business processing node information A i and the new business processing node information A i The node identifiers are the same.
7. The method according to claim 2, characterized in that, The method further includes: Based on the aforementioned business processing logic, the business processing node information A is determined. i The information includes second business processing node information with data dependencies; the N business processing node information includes the second business processing node information, and specifically for the business processing node information A i The execution of node services depends on the node execution results of the second service processing node information; If the execution status information of the second service processing information is detected as execution completed, then the service processing node information A is executed. i The component description information is used to determine the business processing node information A from the component service library. i The process of the associated target business execution component.
8. The method according to claim 1, characterized in that, The method further includes: In response to a component registration request for a first service execution component, the first component association information of the first service execution component is obtained through the component registration request; the first component association information is used to represent the function and attributes of the first service execution component. The association information of the first component is detected to obtain the component detection results for the first service execution component; If the component detection result indicates that the first business execution component is valid, then the first component association information is stored in the component service library.
9. The method according to claim 8, characterized in that, The response addresses a component registration request for the first service execution component, and obtains the first component association information of the first service execution component through the component registration request, including: The component name and component function information carried in the component registration request are obtained. Based on the component name or component function information, a search is performed among the registered components. If a second business execution component is found among the registered components, the component identifier in the second component association information of the second business execution component is determined as the component identifier of the first business execution component. The component identifier of the first business execution component, the component name, component service address, component version, and component function information carried in the component registration request are determined as the first component association information of the first business execution component. The component service address is used to call the first business execution component. The component name or component function information of the second business execution component is the same as that of the first business execution component. Alternatively, a random identifier is generated for the target component, and the random identifier is determined as the component identifier of the first service execution component. The component identifier of the first service execution component, the component name, component service address, component version, and component function information carried in the component registration request are determined as the first component association information of the first service execution component. The component identifier of the first service execution component is different from the component identifier of any service execution component in the component service library.
10. The method according to claim 1, characterized in that, The business data is code instruction data; the step of obtaining N business processing node information corresponding to the business data according to the business processing logic of the business data includes: The code instruction data is parsed to determine the N business nodes indicated by the code instruction data, and the processing logic order between the N business nodes; Obtain the node identifier, node business, and component description information corresponding to the N business nodes from the code instruction data, and encapsulate the node identifier, node business, and component description information corresponding to each business node to obtain N node description information; Based on the description information of the N nodes and the processing logic order among the N business nodes, determine the information of the N business processing nodes corresponding to the business data.
11. The method according to claim 1, characterized in that, The business data is a directed graph of business data, which includes N business nodes; obtaining the information of the N business processing nodes corresponding to the business data according to the business processing logic of the business data includes: By traversing the directed graph data of the business, the business nodes without parent nodes or the business nodes whose parent nodes are already traversed are obtained in the directed acyclic graph data in turn, so as to obtain the processing logic order between the N business nodes and obtain the node description information of each business node. Based on the node description information of the N business nodes and the processing logic order between the N business nodes, N business processing node information corresponding to the business data is generated; one business processing node information corresponds to one node description information and one business node; the node description information includes component description information, node identifier, and node business used to indicate the target business execution component associated with the business processing node information.
12. A data processing apparatus, characterized in that, The device includes: The information acquisition module is used to acquire N business processing node information corresponding to the business data according to the business processing logic of the business data; the N business processing node information is used to represent the processing logic order for the business data; each business processing node information includes component description information for indicating the target business execution component associated with the business processing node information; N is a positive integer; The node service execution module is used to sequentially execute the node services indicated by the N service processing node information based on the target service execution components associated with the N service processing node information, and obtain the node execution results corresponding to the N service processing node information respectively; The result generation module is used to determine the business processing result for the business data based on the execution results of N nodes.
13. A computer device, characterized in that, Includes processor, memory, and input / output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-11.
14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-11.
15. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-11.