Execution process processing method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to an execution process processing method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of computers. The method comprises the following steps: acquiring a plurality of execution flows corresponding to a to-be-processed file, and determining a multiplexing node corresponding to each execution flow; on the basis of the multiplexing node and the node dependency relationship in each execution process, combining each execution process to obtain a total execution process; the total execution process comprises a plurality of execution process branches; and processing the to-be-processed file based on each execution process branch in the total execution process to obtain a file processing result. By adopting the method, the repeated execution of the reusable process nodes can be reduced, and the processing resources required for executing the process are reduced, so that the processing efficiency during the wholesale-to-coupon processing is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an execution process processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] In the financial business system, due to business needs, each file will be processed in batches in different branch business systems or branch servers, such as batch migration of files from one branch server to another.

[0003] At present, when the files of the financial business system are processed in batches, it usually includes multiple processing processes such as file splitting, file parsing, file processing, and file return. In the related art, when processing files according to multiple processing flows, they are usually processed separately according to the execution order of each flow. Each flow contains multiple flow nodes. For example, the file processing flow may include downloading, backup, cleaning, annotation, merging, uploading and other nodes. Executing the above processing flows will consume a lot of computing resources and computing time, reducing the efficiency of batch processing. Summary of the invention

[0004] Based on this, it is necessary to provide an execution process processing method, device, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of batch transfer processing in response to the above technical problems.

[0005] In a first aspect, the present application provides an execution process processing method. The method comprises:

[0006] Obtain multiple execution processes corresponding to the file to be processed, and determine the reuse nodes corresponding to each of the execution processes;

[0007] Based on the reused node and the node dependency relationship in each of the execution flows, each of the execution flows is merged to obtain a total execution flow; the total execution flow includes multiple execution flow branches;

[0008] The to-be-processed file is processed based on each of the execution flow branches in the overall execution flow to obtain a file processing result.

[0009] In one embodiment, each of the execution processes includes a plurality of nodes arranged in an execution order; and determining the reused nodes corresponding to each of the execution processes includes:

[0010] Based on the nodes included in each of the execution flows, the nodes in each of the execution flows that meet the same preset conditions are determined as reused nodes of each of the execution flows.

[0011] In one embodiment, each of the execution processes includes a file parsing process and a file splitting process; the file parsing process includes a file downloading node, a file parsing node and a data sharding node arranged in a preset execution order; the file splitting process includes a file downloading node and a file splitting node arranged in a preset execution order;

[0012] The determining of the reuse nodes corresponding to the execution processes includes:

[0013] The nodes satisfying the same preset conditions in the file parsing process and the file splitting process are determined as the file downloading nodes, and the file downloading nodes are determined as the reuse nodes.

[0014] In one embodiment, based on the reused node and the node dependency relationship in each of the execution flows, each of the execution flows is merged to obtain a total execution flow, including:

[0015] Screen the preset thread pool to get the target thread;

[0016] By using the target thread, each reused node included in each execution process is merged to obtain an initial overall process;

[0017] Based on the node dependency relationship corresponding to each of the execution processes, the non-reuse nodes corresponding to the execution processes are linked to the initial overall process to obtain the overall execution process.

[0018] In one embodiment, the processing of the to-be-processed file based on each of the execution flow branches in the overall execution flow to obtain a file processing result includes:

[0019] If the reuse node has a node execution result of a target execution flow branch, the other execution flow branches except the target execution flow branch are respectively executed based on the node execution result to obtain a file processing result.

[0020] In one embodiment, the overall execution process includes a file parsing process branch and a file splitting process branch; the to-be-processed file is processed based on each of the execution process branches in the overall execution process to obtain a file processing result, including:

[0021] If the file download node in the overall execution process does not include a node execution result, executing the file download node in the file parsing process branch to obtain a first execution result;

[0022] If the file download node in the overall execution process includes the node execution result, obtaining the first execution result;

[0023] Execute the file parsing node in the file parsing process branch based on the first execution result to obtain a second execution result corresponding to the file parsing node;

[0024] The data slicing node in the file parsing process branch is executed based on the second execution result to obtain a third execution result corresponding to the data slicing node, and the third execution result is determined as the file processing result of the file parsing process branch.

[0025] In one embodiment, after executing the file download node in the file parsing process branch and obtaining the first execution result, the method further includes:

[0026] Obtaining the first execution result corresponding to the file download node;

[0027] The file splitting node in the file splitting process branch is executed based on the first execution result to obtain a fourth execution result corresponding to the file splitting node, and is determined as the file processing result of the file splitting process branch.

[0028] In one embodiment, the method further comprises:

[0029] If the real-time execution time of the node reaches the preset time threshold corresponding to the node, stop executing the node; or,

[0030] If the node state of the node is an abnormal state, the execution of the node is stopped.

[0031] In a second aspect, the present application also provides an execution process processing device. The device comprises:

[0032] A node determination module, used to obtain multiple execution processes corresponding to the file to be processed, and determine the reuse node corresponding to each of the execution processes;

[0033] A process merging module, used to merge the execution processes based on the reused nodes and the node dependencies in the execution processes to obtain a total execution process; the total execution process includes multiple execution process branches;

[0034] The process processing module is used to process the to-be-processed file based on each of the execution process branches in the overall execution process to obtain a file processing result.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the following steps are implemented:

[0036] Obtain multiple execution processes corresponding to the file to be processed, and determine the reuse nodes corresponding to each of the execution processes;

[0037] Based on the reused node and the node dependency relationship in each of the execution flows, each of the execution flows is merged to obtain a total execution flow; the total execution flow includes multiple execution flow branches;

[0038] The to-be-processed file is processed based on each of the execution flow branches in the overall execution flow to obtain a file processing result.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0040] Obtain multiple execution processes corresponding to the file to be processed, and determine the reuse nodes corresponding to each of the execution processes;

[0041] Based on the reused node and the node dependency relationship in each of the execution flows, each of the execution flows is merged to obtain a total execution flow; the total execution flow includes multiple execution flow branches;

[0042] The to-be-processed file is processed based on each of the execution flow branches in the overall execution flow to obtain a file processing result.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Obtain multiple execution processes corresponding to the file to be processed, and determine the reuse nodes corresponding to each of the execution processes;

[0045] Based on the reused node and the node dependency relationship in each of the execution flows, each of the execution flows is merged to obtain a total execution flow; the total execution flow includes multiple execution flow branches;

[0046] The to-be-processed file is processed based on each of the execution flow branches in the overall execution flow to obtain a file processing result.

[0047] The above-mentioned execution process processing method, device, computer equipment, storage medium and computer program product obtain multiple execution processes corresponding to the files to be processed and determine the corresponding reused nodes in the multiple execution processes. The server merges the multiple execution processes based on the node dependency relationship between the reused nodes and each node in each execution process to obtain a total execution process. Since the total execution process contains multiple execution process branches linked by reused nodes, the server can execute multiple execution processes corresponding to the files to be processed through the total execution process. Based on this, the server can execute each execution process branch in the total execution process as required, thereby processing the files to be processed and obtaining the file processing results. After executing a reused node in the total execution process once, when the related execution process branch is executed next time, there is no need to repeat the execution of the reused node, and the execution result of the reused node can be directly obtained, thereby reducing the consumption of computing resources and computing time, thereby improving the processing efficiency of batch transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] FIG1 is a diagram of an application environment for executing a process processing method in one embodiment;

[0050] FIG2 is a schematic diagram of a flow chart of an execution process processing method in one embodiment;

[0051] FIG3 is a schematic diagram of a process flow for obtaining the overall execution process steps in one embodiment;

[0052] FIG4 is a schematic flow chart of the steps of obtaining a file processing result in one embodiment;

[0053] FIG5 is a schematic diagram of a flow chart of an execution process processing method in another embodiment;

[0054] FIG6 is a structural block diagram of an execution flow processing device in one embodiment;

[0055] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] The execution process processing method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 can send the to-be-processed files and multiple execution processes corresponding to the to-be-processed files to the server 104. Based on multiple execution processes, the server 104 determines the reuse nodes corresponding to each execution process, and based on the reuse nodes and the node dependencies in each execution process, merges multiple execution force processes into a total execution process containing multiple execution process branches. The server 104 can process multiple execution processes for the to-be-processed files in the total execution process according to the execution instructions issued by the terminal 102, and obtain the file processing results. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0058] It should be understood that the process of batch transfer can be to send each line of data of a batch file to the bank core in real time, and the bank core completes the transaction based on the data. For example, multiple channels such as online banking and counters can send files containing multiple data to the bank. The bank can download the file locally, and then the bank will parse each line of data into the database according to the specified format, and then assemble each data in the database into a message and send it to the online system (bank core), and assemble a return response file based on the return message of the online system, and return it to the channel. In an example, the company system will assemble the employee name, bank card number, salary amount, etc. of each employee into data and put it into a file, and then upload it to the bank. The bank will download and parse the file into the warehouse, and then assemble it into a message based on the information in the warehouse (bank card number, salary amount, etc.) and send it to the bank's core system. The core system will complete the salary payment operation and assemble the results of each employee's salary payment into a response file to the company.

[0059] In an exemplary embodiment, as shown in FIG2 , a method for processing an execution process is provided, which is described by taking the method applied to the server 104 in FIG1 as an example, and includes the following steps S202 to S206 . Among them:

[0060] Step S202, obtaining multiple execution processes corresponding to the file to be processed, and determining the reuse node corresponding to each execution process.

[0061] The to-be-processed files are files used for business processing in various field systems, for example, they may be files to be processed in a batch transfer process, the batch transfer process may be a batch transfer process in a banking system, and the corresponding to-be-processed files may be files to be processed in the batch transfer process in the banking system. The execution process is a file execution process pre-configured for the batch transfer process, and the execution process may include multiple nodes arranged in sequence. A reuse node may be a node that can be repeatedly executed by multiple execution processes during the execution process. In one example, a reuse node may be a common node of multiple execution processes.

[0062] Specifically, the server can obtain multiple execution processes configured for the files to be processed from the terminal. For multiple execution processes, the server can determine common nodes from the nodes of each execution process, and use the common nodes as reused nodes. In one example, the number of reused nodes can be one or more, and the reused nodes exist in at least two execution processes. In an example, there are execution process 1 and execution process 2, execution process 1 contains three nodes A, B, and C arranged in the order of ABC, and execution process 2 contains three nodes BEF arranged in the order of BEF, and node B is determined to be a reused node. If there is also an execution process 3, and execution process 3 contains three nodes ABF arranged in order, it can be determined that nodes A and B of execution process 1 and execution process 3 are reused nodes, node B of execution process 1 and execution process 2 is a reused node, and node F of execution process 2 and execution process 3 is a reused node. In summary, nodes A, B, and F are reused nodes.

[0063] Step S204: Based on the reused nodes and the node dependencies in each execution process, each execution process is merged to obtain a total execution process.

[0064] The total execution process includes multiple execution process branches; one execution process branch can correspond to one execution process. The node dependency relationship is the execution order relationship between multiple nodes in the execution process. The server can execute the next node only after the previous node is executed, until all nodes in the execution process are executed.

[0065] Specifically, the server obtains the reused nodes in each execution process. The server traverses each execution process, obtains multiple nodes in the execution process except the reused nodes, and links the multiple nodes according to the node dependency relationship, and links the multiple nodes with the corresponding reused nodes. If the execution process contains multiple linked reused nodes, the reused nodes can be linked according to the node dependency relationship. After the server completes the traversal of multiple execution processes, the total execution process can be obtained, which contains multiple execution process branches, and each execution process branch corresponds to an execution process.

[0066] Step S206: Process the file to be processed based on each execution flow branch in the overall execution flow to obtain a file processing result.

[0067] Among them, the file processing result can be the result of processing the file to be processed by multiple execution process branches in the overall execution process respectively, the file processing result can be the result of processing the file to be processed by any execution process branch in the overall execution process, and the file processing result can be the processing result corresponding to at least one execution process set according to terminal requirements.

[0068] Specifically, the server obtains the execution flow corresponding to the to-be-processed file, and processes the to-be-processed file in the overall execution flow based on the execution flow to obtain the file processing result. If the reused node is processed multiple times during the processing, the reused node needs to be executed in the order of the flow for the first time, and the execution result of the reused node is obtained, and when the reused node is repeatedly executed subsequently, the execution result of the first execution can be directly used.

[0069] In the above-mentioned execution process processing method, multiple execution processes corresponding to the files to be processed are obtained, and the corresponding reused nodes in the multiple execution processes are determined. Based on the node dependency relationship between the reused nodes and each node in each execution process, the server merges the multiple execution processes to obtain a total execution process. Since the total execution process contains multiple execution process branches linked by reused nodes, the server can execute multiple execution processes corresponding to the files to be processed through the total execution process. Based on this, the server can execute each execution process branch in the total execution process as required, thereby processing the files to be processed and obtaining the file processing results. After executing a reused node in the total execution process once, the next time the related execution process branch is executed, there is no need to repeat the execution of the reused node, and the execution result of the reused node can be directly obtained, thereby reducing the consumption of computing resources and computing time, thereby improving the processing efficiency of batch transfer.

[0070] In an exemplary embodiment, each execution process includes multiple nodes arranged in an execution order, and the specific implementation process of the step of "determining the reused nodes corresponding to each execution process" includes:

[0071] Based on the nodes included in each execution process, the nodes in each execution process that meet the same preset conditions are determined as reused nodes of each execution process.

[0072] The preset same condition may be that two or more execution processes contain at least one same process node.

[0073] Specifically, the server may traverse each node included in multiple execution processes, determine that the same process nodes included in different execution processes are reused nodes, and obtain multiple reused nodes after the traversal is completed. The reused nodes can be reused by two or more execution processes.

[0074] In this embodiment, by presetting the same conditions and determining the reuse nodes corresponding to the multiple execution processes, the reuse nodes in each execution process can be quickly determined.

[0075] In an exemplary embodiment, the specific implementation process of the step of "determining the reuse nodes corresponding to each execution process" includes:

[0076] Nodes that meet the same preset conditions in the file parsing process and the file splitting process are determined as file download nodes, and the file download nodes are determined as reuse nodes.

[0077] Among them, each execution process includes a file parsing process and a file splitting process. The file parsing process includes a file downloading node, a file parsing node and a data sharding node arranged in a preset execution order. The file splitting process includes a file downloading node and a file splitting node arranged in a preset execution order.

[0078] Specifically, the server obtains the file download node, file parsing node and data sharding node in the file parsing process, and the server obtains the file download node and file splitting node in the file splitting process, determines that the file download nodes in the file parsing process and the file splitting process are nodes that meet the same preset conditions, and determines the file download node as a reused node.

[0079] In this embodiment, the reuse node can be determined as the file downloading machine by presetting the same conditions, which can improve the efficiency of determining the reuse node.

[0080] In an exemplary embodiment, as shown in FIG3 , the specific implementation process of the step of “merging each execution process based on the reused node and the node dependency relationship in each execution process to obtain the total execution process” includes steps S302 to S306. Among them:

[0081] Step S302, screening is performed in a preset thread pool to obtain a target thread.

[0082] Specifically, the thread pool contains multiple threads, and the server can call each thread and process the execution process through the thread. The server randomly obtains at least one thread in the preset thread pool and uses it as the target thread.

[0083] Step S304: merging the reused nodes included in each execution process through the target thread to obtain an initial overall process.

[0084] Specifically, the server can merge multiple reused nodes in each execution process in the target thread or through the target thread to obtain an initial total process, which can include all reused nodes, the association between reused nodes, and nodes in some execution processes linked to reused nodes. The server can determine an execution process as a reference process, and traverse each other execution process, merge the reused nodes in other execution processes with the reference process, that is, the server links the reused nodes in other execution processes with the reference process, and obtains the initial total process consisting of the reference process and each reused node after completing the traversal.

[0085] Step S306: Based on the node dependency relationship corresponding to each execution process, the non-reusable nodes corresponding to the execution process are linked to the initial overall process to obtain the overall execution process.

[0086] Among them, the non-reused nodes are other nodes in the execution process except the reused nodes.

[0087] Specifically, the server traverses each other execution process, links the non-reused nodes in the other execution processes with the initial total process, and obtains the total execution process composed of each execution process after completing the traversal.

[0088] In this embodiment, multiple execution processes are merged through the target thread pool, the reused nodes are merged with the benchmark process to obtain the initial total process, and the non-reuse nodes of other execution processes are linked to the initial total process to obtain the total execution process, which can improve the accuracy and efficiency of generating the total execution process.

[0089] In an exemplary embodiment, the specific implementation process of the step of "processing the to-be-processed file based on each execution flow branch in the overall execution flow to obtain the file processing result" includes:

[0090] If the reused node has a node execution result of the target execution flow branch, other execution flow branches except the target execution flow branch are executed respectively based on the node execution result to obtain the file processing result.

[0091] Specifically, the server judges the reused node, and when there is a node execution result of the target execution process branch in the reused node, other execution process branches except the target execution process branch can be directly processed according to the node execution result to obtain the file processing result.

[0092] It should be understood that the reused node has the node execution result of the target execution process branch, indicating that the server has executed the target execution process branch, and the reused node contains the node execution result. If the server needs to execute other execution process branches, it can directly call the node execution result of the target execution process branch, process the corresponding other execution process branches, and obtain the file processing results corresponding to each other execution process branch, so there is no need to repeatedly execute the reused node.

[0093] In this embodiment, other execution process branches are processed through the node execution result of the target execution process branch, which can improve the efficiency of processing process branches and achieve the technical effect of reducing resource consumption.

[0094] In an exemplary embodiment, the overall execution process includes a file parsing process branch and a file splitting process branch, as shown in FIG4 , and the specific implementation process of the step “processing the to-be-processed file based on each execution process branch in the overall execution process to obtain the file processing result” includes steps S402 to S408. Among them:

[0095] Step S402: If the file download node in the overall execution process does not include a node execution result, the file download node in the file parsing process branch is executed to obtain a first execution result.

[0096] Specifically, when the server executes the file download node, it determines whether there is a node execution result for the file download node in the overall execution process. If the file download node does not contain a node execution result, that is, it does not contain the downloaded file to be processed, the server executes the file download node in the execution order of the file parsing process branch to obtain the first execution result corresponding to the file download node.

[0097] Step S404: if the file download node in the overall execution process includes a node execution result, a first execution result is obtained.

[0098] Specifically, when executing the file download node, the server determines whether there is a node execution result for the file download node in the overall execution process. If the file download node contains the node execution result, that is, contains the downloaded file to be processed, the first execution result is directly obtained.

[0099] Step S406: Execute the file parsing node in the file parsing process branch based on the first execution result to obtain a second execution result corresponding to the file parsing node.

[0100] Specifically, the server uses the first execution result as the input of the file parsing node in the file parsing process branch, executes the file parsing node, parses the file to be processed, obtains a data table corresponding to the file to be processed, and determines that the data table is the second execution result.

[0101] Step S408: Execute the data sharding node in the file parsing process branch based on the second execution result, obtain a third execution result corresponding to the data sharding node, and determine it as the file processing result of the file parsing process branch.

[0102] Specifically, the server uses the second execution result as the input of the data sharding node in the file parsing process branch, and executes the data sharding node to perform sharding processing on the data in the data table to obtain the sharded data and determine it as the file processing result.

[0103] In this embodiment, through the execution result of the file download node, the files and data tables to be processed are processed step by step according to the execution order in the file parsing process branch to obtain the final file processing result, which can reduce the execution times of the reused nodes and improve the efficiency of obtaining the file processing results.

[0104] In an exemplary embodiment, after the step of "executing the file download node in the file parsing process branch to obtain the first execution result", the method further includes:

[0105] Obtain a first execution result corresponding to the file download node; execute the file splitting node in the file splitting process branch based on the first execution result, obtain a fourth execution result corresponding to the file splitting node, and determine it as the file processing result of the file splitting process branch.

[0106] Specifically, the server obtains the first execution result corresponding to the file download node, and uses the first execution result as the input of the file splitting node in the file splitting process branch. The server executes the file splitting node, splits the file to be processed, obtains the split data, and determines it as the file processing result.

[0107] In one example, the file processing result may be the processing result corresponding to the file splitting process branch; the file processing result may be the processing result corresponding to the file parsing process branch; the file processing result may be the processing result corresponding to both the file splitting process branch and the file parsing process branch. The present application embodiment is not limited, and the server may define the range of file processing results according to actual needs.

[0108] In this embodiment, the file splitting node in the file splitting process branch is directly executed through the existing first execution result to obtain the file processing result, which can improve the efficiency of obtaining the file processing result.

[0109] In an exemplary embodiment, the method further includes: if the real-time execution time of the node reaches a preset time threshold corresponding to the node, then stopping the execution of the node; or, if the node status of the node is an abnormal state, then stopping the execution of the node.

[0110] The preset time-consuming threshold of each node may be different. Abnormal states may include abnormalities such as node unresponsiveness, deadlock, and no output.

[0111] Specifically, the server monitors each node and determines the real-time execution time of each node. If the real-time execution time exceeds the preset time threshold corresponding to the node, the execution timeout of the node is determined, and the server interrupts the execution of the execution process and stops executing the node. The execution results of the node previously executed can be retained in the previous node. In addition, the server monitors the node status of each node. If the node status is determined to be an abnormal state such as no response, deadlock, no output, etc., the server interrupts the execution of the execution process and stops executing the node.

[0112] In this embodiment, by determining the real-time execution time and the node status, the execution node can be stopped in time, thereby improving the security of the system.

[0113] As shown in Figure 5, the following describes the execution process processing method in detail in conjunction with a specific embodiment. In the banking system, the batch transfer usually includes the execution processes of file splitting, file parsing, file processing, and file return.

[0114] The file parsing process usually includes three nodes: file download, file parsing, and data sharding. The file splitting process includes two nodes: file download and file splitting. In order to reuse the file download node in the file parsing process and the file splitting process, and to use the node to expand other processes, it is necessary to support the sequential combination of nodes.

[0115] Since the files to be processed need to be complete before processing, the files will eventually be downloaded to only one node, and the subsequent file parsing process or file splitting process needs to be performed based on the file download node downloaded to the local computer. Therefore, the file download node and the file parsing node, as well as the file splitting node, are strongly associated, and the server can control the order of the file download node and the file parsing node. The result of the file download node can affect whether the file parsing node is executed. If the download is successful, the file path of the file to be processed downloaded to the local computer needs to be passed to the file parsing. If the download is not successful, there is no need to perform file parsing.

[0116] Specifically, the following points are included:

[0117] 1. Resources are controllable. The threads of the execution process are obtained from the thread pool, which can ensure the controllability of resource occupation and prevent other execution processes (such as data processing processes, file return processes, etc.) from not being able to obtain resources. File download nodes, file parsing nodes, file splitting nodes, and data sharding nodes can be executed in order in the thread.

[0118] 2. Node reuse. The file download node is a common node for the file parsing process and the file splitting process and can be reused.

[0119] 3. Process execution order. In one example, the server uses the next method to organize the file download node, file parsing node, and data sharding node in order to obtain an execution process in the form of a linked list. In the execution process, the next node can only be executed after the previous node is executed.

[0120] 4. Parameter callback. The execution result of each node needs to be passed to the next execution node. For example, if the execution result of the file download node contains the local file path, the local file path will be passed as an input parameter to the file parsing node. The file parsing node will find the file according to the path and then parse it into the database. Then the file parsing result can include a data table, which will be passed as an input parameter to the data sharding node. If a single node executes abnormally, the execution process will be interrupted. If there are new nodes, they can also be added to the execution process in sequence.

[0121] 5. High performance. The CAS (Compare and Swap) mechanism ensures the accuracy of node status changes in multi-threaded situations and avoids repeated execution of tasks.

[0122] 6. Timeout control. The entire process sets a timeout. If one of the file download node, file parsing node, and data sharding node times out during execution, the subsequent processes will no longer be executed.

[0123] In this embodiment, the dependency relationship between nodes can be used to improve component reuse rate and free combination of nodes, and to ensure that resources are controllable and add timeout control.

[0124] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0125] Based on the same inventive concept, the embodiment of the present application also provides an execution process processing device for implementing the execution process processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more execution process processing device embodiments provided below can refer to the limitations of the execution process processing method above, and will not be repeated here.

[0126] In an exemplary embodiment, as shown in FIG6 , an execution process processing device 600 is provided, comprising: a node determination module 601 , a process merging module 602 and a process processing module 603 , wherein:

[0127] The node determination module 601 is used to obtain multiple execution processes corresponding to the files to be processed, and determine the reuse nodes corresponding to each execution process;

[0128] A process merging module 602 is used to merge the execution processes based on the reused nodes and the node dependencies in the execution processes to obtain a total execution process; the total execution process includes multiple execution process branches;

[0129] The process processing module 603 is used to process the files to be processed based on each execution process branch in the overall execution process to obtain a file processing result.

[0130] Furthermore, each execution process includes multiple nodes arranged in an execution order, and the node determination module 601 is specifically used to: based on the nodes included in each execution process, determine the nodes in each execution process that meet the same preset conditions as reused nodes of each execution process.

[0131] Furthermore, each execution process includes a file parsing process and a file splitting process; the file parsing process includes a file downloading node, a file parsing node and a data sharding node arranged in a preset execution order; the file splitting process includes a file downloading node and a file splitting node arranged in a preset execution order; the node determination module 601 is specifically used to: determine that the nodes in the file parsing process and the file splitting process that meet the same preset conditions are file downloading nodes, and determine that the file downloading nodes are reused nodes.

[0132] Furthermore, the process merging module 602 is specifically used to: screen in a preset thread pool to obtain a target thread; merge the multiplexed nodes contained in each execution process through the target thread to obtain an initial total process; based on the node dependency corresponding to each execution process, link the non-multiplexed nodes corresponding to the execution process to the initial total process to obtain the total execution process.

[0133] Furthermore, the process processing module 603 is specifically used to: if the reused node has a node execution result of the target execution process branch, then based on the node execution result, execute other execution process branches except the target execution process branch respectively to obtain a file processing result.

[0134] Furthermore, the overall execution process includes a file parsing process branch and a file splitting process branch, and the process processing module 603 is specifically used for: if the file download node in the overall execution process does not contain a node execution result, then execute the file download node in the file parsing process branch to obtain a first execution result; if the file download node in the overall execution process contains a node execution result, then obtain the first execution result; based on the first execution result, execute the file parsing node in the file parsing process branch to obtain a second execution result corresponding to the file parsing node; based on the second execution result, execute the data sharding node in the file parsing process branch to obtain a third execution result corresponding to the data sharding node, and determine it as the file processing result of the file parsing process branch.

[0135] Furthermore, the process processing module 603 is specifically used to: obtain a first execution result corresponding to the file download node; execute the file splitting node in the file splitting process branch based on the first execution result, obtain a fourth execution result corresponding to the file splitting node, and determine it as the file processing result of the file splitting process branch.

[0136] Furthermore, the device includes a node stopping module, which is specifically used to: stop executing the node if the real-time execution time of the node reaches a preset time threshold corresponding to the node; or stop executing the node if the node status of the node is an abnormal state.

[0137] Each module in the above-mentioned execution process processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0138] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store total execution process data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an execution process processing method is implemented.

[0139] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the following steps are implemented:

[0141] Obtain multiple execution processes corresponding to the files to be processed, and determine the reuse nodes corresponding to each execution process;

[0142] Based on the reused nodes and the node dependencies in each execution process, each execution process is merged to obtain a total execution process; the total execution process includes multiple execution process branches;

[0143] The files to be processed are processed based on each execution flow branch in the overall execution flow to obtain a file processing result.

[0144] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0145] Obtain multiple execution processes corresponding to the files to be processed, and determine the reuse nodes corresponding to each execution process;

[0146] Based on the reused nodes and the node dependencies in each execution process, each execution process is merged to obtain a total execution process; the total execution process includes multiple execution process branches;

[0147] The files to be processed are processed based on each execution flow branch in the overall execution flow to obtain a file processing result.

[0148] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0149] Obtain multiple execution processes corresponding to the files to be processed, and determine the reuse nodes corresponding to each execution process;

[0150] Based on the reused nodes and the node dependencies in each execution process, each execution process is merged to obtain a total execution process; the total execution process includes multiple execution process branches;

[0151] The files to be processed are processed based on each execution flow branch in the overall execution flow to obtain a file processing result.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for executing a process, characterized in that: The method comprises: Obtain multiple execution processes corresponding to the file to be processed, and determine the reuse nodes corresponding to each of the execution processes; Based on the reused node and the node dependency relationship in each of the execution flows, each of the execution flows is merged to obtain a total execution flow; the total execution flow includes multiple execution flow branches; The to-be-processed file is processed based on each of the execution flow branches in the overall execution flow to obtain a file processing result.

2. The method according to claim 1, characterized in that: Each of the execution processes includes a plurality of nodes arranged in an execution order; The determining of the reuse nodes corresponding to the execution processes includes: Based on the nodes included in each of the execution flows, the nodes in each of the execution flows that meet the same preset conditions are determined as reused nodes of each of the execution flows.

3. The method according to claim 2, characterized in that Each of the execution processes includes a file parsing process and a file splitting process; the file parsing process includes a file downloading node, a file parsing node and a data sharding node arranged in a preset execution order; The file splitting process includes file downloading nodes and file splitting nodes arranged in a preset execution order; The determining of the reuse nodes corresponding to the execution processes includes: The nodes satisfying the same preset conditions in the file parsing process and the file splitting process are determined as the file downloading nodes, and the file downloading nodes are determined as the reuse nodes.

4. The method according to claim 2, characterized in that: Based on the reused node and the node dependency relationship in each of the execution flows, each of the execution flows is merged to obtain a total execution flow, including: Screen the preset thread pool to get the target thread; By using the target thread, each reused node included in each execution process is merged to obtain an initial overall process; Based on the node dependency relationship corresponding to each of the execution processes, the non-reuse nodes corresponding to the execution processes are linked to the initial overall process to obtain the overall execution process.

5. The method according to claim 1, characterized in that The step of processing the to-be-processed file based on each of the execution flow branches in the overall execution flow to obtain a file processing result includes: If the reuse node has a node execution result of a target execution flow branch, the other execution flow branches except the target execution flow branch are respectively executed based on the node execution result to obtain a file processing result.

6. The method according to claim 5, characterized in that The overall execution process includes a file parsing process branch and a file splitting process branch; The step of processing the to-be-processed file based on each of the execution flow branches in the overall execution flow to obtain a file processing result includes: If the file download node in the overall execution process does not include a node execution result, executing the file download node in the file parsing process branch to obtain a first execution result; If the file download node in the overall execution process includes the node execution result, obtaining the first execution result; Execute the file parsing node in the file parsing process branch based on the first execution result to obtain a second execution result corresponding to the file parsing node; The data slicing node in the file parsing process branch is executed based on the second execution result to obtain a third execution result corresponding to the data slicing node, and the third execution result is determined as the file processing result of the file parsing process branch.

7. The method according to claim 6, characterized in that After executing the file download node in the file parsing process branch and obtaining the first execution result, the method further includes: Obtaining the first execution result corresponding to the file download node; The file splitting node in the file splitting process branch is executed based on the first execution result to obtain a fourth execution result corresponding to the file splitting node, and is determined as the file processing result of the file splitting process branch.

8. The method according to claim 2, characterized in that: The method further comprises: If the real-time execution time of the node reaches the preset time threshold corresponding to the node, stop executing the node; or, If the node state of the node is an abnormal state, the execution of the node is stopped.

9. An execution process processing device, characterized in that: The device comprises: A node determination module, used to obtain multiple execution processes corresponding to the file to be processed, and determine the reuse node corresponding to each of the execution processes; A process merging module, used to merge the execution processes based on the reused nodes and the node dependencies in the execution processes to obtain a total execution process; the total execution process includes multiple execution process branches; The process processing module is used to process the to-be-processed file based on each of the execution process branches in the overall execution process to obtain a file processing result.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images