A task processing method, electronic device, storage medium, and program product

Abstract

This application discloses a task processing method, electronic device, storage medium, and program product, relating to the field of distributed technology. The method includes: a scheduling node constructing corresponding tasks to be executed based on product data and multiple processing procedures of each product to be processed that meet preset splitting processing conditions; writing each task to be executed into a task to be executed list; and sending the task to be executed list and pre-set target scheduling parameters for splitting services to a management center node; the management center node writing the tasks to be executed from the task to be executed list into a task to be executed queue according to preset writing rules; allocating execution nodes to the tasks to be executed dequeued from the task to be executed queue based on the target scheduling parameters; and distributing the dequeued tasks to the corresponding execution nodes; and the execution nodes parsing product data and multiple processing procedures from the received tasks to be executed, and executing multiple processing procedures based on the product data, thereby improving task processing efficiency and scalability.

Description

Technical Field

[0001] This application relates to the field of distributed technology, and in particular to a task processing method, electronic device, storage medium, and program product. Background Technology

[0002] Currently, the data splitting business for financial products is mainly implemented based on the database job scheduling function. That is, the database job scheduling component is used to periodically filter out products that meet the data splitting conditions from the data table. Then, the database stored procedures are used to assign execution identifiers to the products to be processed, and structured query language statements and stored procedures are written according to preset logic to execute the splitting business operations corresponding to the products to be processed in sequence, such as calculation and update operations.

[0003] However, with the expansion of financial product business, the amount of product data has grown explosively. The serial execution of the above solutions is difficult to cope with the parallel processing needs of large amounts of data, resulting in low task processing efficiency and failing to meet the business needs of efficient data splitting and processing. Furthermore, when the business needs of data splitting for financial products change, a large number of modifications are required to the database stored procedures and job scheduling configurations. The modification process is complex and prone to causing other functional abnormalities, resulting in poor scalability and difficulty in adapting to the ever-expanding needs of financial product business. Summary of the Invention

[0004] This application provides a task processing method, electronic device, storage medium, and program product, which realizes the data splitting task processing function of financial products, so as to solve the problems of low task processing efficiency and poor scalability in the prior art.

[0005] In a first aspect, embodiments of this application provide a task processing method applied to a task processing system. The task processing system includes a scheduling node, a management center node, and at least two execution nodes. The method includes: the scheduling node extracts product data of each product to be processed that meets preset splitting processing conditions from a product information table and a splitting parameter table, and extracts multiple processing procedures corresponding to each product to be processed from a process information table based on the product code of each product to be processed; the product data includes product codes; the scheduling node constructs tasks to be executed for each product to be processed based on the product data and multiple processing procedures, writes each task to be executed into a task to be executed list, and sends the task to be executed list and target scheduling parameters pre-set for splitting business to the management center node; the management center node writes the tasks to be executed in the task to be executed list into a task to be executed queue according to preset writing rules, assigns execution nodes to the tasks to be executed dequeued from the task to be executed queue based on the target scheduling parameters, and distributes the dequeued tasks to the corresponding execution nodes; the execution nodes parse the product data and multiple processing procedures from the received tasks to be executed, and execute the multiple processing procedures based on the product data.

[0006] Secondly, embodiments of this application provide a task processing system, including a scheduling node, a management center node, and at least two execution nodes. The scheduling node is used to extract product data for each product to be processed that meets preset splitting processing conditions from a product information table and a splitting parameter table, and to extract multiple processing procedures corresponding to each product to be processed from a process information table based on the product code of each product to be processed. The product data includes product codes. Based on the product data and multiple processing procedures of each product to be processed, it constructs tasks to be executed for the corresponding product to be processed, writes each task to be executed into a task to be executed list, and sends the task to be executed list and target scheduling parameters pre-set for the splitting business to the management center node. The management center node is used to write the tasks to be executed from the task to be executed list into a task to be executed queue according to preset writing rules, allocate execution nodes to the tasks to be executed dequeued from the task to be executed queue based on the target scheduling parameters, and distribute the dequeued tasks to the corresponding execution nodes. The execution nodes are used to parse product data and multiple processing procedures from the received tasks to be executed, and execute multiple processing procedures based on the product data.

[0007] Thirdly, embodiments of this application provide an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the task processing method of any embodiment of this application.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the task processing method as described in any embodiment of this application.

[0009] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements a task processing method as described in any embodiment of this application.

[0010] In this embodiment, the scheduling node extracts product data for each product to be processed that meets preset splitting processing conditions from the product information table and the splitting parameter table, and extracts multiple processing procedures corresponding to each product to be processed from the process information table based on the product code of each product to be processed; the product data includes the product code; the scheduling node constructs the tasks to be executed for each product to be processed based on the product data and multiple processing procedures, writes each task to be executed into the task to be executed list, and sends the task to be executed list and the target scheduling parameters pre-set for the splitting business to the management center node; the management center node writes the tasks to be executed in the task to be executed list into the task to be executed queue according to preset writing rules, assigns execution nodes to the tasks to be executed dequeued from the task to be executed queue based on the target scheduling parameters, and distributes the dequeued tasks to the corresponding execution nodes; the execution node parses the product data and multiple processing procedures from the received tasks to be executed, and executes multiple processing procedures based on the product data. In the above technical solution, the asynchronous processing function of data splitting tasks for financial products can be realized through the scheduling node, management center node, and at least two execution nodes in the task processing system. This eliminates the need to rely on database job scheduling components and stored procedures. The at least two execution nodes enable asynchronous parallel processing of data splitting tasks, effectively addressing the parallel processing needs of large amounts of data, thereby improving task processing efficiency and meeting the business requirements for efficient data splitting processing. Furthermore, the task processing system supports flexible expansion of execution nodes, allowing for the addition of execution nodes as the scale of financial product business grows. When the data splitting business requirements of financial products change, only the relevant algorithm and rule configurations of the task processing module in the asynchronous task framework need to be modified to quickly adapt to business changes, thereby improving scalability, adapting to the ever-evolving needs of financial product business, and reducing system maintenance costs. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a task processing method provided in an embodiment of this application; Figure 2 This is another flowchart illustrating the task processing method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the task processing system provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0014] It should be noted that the terms "first," "second," "target," and "original," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprising," "having," and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] Figure 1 This is a flowchart illustrating a task processing method provided in an embodiment of this application. This embodiment can be applied to scenarios in financial product data splitting operations where data splitting tasks need to be processed. The task processing method provided in this embodiment can be executed by node devices in the task processing system provided in this application. For example, a node device can be an electronic device such as a computer or server.

[0016] In one specific embodiment, the task processing system may include a scheduling node, a management center node, and at least two execution nodes.

[0017] The scheduling node is responsible for coordinating task selection, processing orchestration, and scheduling strategy formulation. Before executing the method of this application embodiment, the scheduling node is built in the target container. That is, the asynchronous task framework package is imported into the code of the scheduling node in the form of an import package to implement the business logic of data splitting and scheduling, and the code of the scheduling node is deployed in the target container (such as a platform as a service container). Furthermore, the task processing system may include one or more scheduling nodes.

[0018] The asynchronous task framework is a pre-developed software architecture for implementing asynchronous task processing, scheduling, and distribution. It features functions such as task queue management, task priority scheduling, and execution node load monitoring and balancing. It enables efficient and orderly task processing without relying on external third-party frameworks or database products, thereby effectively avoiding security risks such as data leakage and technological monopolies, and effectively ensuring the security of financial product business data and financial information security. The asynchronous task framework package is a toolkit that encapsulates the asynchronous task framework.

[0019] The management center node, acting as a scheduling gateway, is responsible for connecting scheduling nodes and execution nodes, as well as distributing tasks. Before executing the method of this application embodiment, the management center node is built in the target container. That is, the asynchronous task framework package is imported into the code of the management center node in the form of an import package to implement the business logic of data splitting scheduling management, and the code of the management center node is deployed in the target container.

[0020] The execution node is responsible for receiving and distributing data splitting tasks, calling the corresponding processing modules and algorithms, and completing specific operations such as data calculation, updating, and logging during the data splitting process. The high availability design of the asynchronous task framework ensures the stability and accuracy of task execution. Before executing the method of this application embodiment, the execution node is built in the target container. That is, the asynchronous task framework package is imported into the code of the execution node in the form of an import package to implement the business logic of data splitting execution, and the code of the execution node is deployed in the target container.

[0021] The entity executing this method can be a node device in a task processing system. See also... Figure 1 The task processing method in this embodiment includes, but is not limited to, the following steps: S110, the scheduling node extracts product data of each product to be processed that meets the preset splitting processing conditions from the product information table and the splitting parameter table, and extracts multiple processing processes corresponding to each product to be processed from the process information table based on the product code of each product to be processed.

[0022] The product information table stores basic information about financial products, including fields such as product identifier, product code, product status, and asset type. In this embodiment, the financial product can be a fund product. In the business scenario of fund products, the data splitting task is an operation to process and split the relevant data of fund products based on preset business rules.

[0023] The splitting parameter table is used to store parameter information related to the data splitting task, including fields such as product code, splitting type, splitting date, region code, and running status. The running status can include initialization, running, running successfully, and running failed.

[0024] The preset splitting processing conditions are data splitting processing conditions set in advance based on actual business needs; the products to be processed are financial products in the product information table that meet the preset splitting processing conditions; the product data is the combined data of the products to be processed, including product codes and other data required for the data splitting task.

[0025] The process information table is used to store the processing procedures corresponding to the product code, including fields such as process identifier, product code, and specific processing procedure. One product code corresponds to one or more processing procedures. The processing procedure, or processing step, is the processing operation corresponding to the data splitting task of a specific product.

[0026] Specifically, the scheduling node can directly access the database; the user sends the database to be processed, the split initiation command, and the preset split processing conditions to the scheduling node through the user terminal; then, the scheduling node receives the database to be processed, the split initiation command, and the preset split processing conditions sent by the user terminal, and periodically scans the product information table and split parameter table in the database to be processed according to the preset scanning time interval, and extracts the product data of each product to be processed that meets the preset split processing conditions from the product information table and the split parameter table. At this time, the product data is a combination of the data in the product information table and the split parameter table, such as product code and other data required by the data splitting task.

[0027] Then, the scheduling node extracts the process identifier and specific processing procedure of each processing procedure corresponding to each product to be processed from the process information table based on the product code of each product to be processed, thereby extracting multiple processing procedures corresponding to each product to be processed.

[0028] Optionally, the preset splitting processing conditions may include the splitting business being in the on state, the product not existing in the splitting parameter table and meeting the preset splitting business rule conditions.

[0029] Among them, the preset splitting business rule conditions are the business rules that users set in advance for the data splitting task of financial products that need to be met, such as the product status not being paused or the product running status not being running.

[0030] Specifically, the scheduling node can query the switch table based on the service identifier of the split service to obtain the switch status corresponding to the split service. The switch table includes fields such as service identifier and switch status. If the switch status of the split service is on and there are no products in the running status in the split parameter table, it indicates that the product data splitting task can be performed. Then, the product data of products that meet the preset split service rules (i.e. products to be processed) can be obtained from the product information table and the split parameter table. This can improve the accuracy of the determination of products to be processed and provide an accurate data foundation for the subsequent determination of tasks to be executed.

[0031] If a product in the splitting parameter table is in a running state, it indicates that another user's splitting task is being executed. In this case, a prompt message "Other users are splitting, please wait" can be generated and sent to the user's terminal to remind the user to try again later. This can avoid data anomalies caused by executing splitting tasks of different users at the same time.

[0032] Optionally, the preset splitting processing conditions also include data compliance in the product information table and splitting parameter table. This can ensure the accuracy and consistency of data during the data splitting process, thereby avoiding abnormal splitting results due to data errors, protecting the legitimate rights and interests of financial product holders, and reducing the compliance risks of financial product managers.

[0033] Optionally, the task management system also includes a data entry node; the data entry node obtains various external data related to financial products from external systems, that is, the raw data related to data splitting, and writes it into the corresponding data table in the database after standardization, thereby obtaining the product information table and the splitting parameter table. This process is called external data entry.

[0034] S120: The scheduling node constructs the corresponding tasks to be executed for each product to be processed based on the product data and multiple processing procedures of each product to be processed. It writes each task to be executed into the task to be executed list and sends the task to be executed list and the target scheduling parameters pre-set for splitting the business to the management center node.

[0035] Among them, the tasks to be executed are data splitting tasks corresponding to the products to be processed, which are constructed based on the product data and processing procedures of the products to be processed; the list of tasks to be executed is used to store the tasks to be executed.

[0036] The target scheduling parameters are the scheduling parameters corresponding to the split services, including scheduling policies and timeout durations.

[0037] Specifically, after the scheduling node obtains the product data and multiple processing procedures for each product to be processed, it constructs the corresponding tasks to be executed for each product to be processed based on the product data and multiple processing procedures. For example, it selects any product to be processed as the current product to be processed. For the multiple processing procedures corresponding to the current product to be processed, it encapsulates the process identifier and specific processing procedure of each processing procedure according to a preset format (such as key-value pair format), and uses the encapsulated process identifier and specific processing procedure of each processing procedure as an element to construct the process list of the current product to be processed. Then, it encapsulates the product data and process list of the current product to be processed according to a preset format, and names the encapsulated data as the tasks to be executed for the current product to be processed. Then, it iterates through each product to be processed and repeats the above process to obtain the tasks to be executed for each product to be processed.

[0038] Next, the scheduling node writes the tasks to be executed for each product to be processed into the task list. Then, it obtains the target scheduling parameters, such as the scheduling policy, which is pre-set for splitting the business. At this time, the scheduling policy specifies the cluster of execution nodes that will process the data splitting tasks, and sends the task list and target scheduling parameters to the management center node.

[0039] S130. The management center node writes the tasks to be executed from the task list to the task queue according to the preset writing rules, assigns execution nodes to the tasks to be executed dequeued from the task queue based on the target scheduling parameters, and distributes the dequeued tasks to the corresponding execution nodes.

[0040] The preset write rules are pre-defined rules for writing tasks to the task queue, such as writing tasks in descending order of priority. The task queue is used to store data splitting tasks waiting to be executed by the execution node.

[0041] Specifically, the management center node receives the list of tasks to be executed and the target scheduling parameters sent by the scheduling node, and writes the tasks to be executed in the list of tasks to be executed into the task queue according to the preset writing rules. For example, the splitting parameter table also includes a task priority field. Based on the product code of the task to be executed, the task priority of the corresponding task to be executed is obtained from the splitting parameter table, and the tasks to be executed in the list of tasks to be executed are written into the task queue in descending order of task priority. That is, the highest priority task to be executed is located at the head of the task queue, and the lowest priority task to be executed is located at the tail of the task queue.

[0042] Next, the management center node dequeues the task at the head of the queue and assigns execution nodes to the dequeueed tasks based on the target scheduling parameters. For example, the execution node cluster for processing data splitting tasks is determined based on the scheduling policy in the target scheduling parameters, i.e., the target execution node cluster. An execution node is selected from the target execution node cluster according to a preset selection policy as the execution node for the dequeue. The preset selection policy at this time can be to select the execution node with the lowest network latency. Then, the dequeueed tasks are distributed to the corresponding execution nodes.

[0043] Afterwards, the management center node continues to dequeue the tasks at the head of the queue, and continues to assign execution nodes to the dequeued tasks and distribute the corresponding tasks to the corresponding execution nodes.

[0044] S140, The execution node parses the product data and multiple processing procedures from the received tasks to be executed, and executes the multiple processing procedures based on the product data.

[0045] Specifically, the execution node receives the tasks to be executed distributed by the management center node, parses the received tasks to obtain product data and multiple processing procedures, and then calls the corresponding processing modules and algorithms to execute multiple processing procedures based on the product data, completing specific operations such as data calculation, updating, and logging during the data splitting process.

[0046] Optionally, the task processing system also includes a task monitoring node; the task monitoring node is responsible for tracking the entire lifecycle status of a task in real time, from data entry to completion; the task monitoring node automatically records key nodes such as task start, timeout, exception, and completion, enabling rapid problem localization.

[0047] Optionally, when the execution node starts executing the task to be executed, it can generate an execution start message based on the product code and send the execution start message to the task monitoring node through the management center node; the task monitoring node parses the product code from the received execution start message and updates the running status of the corresponding product in the split parameter table to running based on the product code.

[0048] Optionally, after completing multiple processing steps, the execution node can report the task execution results to the management center node. The task execution results include the product code, task execution status (i.e., execution successful or failed), execution time, execution result details, and the process execution status corresponding to each process identifier. After receiving the task execution results from the execution node, the management center node synchronizes the task execution results to the scheduling node and the task monitoring node. After receiving the task execution results, the task monitoring node updates the running status of the corresponding product in the split parameter table based on the product code and task execution status in the task execution results.

[0049] Optionally, the management center node extracts the timeout duration from the target scheduling parameters. If no task execution result is received from the execution node within the timeout duration after the corresponding task is distributed to the execution node, the task is determined to have timed out. In this case, the timed-out task is retried based on the exponential backoff retry mechanism. If the task execution status in the task execution result is "execution failed," an early warning message is generated based on the product code of the task and sent to the administrator's email address to notify the administrator to intervene manually in a timely manner. This achieves the function of timely early warning of anomalies. At this time, the administrator can quickly locate the abnormal link and the cause of the anomaly through detailed execution logs and status records, effectively shortening the problem handling time and reducing the data splitting delay and loss caused by task anomalies.

[0050] The technical solution of this application embodiment involves a scheduling node extracting product data for each product to be processed that meets preset splitting processing conditions from a product information table and a splitting parameter table, and extracting multiple processing procedures corresponding to each product to be processed from a process information table based on the product code of each product to be processed; the product data includes product codes; the scheduling node constructs tasks to be executed for each product to be processed based on the product data and multiple processing procedures, writes each task to be executed into a task to be executed list, and sends the task to be executed list and the target scheduling parameters pre-set for the splitting business to the management center node; the management center node writes the tasks to be executed in the task to be executed list into a task to be executed queue according to preset writing rules, assigns execution nodes to the tasks to be executed dequeued from the task to be executed queue based on the target scheduling parameters, and distributes the dequeued tasks to the corresponding execution nodes; the execution node parses the product data and multiple processing procedures from the received tasks to be executed, and executes multiple processing procedures based on the product data. In the above technical solution, the asynchronous processing function of data splitting tasks for financial products can be realized through the scheduling node, management center node, and at least two execution nodes in the task processing system. This eliminates the need to rely on database job scheduling components and stored procedures. The at least two execution nodes enable asynchronous parallel processing of data splitting tasks, effectively addressing the parallel processing needs of large amounts of data, thereby improving task processing efficiency and meeting the business requirements for efficient data splitting processing. Furthermore, the task processing system supports flexible expansion of execution nodes, allowing for the addition of execution nodes as the scale of financial product business grows. When the data splitting business requirements of financial products change, only the relevant algorithm and rule configurations of the task processing module in the asynchronous task framework need to be modified to quickly adapt to business changes, thereby improving scalability, adapting to the ever-evolving needs of financial product business, and reducing system maintenance costs.

[0051] The following further describes a task processing method provided by an embodiment of this application. Figure 2 This is another schematic flowchart of the task processing method provided in this application. This application's embodiments are optimizations based on the above embodiments. See also... Figure 2 The method in this embodiment includes, but is not limited to, the following steps: S210, the scheduling node extracts product data of each product to be processed that meets the preset splitting processing conditions from the product information table and the splitting parameter table, and extracts multiple processing processes corresponding to each product to be processed from the process information table based on the product code of each product to be processed.

[0052] S220: The scheduling node constructs the corresponding tasks to be executed for each product to be processed based on the product data and multiple processing procedures of each product to be processed. It writes each task to be executed into the task to be executed list and sends the task to be executed list and the target scheduling parameters pre-set for splitting the business to the management center node.

[0053] Specifically, the scheduling node constructs the tasks to be executed for each product to be processed based on the product data and multiple processing procedures, including Sa1-Sa3: Sa1: The scheduling node selects any one of the products to be processed as the current product to be processed, obtains the parent process identifier of each processing process corresponding to the current product to be processed from the process information table, and determines the dependency relationship between multiple processing processes based on the parent process identifier of each processing process.

[0054] Specifically, the process information table also includes a parent process identifier field; based on the process identifier of each processing process corresponding to the current product to be processed, the process information table is queried to obtain the parent process identifier of the corresponding processing process. Then, based on the parent process identifier of each processing process, the dependency relationship between multiple processing processes, i.e., the parent-child relationship, is determined.

[0055] Sa2: The scheduling node uses the process identifier as the node and the dependency relationship between the processing processes as the edge to construct the process relationship graph of the product to be processed.

[0056] It should be noted that the process relationship graph is a directed graph, and the direction of the edges is from the parent process to the child process.

[0057] Sa3: The scheduling node constructs the tasks to be executed for the current product based on the product data and process relationship diagram of the current product to be processed.

[0058] Specifically, for the multiple processing procedures corresponding to the current product to be processed, the process identifier and specific processing procedure of each processing procedure are encapsulated according to a preset format. The encapsulated process identifier and specific processing procedure of each processing procedure are used as an element to construct the process list of the current product to be processed. Then, the process relationship diagram is written as an element into the process list. Then, the product data and process list of the current product to be processed are encapsulated according to a preset format, and the encapsulated data is named the task to be executed for the current product to be processed, thereby constructing the task to be executed for the current product to be processed.

[0059] In this embodiment, the task to be executed can include the dependencies between processing processes, providing an accurate basis for the execution order of multiple processing processes by subsequent execution nodes, thereby improving the accuracy and efficiency of task processing.

[0060] Optionally, when writing each task to be executed to the task list, the time when each task is written to the task list can be recorded.

[0061] Optionally, the preset write rules include writing according to the order in which tasks are written to the task list.

[0062] S230. The management center node performs integrity verification on each task in the task list to be executed and obtains the verification result of the corresponding task.

[0063] Specifically, the management center node receives the list of tasks to be executed and the target scheduling parameters sent by the scheduling node, and parses the list of tasks to be executed to obtain the product data and process list for each task. Then, it selects any one task from the multiple tasks to be executed as the current task and verifies whether the product data and process list of the current task are complete. That is, whether the product data includes all the data required for the data splitting task, and whether the process list includes process identifiers, specific processing procedures, and process relationship diagrams. If the product data includes all the data required for the data splitting task, and the process list includes process identifiers, specific processing procedures, and process relationship diagrams, it indicates that the fields in the current task are complete, and the verification result of the current task is determined to be successful. Otherwise, it indicates that the fields in the current task are incomplete, and the verification result of the current task is determined to be unsuccessful. Then, it iterates through each task and repeats the above process to determine the verification result of each task.

[0064] S240. The management center node writes the tasks that have passed the verification to the task queue in the order in which they were written to the task list.

[0065] Specifically, the earliest pending task is located at the head of the pending task queue, and the latest pending task is located at the tail of the pending task queue.

[0066] Optionally, the management center node generates data anomaly information for the product code of the pending task that fails the verification based on the verification result, and feeds back the data anomaly information to the scheduling node; the scheduling node can parse the data anomaly information sent by the management center node, obtain the product code, and reconstruct the pending task corresponding to the product code.

[0067] Optionally, before the management center node assigns execution nodes to the tasks to be executed from the queue based on the target scheduling parameters, the execution node generates a heartbeat packet based on the node identifier and sends the heartbeat packet to the management center node at a first set time interval, where the first set time interval is a pre-set time interval between two consecutive heartbeat packets sent by the execution node, such as 2 minutes.

[0068] The management center node determines the node status of each execution node based on the received heartbeat packets. That is, it identifies the execution node corresponding to the heartbeat packet based on the node identifier in the heartbeat packet. When a heartbeat packet of the current execution node is received, the node status of the current execution node is determined to be alive. If no heartbeat packet of the current execution node is received, or if no heartbeat packet of the current execution node is received within a first preset time period since the most recent heartbeat packet of the current execution node was received, the node status of the current execution node is determined to be faulty. Then, a node fault warning is generated based on the node identifier of the current execution node and sent to the administrator's email address to notify the administrator to handle the execution node anomaly in a timely manner. The first preset time period is a pre-set time period, such as 6 minutes.

[0069] Then, the management center node updates the list of live nodes based on the node status of each execution node. The list of live nodes includes execution nodes whose node status is live. It can detect whether the execution nodes are alive in real time, providing an accurate allocation basis for assigning execution nodes to tasks to be dequeued, thereby improving task processing efficiency.

[0070] S250, the management center node determines the target execution node cluster corresponding to the split business based on the scheduling strategy in the target scheduling parameters.

[0071] The target execution node cluster is a set of node identifiers for the execution nodes used to execute the split business.

[0072] Optionally, the list of idle nodes includes execution nodes whose node status is idle. The list of idle nodes is determined by the management center node based on the task acquisition requests sent by the execution nodes. The task acquisition requests are generated by the idle execution nodes based on the node identifier and sent to the management center node at a second set time interval.

[0073] The second set time interval is a pre-set time interval between two consecutive task acquisition requests sent by the execution node, such as 1 second; the second preset duration is a pre-set value, such as 3 seconds.

[0074] Specifically, the idle execution node generates a task acquisition request based on the node identifier and sends the task acquisition request to the management center node according to the second set time interval.

[0075] The management center node determines the execution node corresponding to the task acquisition request based on the node identifier in the task acquisition request. When the management center node receives a task acquisition request from the current execution node, it determines that the node status of the current execution node is idle. When the management center node receives a heartbeat packet from the current execution node, but does not receive a task acquisition request from the current execution node within a second preset time period since the last time it received a task acquisition request from the current execution node, it determines that the node status of the current execution node is busy. This determines the node status of each execution node. Then, the management center node adds execution nodes that are not in the idle node list and whose node status is idle to the idle node list, and deletes execution nodes in the idle node list whose node status changes from idle to busy.

[0076] Optionally, during the process of writing execution nodes that do not exist in the list of idle nodes and whose node status is idle into the list of idle nodes, the management center node obtains the current time and records it as the idle start time. Then, it encapsulates the node identifier and idle start time of the execution node according to a preset format, and writes the encapsulated data as an element into the list of idle nodes.

[0077] S260, the management center node dequeues the first task from the queue of tasks to be executed, and determines the execution node corresponding to the dequeued task from the target execution node cluster based on the list of live nodes and the list of idle nodes.

[0078] Specifically, after determining the target execution node cluster, the management center node dequeues the first task in the queue of tasks to be executed. Based on the list of live nodes and the list of idle nodes, it determines the execution node corresponding to the dequeued task from the target execution node cluster. That is, it determines the intersection of the list of live nodes, the list of idle nodes, and the multiple node identifiers included in the target execution node cluster to obtain a candidate execution node cluster. It then determines the node identifier of the candidate execution node cluster with the earliest start time of idleness and determines the execution node corresponding to the dequeued task.

[0079] S270, the management center node distributes the tasks to be executed to the corresponding execution nodes.

[0080] S280: The execution node parses product data and multiple processing procedures from the received tasks to be executed, and executes multiple processing procedures based on the product data.

[0081] Specifically, the execution node parses product data, multiple processing procedures, and process relationship diagrams from the received tasks to be executed. That is, it parses product data and a process list from the received tasks to be executed, and parses process identifiers and specific processing procedures for multiple processing procedures, as well as a process relationship diagram, from the process list.

[0082] Next, the execution node determines associated processing process sets and independent processing processes from multiple processing processes based on the process relationship diagram. An associated processing process set includes multiple associated processing processes, which are processes that have dependencies on other processing processes. All associated processing processes within an associated processing process set have dependencies on each other, while associated processing processes within different associated processing process sets do not have dependencies on each other. An independent processing process is a process that does not have dependencies on other processing processes. Specifically, multiple nodes with dependencies are obtained from the process relationship diagram, and the processing processes corresponding to these nodes are identified as associated processing processes, thus obtaining multiple associated processing processes. These multiple associated processing processes with dependencies are then combined into an associated processing process set, which includes process identifiers for multiple associated processing processes. At this point, there may be one or more associated processing process sets. Simultaneously, nodes that do not have dependencies on other nodes are obtained from the process relationship diagram, and the processing processes corresponding to these nodes are identified as independent processing processes, thus obtaining one or more independent processing processes.

[0083] Then, the execution node assigns a corresponding first thread to each independent processing procedure and uses the first thread to execute the corresponding independent processing procedure based on the product data. That is, if there are multiple independent processing procedures, a first thread is assigned to each independent processing procedure, and multiple first threads are executed in parallel. At the same time, the execution node assigns a corresponding second thread to each set of associated processing procedures and uses the second thread to execute the corresponding set of associated processing procedures based on the product data and the process relationship diagram. That is, if there are multiple sets of associated processing procedures, a second thread is assigned to each set of associated processing procedures, and multiple second threads are executed in parallel, as are the first and second threads.

[0084] Furthermore, the second thread executes the corresponding set of associated processing procedures based on product data and process relationship diagram. Specifically, the second thread obtains the process identifiers of multiple associated processing procedures from the corresponding set of associated processing procedures, and queries the process relationship diagram based on these process identifiers to determine the execution order of the multiple associated processing procedures. That is, the processing procedure corresponding to the parent process identifier is executed first, and the processing procedure corresponding to the child process identifier is executed later. Then, the multiple associated processing procedures are executed sequentially based on the product data according to the execution order of the multiple associated processing procedures.

[0085] In this embodiment, a first thread is allocated to each independent processing procedure, and a second thread is allocated to each set of associated processing procedures. This allows for the parallel execution of independent processing operations, thereby improving task processing efficiency. Furthermore, the second thread executes each associated processing procedure sequentially according to the dependencies between them, thus improving task processing accuracy.

[0086] The technical solution of this application embodiment involves a scheduling node extracting product data for each product to be processed that meets preset splitting processing conditions from a product information table and a splitting parameter table. Based on the product code of each product to be processed, the scheduling node extracts multiple processing procedures corresponding to that product from a process information table. Then, based on the product data and multiple processing procedures, it constructs tasks to be executed for each product to be processed, writes each task to be executed into a task to be executed list, and sends the task to be executed list and the target scheduling parameters pre-set for the splitting business to the management center node. The management center node performs integrity verification on each task to be executed in the task to be executed list, obtains the verification result, and sequentially writes tasks with verified results into the task to be executed queue according to the order in which they were written into the task to be executed list. This verifies the integrity of the tasks to be executed, ensuring that the tasks in the task to be executed queue are complete, effectively avoiding task execution failures due to missing data, and thus improving task processing efficiency and accuracy. Next, the management center node determines the target execution node cluster corresponding to the splitting business based on the scheduling strategy in the target scheduling parameters and dequeues the first task to be executed from the task to be executed queue. Based on the list of live nodes and the list of idle nodes, the system determines the execution nodes corresponding to the dequeued tasks from the target execution node cluster, and then distributes the dequeued tasks to the corresponding execution nodes. This prioritizes assigning tasks to execution nodes with low load (i.e., idle nodes), achieving load balancing and improving task processing efficiency. The execution nodes parse product data and multiple processing procedures from the received tasks and execute these procedures based on the product data. This enables asynchronous processing of financial product data splitting tasks without relying on database job scheduling components and stored procedures. At least two execution nodes enable asynchronous parallel processing of data splitting tasks, effectively handling the parallel processing needs of large amounts of data, thus improving task processing efficiency and meeting the business requirements for efficient data splitting processing. Furthermore, the task processing system supports flexible expansion of execution nodes, allowing for the addition of execution nodes as the scale of financial product business grows. When the data splitting business requirements of financial products change, only the relevant algorithms and rules configurations in the task processing module of the asynchronous task framework need to be modified to quickly adapt to business changes, thereby improving scalability, adapting to the ever-evolving needs of financial product business, and reducing system maintenance costs.

[0087] Figure 3 This is a schematic diagram of a task processing system provided in an embodiment of this application, referring to... Figure 3 The task processing system may include a scheduling node, a management center node, and at least two execution nodes; wherein the management center node and the scheduling node can communicate with each other, and the management center node and each execution node can communicate with each other.

[0088] The scheduling node is used to extract product data of each product to be processed that meets the preset splitting processing conditions from the product information table and the splitting parameter table, and to extract multiple processing procedures corresponding to each product to be processed from the process information table based on the product code of each product to be processed; the product data includes the product code; based on the product data of each product to be processed and the multiple processing procedures, the node constructs the tasks to be executed for each product to be processed, writes each task to be executed into the task to be executed list, and sends the task to be executed list and the target scheduling parameters pre-set for the splitting business to the management center node.

[0089] The management center node is used to write the tasks to be executed from the list of tasks to be executed into the task queue according to the preset writing rules, assign execution nodes to the tasks to be executed dequeued from the task queue based on the target scheduling parameters, and distribute the dequeued tasks to the corresponding execution nodes.

[0090] The execution node is used to parse product data and multiple processing procedures from the received tasks to be executed, and to execute multiple processing procedures based on the product data.

[0091] The task processing system provided in this embodiment can be applied to the task processing methods provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0092] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 A block diagram is shown of an exemplary electronic device 11 suitable for implementing embodiments of the present application. The electronic device may be any node device in a task processing system, namely a scheduling node, a management center node, or an execution node. Figure 4 The electronic device 11 shown is merely an example and should not impose any limitations on the functionality and scope of use of this embodiment.

[0093] like Figure 4 As shown, the electronic device 11 is represented in the form of a general-purpose computing electronic device. The components of the electronic device 11 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0094] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, industry-standard architecture buses, microchannel architecture buses, enhanced industry-standard architecture buses, Video Electronics Standards Association (VESA) local buses, and peripheral component interconnect buses.

[0095] Electronic device 11 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 11, including volatile and non-volatile media, removable and non-removable media.

[0096] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory 30 and / or cache memory 32. Electronic device 11 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0097] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this application.

[0098] Electronic device 11 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 11, and / or with any device that enables electronic device 11 to communicate with one or more other computing devices (e.g., network interface card and modem, etc.). Such communication can be performed through input / output interface 22. Furthermore, electronic device 11 can also communicate with one or more networks (e.g., local area network, wide area network, and / or public network) through network adapter 20.

[0099] like Figure 4 As shown, network adapter 20 communicates with other modules of electronic device 11 via bus 18. It should be understood that, although... Figure 4 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 11, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, tape drives, and data backup storage systems.

[0100] The processing unit 16 executes various functional applications and page displays by running programs stored in the system memory 28, such as implementing a task processing method provided in any embodiment of this application.

[0101] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a task processing method, such as that provided in any embodiment of this application.

[0102] The computer storage medium of this embodiment can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0103] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0104] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, radio frequency, etc., or any suitable combination thereof.

[0105] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​as well as conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0106] This application also provides a computer program product, including a computer program that, when executed by a processor, implements a task processing method as provided in any embodiment of this application.

[0107] In the implementation of a computer program product, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​as well as conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] Those skilled in the art will understand that the modules or steps described above in this application can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, which can then be stored in a storage device for execution by a computing device. Alternatively, they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0109] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the inventive concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. A task processing method, characterized in that, Applied to a task processing system, the task processing system including a scheduling node, a management center node, and at least two execution nodes, the method includes: The scheduling node extracts product data for each product to be processed that meets the preset splitting processing conditions from the product information table and the splitting parameter table, and extracts multiple processing procedures for the corresponding product from the process information table based on the product code of each product to be processed; the product data includes the product code; The scheduling node constructs the corresponding tasks to be executed for each product to be processed based on the product data and multiple processing procedures, writes each task to be executed into the task list, and sends the task list and the target scheduling parameters pre-set for splitting the business to the management center node. The management center node writes the tasks to be executed from the task list to the task queue according to the preset writing rules, assigns execution nodes to the tasks to be executed dequeued from the task queue based on the target scheduling parameters, and distributes the dequeued tasks to the corresponding execution nodes; The execution node parses product data and multiple processing procedures from the received tasks to be executed, and executes multiple processing procedures based on the product data.

2. The task processing method according to claim 1, characterized in that, The scheduling node constructs the corresponding tasks to be executed for each product to be processed based on the product data and multiple processing procedures, including: Select any one of the products to be processed as the current product to be processed, obtain the parent process identifier of each processing process corresponding to the current product to be processed from the process information table, and determine the dependency relationship between multiple processing processes based on the parent process identifier of each processing process. Using process identifiers as nodes and dependencies between processing processes as edges, construct a process relationship graph for the product to be processed. Construct the tasks to be executed for the current product based on the product data and process relationship diagram of the current product to be processed.

3. The task processing method according to claim 2, characterized in that, The execution node parses product data and multiple processing procedures from the received tasks to be executed, and executes multiple processing procedures based on the product data, including: Parse product data, multiple processing procedures, and process relationship diagrams from the received tasks to be executed; Based on process relationship diagrams, related sets of processes and independent processes are identified from multiple processes; Assign a first thread to each independent processing step, and use the first thread to execute the corresponding independent processing step based on the product data; Assign a corresponding second thread to the associated processing procedure set, and use the second thread to execute the corresponding associated processing procedure set based on product data and process relationship diagram; the first thread and the second thread execute in parallel.

4. The task processing method according to claim 1, characterized in that, The preset write rules include writing tasks in the order they were written to the task list. The management center node writes the tasks in the task list to the task queue according to the preset write rules, including: Perform integrity verification on each task in the task list to obtain the verification result for the corresponding task. Based on the order in which tasks were written into the task list, tasks that passed the verification are written into the task queue in sequence.

5. The task processing method according to claim 1, characterized in that, Before the management center node assigns execution nodes to the tasks dequeued from the task queue based on the target scheduling parameters, the process also includes: The execution node generates a heartbeat packet based on the node identifier and sends the heartbeat packet to the management center node at a first set time interval; The management center node determines the node status of each execution node based on the received heartbeat packets, and updates the list of live nodes based on the node status of each execution node; the list of live nodes includes execution nodes whose node status is live.

6. The task processing method according to claim 5, characterized in that, The management center node assigns execution nodes to tasks dequeued from the task queue based on target scheduling parameters, including: The target execution node cluster corresponding to the split business is determined based on the scheduling strategy in the target scheduling parameters; The first task to be executed in the queue is dequeued, and the execution node corresponding to the dequeued task is determined from the target execution node cluster based on the list of live nodes and the list of idle nodes. The list of idle nodes includes execution nodes whose node status is idle. The list of idle nodes is determined by the management center node based on the task acquisition requests sent by the execution nodes. The task acquisition requests are generated by the idle execution nodes based on the node identifier and sent to the management center node at a second set time interval.

7. The task processing method according to claim 1, characterized in that, The preset splitting processing conditions include the splitting business being in the on state, the product not existing in the splitting parameter table and meeting the preset splitting business rule conditions.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the task processing method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the task processing method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the task processing method according to any one of claims 1 to 7.

Images