A task processing method, device, electronic device and storage medium

Abstract

The embodiment of the present application discloses a task processing method, device, electronic device and storage medium, the method comprising: for each subtask, obtaining a subtask attribute information set; wherein each subtask is obtained by splitting the same main task; according to the type of the subtask, determining a piece of attribute information in the attribute information set as a subtask grouping identifier; grouping each subtask according to the grouping identifier of each subtask; wherein each subtask group includes a plurality of subtasks; sequentially executing each subtask in each subtask group; concurrently executing each subtask in different subtask groups. While improving the task processing speed, the problem of concurrent conflict is avoided.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task processing method, device, electronic device and storage medium. Background Art

[0002] In the banking business scenarios, high-performance business processing hosts have always been favored by major banks for their high availability and high throughput. Single-host equipment relies on a single device, which is not conducive to autonomous availability. With the rapid growth of core business volume, the host is facing increasingly severe tests.

[0003] In order to solve the above problems, the batch processing function of the host needs to be implemented by a batch processing framework of a distributed architecture. In the related art, in the implementation of the batch processing framework, each subtask is either executed concurrently or sequentially. If all are executed sequentially, the execution speed will be slow; if all are executed concurrently, concurrency conflicts will occur, for example, data rows that are frequently added, deleted, modified, and checked (i.e., hot rows), deadlocks, and confusion will appear in the database. Summary of the invention

[0004] The embodiments of the present application provide a task processing method, device, electronic device and storage medium, which are used to improve the task processing speed while avoiding the problem of concurrency conflict.

[0005] In a first aspect, an embodiment of the present application provides a task processing method, including:

[0006] For each subtask, obtaining a set of attribute information of the subtask; wherein each of the subtasks is obtained by splitting the same main task;

[0007] According to the type of the subtask, determining a piece of attribute information in the attribute information set as a grouping identifier of the subtask;

[0008] Grouping the subtasks according to their grouping identifiers; wherein each subtask group includes a number of subtasks;

[0009] Execute each subtask in each subtask group sequentially; execute each subtask in different subtask groups concurrently.

[0010] In some exemplary embodiments, the sequential execution of each subtask in each subtask group includes:

[0011] For each subtask group, start a thread;

[0012] Based on the thread, each subtask in the subtask group is executed sequentially.

[0013] In some exemplary embodiments, the concurrently executing each subtask in different subtask groups includes:

[0014] For each subtask group, based on the principle that each thread corresponds to one subtask group, threads of the same number as the number of subtask groups are started, so that each thread executes a subtask in a subtask group.

[0015] In some exemplary embodiments, for each subtask group, sequentially executing each subtask in the subtask group includes:

[0016] Determine the status flag of the subtask group;

[0017] If the status flag indicates that no subtask in the subtask group is in an execution state, an unexecuted subtask is selected from the subtask group according to the business configuration rule, and the status flag of the subtask group is updated, wherein the updated status flag indicates that there is a subtask in the subtask group in an execution state.

[0018] In some exemplary embodiments, after determining the status flag of the subtask group, the method further includes:

[0019] If the status flag indicates that there is a subtask in the subtask group that is in an execution state, wait for the subtask to be completed, and update the status flag of the subtask group, wherein the updated status flag indicates that there is no subtask in the subtask group that is in an execution state;

[0020] An unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

[0021] In some exemplary embodiments, for each subtask group, sequentially executing each subtask in the subtask group includes:

[0022] For each subtask group, determining the status of each subtask in the subtask group;

[0023] If there is no subtask in execution at present, an unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

[0024] In some exemplary embodiments, after determining the execution status of each subtask in each subtask group, the method further includes:

[0025] If there is a subtask currently in execution, wait for the subtask to be completed, and select an unexecuted subtask from the subtask group for execution according to the business configuration rule.

[0026] In some exemplary embodiments, before acquiring the attribute information set of each subtask, the method further includes:

[0027] Filter the main tasks in the unprocessed state by the main task status identifier;

[0028] The main task is divided into several subtasks.

[0029] In some exemplary embodiments, the method further comprises:

[0030] For each main task, determining the execution status of each subtask in the main task;

[0031] If all subtasks in the main task are completed, a message that the main task has been completed is fed back.

[0032] In a second aspect, an embodiment of the present application provides a task processing device, including:

[0033] An information acquisition module, used for acquiring a set of attribute information of each subtask, wherein each subtask is obtained by splitting the same main task;

[0034] A group identification determination module, used to determine, according to the type of the subtask, a piece of attribute information in the attribute information set as the group identification of the subtask;

[0035] A grouping module, used to group each subtask according to the grouping identifier of each subtask; wherein each subtask group includes a plurality of subtasks;

[0036] The execution module is used to sequentially execute each subtask in each subtask group; and concurrently execute each subtask in different subtask groups.

[0037] In some exemplary implementations, the execution module is specifically used to: start a thread for each subtask group;

[0038] Based on the thread, each subtask in the subtask group is executed sequentially.

[0039] In some exemplary embodiments, the execution module is specifically configured to:

[0040] For each subtask group, based on the principle that each thread corresponds to one subtask group, threads of the same number as the number of subtask groups are started, so that each thread executes a subtask in a subtask group.

[0041] In some exemplary embodiments, the execution module is specifically configured to:

[0042] Determine the status flag of the subtask group;

[0043] If the status flag indicates that no subtask in the subtask group is in an execution state, an unexecuted subtask is selected from the subtask group according to the business configuration rule, and the status flag of the subtask group is updated, wherein the updated status flag indicates that there is a subtask in the subtask group in an execution state.

[0044] In some exemplary embodiments, the execution module is further configured to, after determining the status flag of the subtask group, if the status flag indicates that there is a subtask in the subtask group that is in an execution state, wait for the subtask to be completed, and update the status flag of the subtask group, wherein the updated status flag indicates that there is no subtask in the subtask group that is in an execution state;

[0045] An unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

[0046] In some exemplary embodiments, the execution module is further configured to:

[0047] For each subtask group, determining the status of each subtask in the subtask group;

[0048] If there is no subtask in execution at present, an unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

[0049] In some exemplary embodiments, the execution module is specifically used to, for each subtask group, after determining the execution status of each subtask in the subtask group, if there is a subtask currently in execution, wait for the subtask to be executed, and select an unexecuted subtask from the subtask group for execution according to the business configuration rule.

[0050] In some exemplary embodiments, a screening module is further included, which is used for, before obtaining the attribute information set of each subtask:

[0051] Filter the main tasks in the unprocessed state by the main task status identifier;

[0052] The main task is divided into several subtasks.

[0053] In some exemplary embodiments, a state feedback module is further included, which is used to:

[0054] For each main task, determining the execution status of each subtask in the main task;

[0055] If all subtasks in the main task are completed, a message that the main task has been completed is fed back.

[0056] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0057] processor;

[0058] a memory for storing instructions executable by the processor;

[0059] The processor is configured to execute the instructions to implement any one of the methods provided in the first aspect of the present application.

[0060] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any method provided in the first aspect of the present application.

[0061] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements any of the methods provided in the first aspect of the present application.

[0062] The technical solution provided by the embodiment of the present application brings at least the following beneficial effects: for each subtask obtained by splitting the same main task, a set of attribute information of the subtask is obtained; since different subtask types have different attribute information that affects concurrency conflicts, when determining the grouping identifier, the subtask type is considered, and an attribute information is selected from the attribute set as the grouping identifier of the corresponding subtask, and then each subtask is grouped according to the grouping identifier of the subtask. Since the grouping identifiers of the same subtask group are the same, and the grouping identifiers of different subtask groups are different, the problem of concurrency conflicts can be avoided by sequentially executing each subtask in the same subtask group; the problem of concurrency conflicts will not occur for the concurrent execution of each subtask in different subtask groups, and the task execution speed can also be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 A schematic diagram of an application scenario of a task processing method provided in one embodiment of the present application;

[0065] Figure 2A flowchart of a task processing method provided in one embodiment of the present application;

[0066] Figure 3 A schematic diagram of a subtask grouping provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of a flow chart for sequentially executing each subtask in a subtask group provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of another flow chart for sequentially executing each subtask in a subtask group provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of the structure of a task processing device provided in one embodiment of the present application;

[0070] Figure 7 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0072] For ease of understanding, the terms involved in the embodiments of the present application are explained below:

[0073] Any number of elements in the drawings is for illustration and not limitation, and any naming is for distinction only and does not have any limiting meaning.

[0074] In the actual practice, in the banking business scenarios, high-performance business processing hosts have always been favored by major banks for their high availability and high throughput. Single-host equipment relies on a single device, which is not conducive to autonomous availability. With the rapid growth of core business volume, the host is facing increasingly severe tests.

[0075] In order to solve the above problems, the batch processing function of the host needs to be implemented by a batch processing framework of a distributed architecture. In the related art, in the implementation of the batch processing framework, each subtask is either executed concurrently, such as through thread pool management, all subtasks to be executed are thrown into the thread pool to achieve concurrency; or executed sequentially, such as through thread management, only one thread is opened to process subtasks, and only when one subtask is completed can the next subtask be carried out, thus achieving sequential execution. If all are executed concurrently, the execution speed will be slow; if all are executed sequentially, the database will have problems such as data rows that are frequently added, deleted, modified, and checked (i.e., hot rows), deadlocks, and confusion.

[0076] For example, in a specific scenario, in the bank's batch collection and payment, customer information splitting and merging, and account entry businesses, there are about 20-30 million transactions a day, of which about 2-3 million batches of batch main tasks are initiated, including some large batch splitting tasks. Often, a main task has up to 100,000 subtasks to be processed. Among them, some subtasks have scenarios such as the same customer's card number, the same account, and the same institution. For the same card number, account, or institution in the same main task, they need to be processed in sequence, otherwise there will be problems such as database hotspots, deadlocks, and financial confusion.

[0077] To this end, the present application provides a task processing method, in which, for each subtask, a set of attribute information of the subtask is obtained; wherein each subtask is obtained by splitting the same main task; according to the type of the subtask, a piece of attribute information in the set of attribute information is determined as the grouping identifier of the subtask; each subtask is grouped according to the grouping identifier of each subtask; wherein each subtask group includes a number of subtasks; each subtask in each subtask group is executed sequentially; and each subtask in different subtask groups is executed concurrently. While improving the task processing speed, the problem of concurrent conflicts is avoided.

[0078] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0079] refer to Figure 1 , which is a schematic diagram of an application scenario of the task processing method provided in an embodiment of the present application. The application scenario includes multiple terminal devices 101 (including terminal device 101-1, terminal device 101-2, ... terminal device 101-n) and a server 102. Among them, the terminal device 101 and the server 102 are connected via a wireless or wired network, and the terminal device 101 includes but is not limited to electronic devices such as desktop computers, mobile phones, mobile computers, tablet computers, media players, smart wearable devices, smart TVs, etc. The server 102 can be a server cluster or a cloud computing center composed of a plurality of servers. The server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0080] The server 102 obtains a set of attribute information of each subtask, wherein each subtask is obtained by splitting the same main task; according to the type of the subtask, a piece of attribute information in the set of attribute information is determined as a grouping identifier of the subtask; each subtask is grouped according to the grouping identifier of each subtask; wherein each subtask group includes a plurality of subtasks; each subtask in each subtask group is executed sequentially; each subtask in different subtask groups is executed concurrently. The server 102 can also feed back the execution results of each subtask to the terminal device 101 for display.

[0081] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here.

[0082] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0083] Combine the following Figure 1 The application scenario shown illustrates the technical solution provided by the embodiment of the present application.

[0084] refer to Figure 2 , the present application embodiment provides a task processing method, comprising the following steps:

[0085] S201, for each subtask, obtaining a set of attribute information of the subtask; wherein each subtask is obtained by splitting the same main task.

[0086] S202: According to the type of the subtask, determine a piece of attribute information in the attribute information set as a grouping identifier of the subtask.

[0087] S203: grouping the subtasks according to their grouping identifiers; wherein each subtask group includes a plurality of subtasks.

[0088] S204, sequentially executing each subtask in each subtask group; and concurrently executing each subtask in different subtask groups.

[0089] In the embodiment of the present application, for each subtask obtained by splitting the same main task, a set of attribute information of the subtask is obtained; since different subtask types have different attribute information that affects concurrency conflicts, when determining the grouping identifier, the subtask type is considered, and an attribute information is selected from the attribute set as the grouping identifier of the corresponding subtask, and then each subtask is grouped according to the grouping identifier of the subtask. Since the grouping identifiers of the same subtask group are the same, and the grouping identifiers of different subtask groups are different, the problem of concurrency conflicts can be avoided by sequentially executing each subtask in the same subtask group; the problem of concurrency conflicts will not occur for the concurrent execution of each subtask in different subtask groups, and the task execution speed can also be improved.

[0090] Exemplarily, the task processing method of the embodiment of the present application can be implemented based on a high-frequency processing framework, and the processing process can divide the processing process of the main task into three stages, namely the main task processing stage, the subtask execution stage, and the main task post-processing stage. Among them, the main task processing stage is to split the main task into single subtasks, such as the subtask execution stage, which is responsible for the execution of the subtask, that is, a single online transaction, and the main task post-processing stage is responsible for summarizing the execution of the subtasks into a report or other required forms for processing.

[0091] In the actual application process, the task processing equipment (such as high-frequency scheduling device) is implemented by setting up a scheduled patroller. For example, the main task pre-processing patroller screens out the main tasks that meet the conditions (such as being in the unstarted state) and distributes them into the pre-processing stage, that is, the stage of splitting a main task into several sub-tasks; the sub-task execution completion patroller counts the execution status of each sub-task under a main task, and those that are completed enter the post-processing stage; the main task post-processing patroller screens out the main tasks that meet the conditions (such as all sub-tasks have been executed) and distributes them into the post-processing stage.

[0092] More specifically, the execution process of the main task includes the following stages: not started, preprocessing, preprocessing completed, subtask processing, subtask processing completed, post-processing, and completed. In each stage, the status of each subtask or main task is updated. For example, if there are unfinished subtasks, the status of the main task at this time is subtask processing. In a specific example, Table 1 shows a high-frequency main task status table, which determines the status of the main task at each moment by reading the status code.

[0093] In this way, if there are multiple main tasks, the main tasks in the unprocessed state can be screened by the main task status identifier (such as the status code). If there are multiple main tasks in the unprocessed state, it can be determined which main task to execute first according to the characteristics of different main tasks. For example, if a main task A to be executed is determined, the main task is split into several subtasks. The specific splitting principle can refer to the type and attribute information of the subtask.

[0094] Table 1 High-frequency main task status table

[0095] state Status Code Not started 00 Preprocessing 10 Preprocessing completed 11 Subtask processing 20 Subtask processing completed 21 Post-processing 30 Finish 40

[0096] Taking main task A as an example, main task A is split into several subtasks, and the task processing process is explained.

[0097] In S201, for each subtask, a set of attribute information of the subtask is obtained.

[0098] Taking the example of a company distributing wages to employees in batches, the attribute information set of each subtask is a set of employee name, bank card account number, payment account and payment institution code.

[0099] In S202, according to the type of the subtask, a piece of attribute information in the attribute information set is determined as a grouping identifier of the subtask.

[0100] For example, different types of subtasks may cause different attribute information that may conflict with other subtasks in concurrent execution. Therefore, in the process of determining the grouping identifier, the type of subtask is taken into consideration, so that subtasks in different groups will not have concurrent conflicts. For example, if the subtask is to pay employees' salaries in batches, the payment account may cause concurrent conflicts between subtasks. Therefore, in this example, the payment account can be selected as the grouping identifier of the subtask. In this way, all subtasks under a main task are grouped, and the grouping identifier of each subtask is determined.

[0101] In step S203, each subtask is grouped according to its grouping identifier, wherein each subtask group includes a plurality of subtasks. For example, a subtask group with a grouping identifier of X is one subtask group, and a subtask group with a grouping identifier of Y is another subtask group.

[0102] Regarding S204, due to the aforementioned grouping principle, subtasks within the same group need to be executed sequentially to avoid conflicts. For subtasks in different groups, since the group identifiers are different, there will be no concurrent conflict factors, and therefore they can be executed concurrently.

[0103] During the execution of the task, for each main task, the execution status of each subtask in the main task is determined; if all subtasks in the main task are completed, the main task is determined to be completed, and a message of the completion of the main task is fed back. The feedback form can be, for example, a voice or text prompt on the task execution terminal.

[0104] Exemplarily, each subtask in each subtask group may be executed sequentially in the following manner:

[0105] For each subtask group, a thread is started; based on the thread, each subtask in the subtask group is executed sequentially. Specifically, each subtask group starts a thread, so that the thread can be used to sequentially execute each subtask in the subtask group.

[0106] Exemplarily, each subtask in different subtask groups may be executed concurrently in the following manner:

[0107] For each subtask group, based on the principle that each thread corresponds to one subtask group, the same number of threads as the number of subtask groups are opened, so that each thread executes a subtask in a subtask group. Specifically, for example, if there are n subtask groups, one thread is opened for each subtask group, so that each thread executes a subtask in a subtask group. Since the grouping identifiers of subtasks in different groups are different, that is, there is no concurrency conflict between subtasks in different groups.

[0108] In a specific example, Figure 3 A schematic diagram of subtask grouping is shown, wherein a main task N is divided into n subtask groups, the first subtask has N1 subtasks, the second subtask group has N2 subtasks, ..., the nth subtask group has Nn subtasks, and N and n are both integers. The subtasks in each subtask group are executed sequentially, and different subtask groups can be executed concurrently.

[0109] Next, the specific implementation method of sequentially executing each subtask in each subtask group is described; wherein, Figure 4 A schematic diagram of a process for sequentially executing each subtask in a subtask group is shown; Figure 5 Another flowchart for sequentially executing each subtask in a subtask group is shown.

[0110] The first one, reference Figure 4 , the way to apply the status flag of the subtask group:

[0111] S401: Determine the status flag of the subtask group.

[0112] S402. If the status flag indicates that no subtask is in the execution state in the subtask group, select an unexecuted subtask from the subtask group according to the business configuration rule, and update the status flag of the subtask group, wherein the updated status flag indicates that there is a subtask in the execution state in the subtask group.

[0113] S403. If the status flag indicates that there is a subtask in the subtask group that is in execution state, wait for the subtask to be executed and update the status flag of the subtask group, wherein the updated status flag indicates that there is no subtask in the subtask group that is in execution state; select an unexecuted subtask from the subtask group according to the business configuration rule for execution.

[0114] Specifically, each subtask group corresponds to a status flag, and the value of the status flag switches between 0 and 1. For example, 0 indicates that no subtask is currently being executed, and 1 indicates that a subtask is currently being executed. In the actual task status monitoring process, the status flag of the subtask group is determined. If the status flag indicates that no subtask is in the execution state in the subtask group, that is, it is 0, then an unexecuted subtask is selected from the subtask group according to the configuration rules, and then the status flag of the subtask group is updated to indicate that there is a subtask in the execution state in the subtask group, for example, it is updated to 1.

[0115] If the current status flag indicates that there is a subtask in the subtask group that is in execution state, for example, it is 1, then no new subtask can be executed at this time. Wait for the subtask being executed to be completed and update the status flag to 0. At this time, you can filter out an unexecuted subtask from the subtask group according to the business configuration rules.

[0116] Second, reference Figure 5 , by determining the status of the subtask:

[0117] S501 : For each subtask group, determine the status of each subtask in the subtask group.

[0118] S502: If there is no subtask currently in execution, select an unexecuted subtask from the subtask group for execution according to the business configuration rule.

[0119] S503: If there is a subtask currently in execution, wait for the subtask to be completed, and select an unexecuted subtask from the subtask group for execution according to the business configuration rule.

[0120] Specifically, for each subtask group, the status of each subtask in the subtask group is directly determined, for example, some subtasks have been executed, some subtasks have not been executed, and at most one subtask is being executed, that is, in the execution state. If there is currently no subtask in the execution state, at this time, an unexecuted subtask can be selected from the subtask group according to the business configuration rules for execution; if there is currently a subtask in the execution state, wait for the subtask to be executed, and then select an unexecuted subtask from the subtask group according to the business configuration rules for execution.

[0121] Exemplarily, the service configuration rule may be determined according to the order of each subtask in the subtask group, with the earlier subtasks being executed first and the later subtasks being executed later.

[0122] The following example is used to illustrate the execution process of the task processing method of the embodiment of the present application:

[0123] According to the characteristics of the high-frequency framework, the function of sequential processing according to the group identifier is designed in the preprocessing and splitting stages. In the preprocessing stage, during the task splitting process, by inserting the subtask table, for the task types that require grouped sequential execution, a group identifier field is designed in the configuration table (HFT_TAK_CFG). This field can configure the attribute information of the subtasks that need to be grouped, such as the Java field names of the card number, institution number, account number, etc.

[0124] In a specific example, the following configuration of the organization number indicates that the same organization number of the subtasks in the main task is executed serially (sequentially), and different organization numbers are executed in parallel (concurrently). The example organization number configuration is as follows:

[0125] LOCK_ECD

[0126] CCAFINPTBo.txnInsid.

[0127] At the same time, the LOCK_ECD field is also designed in the subtask table (HFT_TASK_SUB). When the main task is split and inserted into the subtask table in the preprocessing stage, it is found by reading the LOCK_ECD field of the HFT_TASK_CFG table that this task needs to be grouped and executed serially by organization number. When parsing the file record, the grouping identifier of each record will be recorded in this field. In this way, the grouping of each subtask under the main task is completed in the preprocessing stage.

[0128] After the grouping is completed, the task execution process begins. For example, the task execution process can be implemented by the Kafka message queue. In the Kafka message queue, during the task consumption process (execution), a map is first established when consuming this shard. The key value is "main task ID + grouping identifier", and the value is a counter. The attribute of the counter is a status flag, which is used to represent the execution status of the current subtask master.

[0129] Data is filtered out from the subtask table in sequence for asynchronous execution. When the counter value of "main task ID + group identifier" is 0, it means that the current task can be executed. The counter value is 1 during execution. Subsequent subtasks will wait for the counter to be 0 before running. The counter value is set to 1 during execution. This mechanism ensures that the unified group identifier is executed in sequence. Different group identifiers have different key values ​​and can be executed in parallel.

[0130] In a specific example, Table 2 shows a Kafka message status table, in which "not sent" indicates that the task has not been executed, "sent" indicates that the task is to be executed, and "consumed" indicates that the task has been executed.

[0131] Table 2 Kafka message status table

[0132] state Status Code Not issued 0 Issued 1 Consumed 2

[0133] In summary, in the embodiments of the present application, execution in the order of group identifiers can be achieved by configuring one field (group identifier), the configuration is simple, and the problem of concurrency conflicts is avoided.

[0134] The task scheduling method in the embodiment of the present application can be applied to the batch processing framework of the distributed architecture, so as to support the batch processing function of the original host. The high-frequency scheduling device in the parallel computing of the batch processing framework is responsible for the functions of disassembling, distributing, executing, and summarizing the batch splitting business of the core bank. The mechanism of sequentially processing the subtasks according to the grouping identifier can also meet the processing requirements of specific business scenarios of the distributed banking core system.

[0135] like Figure 6 As shown, based on the same inventive concept as the above-mentioned task processing method, an embodiment of the present application also provides a task processing device, including an information acquisition module 61, a group identification determination module 62, a grouping module 63 and an execution module 64.

[0136] The information acquisition module 61 is used to acquire a set of attribute information of each subtask, wherein each subtask is obtained by splitting the same main task;

[0137] A group identification determination module 62, used to determine a piece of attribute information in the attribute information set as a group identification of the subtask according to the type of the subtask;

[0138] A grouping module 63, used to group each subtask according to the grouping identifier of each subtask; wherein each subtask group includes a plurality of subtasks;

[0139] The execution module 64 is used to sequentially execute each subtask in each subtask group and concurrently execute each subtask in different subtask groups.

[0140] In some exemplary implementations, the execution module is specifically used to: start a thread for each subtask group;

[0141] Based on threads, each subtask in the subtask group is executed sequentially.

[0142] In some exemplary embodiments, the execution module 64 is specifically configured to:

[0143] For each subtask group, based on the principle that each thread corresponds to one subtask group, threads of the same number as the number of subtask groups are started, so that each thread executes a subtask in a subtask group.

[0144] In some exemplary embodiments, the execution module 64 is specifically configured to:

[0145] Determine the status flag of the subtask group;

[0146] If the status flag indicates that no subtask is in the execution state in the subtask group, an unexecuted subtask is selected from the subtask group according to the business configuration rule, and the status flag of the subtask group is updated, wherein the updated status flag indicates that there is a subtask in the execution state.

[0147] In some exemplary embodiments, the execution module 64 is further configured to, after determining the status flag of the subtask group, if the status flag indicates that there is a subtask in the subtask group that is in the execution state, wait for the subtask to be executed, and update the status flag of the subtask group, wherein the updated status flag indicates that there is no subtask in the subtask group that is in the execution state;

[0148] Filter an unexecuted subtask from the subtask group according to the business configuration rule.

[0149] In some exemplary embodiments, the execution module 64 is further configured to:

[0150] For each subtask group, determining the status of each subtask in the subtask group;

[0151] If there is no subtask in execution status, an unexecuted subtask is selected from the subtask group according to the business configuration rule for execution.

[0152] In some exemplary embodiments, the execution module is specifically used to, for each subtask group, determine the execution status of each subtask in the subtask group. If there is a subtask currently in execution, wait for the subtask to be executed, and select an unexecuted subtask from the subtask group for execution according to the business configuration rules.

[0153] In some exemplary embodiments, a screening module is further included, which is used for obtaining the attribute information set of each subtask before:

[0154] Filter the main tasks in the unprocessed state by the main task status identifier;

[0155] Split the main task into several subtasks.

[0156] In some exemplary embodiments, a state feedback module is further included, which is used to:

[0157] For each main task, determine the execution status of each subtask in the main task;

[0158] If all subtasks in the main task are completed, a message indicating that the main task has been completed will be fed back.

[0159] The task processing device proposed in the embodiment of the present application adopts the same inventive concept as the above-mentioned task processing method and can achieve the same beneficial effects, which will not be repeated here.

[0160] Based on the same inventive concept as the above-mentioned task processing method, the embodiment of the present application also provides an electronic device, which can be a control device or control system inside a smart device, or an external device that communicates with the smart device, such as a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), a server, etc. Figure 7 As shown, the electronic device may include a processor 801 and a memory 802 .

[0161] Processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0162] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0163] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; the above computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to: mobile storage devices, random access memory (RAM, Random Access Memory), magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)) and other media that can store program codes.

[0164] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a mobile storage device, a random access memory (RAM, Random Access Memory), a magnetic storage device (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage device (such as CD, DVD, BD, HVD, etc.), and a semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), a solid-state drive (SSD)) and other various media that can store program codes.

[0165] The above embodiments are only used to introduce the technical solutions of the present application in detail, but the description of the above embodiments is only used to help understand the methods of the embodiments of the present application and should not be understood as limiting the embodiments of the present application. Any changes or substitutions that can be easily thought of by technicians in this technical field should be included in the protection scope of the embodiments of the present application.

Claims

1. A task processing method, characterized in that: include: Filter the main tasks in the unprocessed state by the main task status identifier; If there are multiple main tasks in an unprocessed state, the main task to be executed first is determined among the multiple main tasks in an unprocessed state; According to the types and attribute information of the subtasks included in the main task to be executed with priority, the main task to be executed with priority is divided into a plurality of subtasks; For each subtask, obtain a set of attribute information of the subtask; wherein each of the subtasks is obtained by splitting the same main task; if the type of the subtask is that the enterprise pays wages to employees in batches, then the set of attribute information of the subtask attributes is a set of employee name, bank card account number, payment account and payment institution code; According to the type of the subtask, determine a piece of attribute information in the attribute information set as a grouping identifier of the subtask; wherein the piece of attribute information is the attribute information in the attribute set that causes concurrency conflicts between different subtasks; The subtasks are grouped according to their grouping identifiers; wherein each subtask group includes a plurality of subtasks; and there is no concurrency conflict between subtasks in different subtask groups, and the grouping identifier is a payment account; Execute each subtask in each subtask group sequentially; execute each subtask in different subtask groups concurrently.

2. The method according to claim 1, characterized in that: The sequential execution of each subtask in each subtask group includes: For each subtask group, start a thread; Based on the thread, each subtask in the subtask group is executed sequentially.

3. The method according to claim 1, characterized in that The concurrent execution of each subtask in different subtask groups includes: For each subtask group, based on the principle that each thread corresponds to one subtask group, threads of the same number as the number of subtask groups are started, so that each thread executes a subtask in a subtask group.

4. The method according to claim 1, characterized in that For each subtask group, sequentially executing each subtask in the subtask group includes: Determine the status flag of the subtask group; If the status flag indicates that no subtask in the subtask group is in an execution state, an unexecuted subtask is selected from the subtask group according to the business configuration rule, and the status flag of the subtask group is updated, wherein the updated status flag indicates that there is a subtask in the subtask group in an execution state.

5. The method according to claim 4, characterized in that After determining the status flag of the subtask group, the method further includes: If the status flag indicates that there is a subtask in the subtask group that is in an execution state, wait for the subtask to be completed, and update the status flag of the subtask group, wherein the updated status flag indicates that there is no subtask in the subtask group that is in an execution state; An unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

6. The method according to claim 1, characterized in that For each subtask group, sequentially executing each subtask in the subtask group includes: For each subtask group, determining the status of each subtask in the subtask group; If there is no subtask in execution at present, an unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

7. The method according to claim 6, characterized in that After determining the execution status of each subtask in each subtask group, the method further includes: If there is a subtask currently in execution, wait for the subtask to be completed, and select an unexecuted subtask from the subtask group for execution according to the business configuration rule.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: For each main task, determining the execution status of each subtask in the main task; If all subtasks in the main task are completed, a message that the main task has been completed is fed back.

9. A task processing device, characterized in that: include: A screening module is used to: screen the main tasks in an unprocessed state according to the main task status identifier; If there are multiple main tasks in an unprocessed state, determine a main task to be executed first among the multiple main tasks in an unprocessed state; split the main task to be executed first into several subtasks according to the type and attribute information of the subtasks included in the main task to be executed first; An information acquisition module is used to acquire, for each subtask, a set of attribute information of the subtask; wherein each of the subtasks is obtained by splitting the same main task; if the type of the subtask is that the enterprise pays wages to employees in batches, then the set of attribute information of the subtask attributes is a set of employee name, bank card account number, payment account and payment institution code; A group identification determination module, used to determine, according to the type of the subtask, a piece of attribute information in the attribute information set as the group identification of the subtask; wherein the piece of attribute information is the attribute information in the attribute set that causes concurrency conflicts between different subtasks; A grouping module, used for grouping each subtask according to a grouping identifier of each subtask; wherein each subtask group includes a plurality of subtasks; there is no concurrency conflict between subtasks in different subtask groups, and the grouping identifier is a payment account; The execution module is used to sequentially execute each subtask in each subtask group; and concurrently execute each subtask in different subtask groups.

10. The device according to claim 9, characterized in that The execution module is specifically used to: start a thread for each subtask group; Based on the thread, each subtask in the subtask group is executed sequentially.

11. The device according to claim 9, characterized in that The execution module is specifically used for: For each subtask group, based on the principle that each thread corresponds to one subtask group, threads of the same number as the number of subtask groups are started, so that each thread executes a subtask in a subtask group.

12. The device according to claim 9, characterized in that The execution module is specifically used for: Determine the status flag of the subtask group; If the status flag indicates that no subtask in the subtask group is in an execution state, an unexecuted subtask is selected from the subtask group according to the business configuration rule, and the status flag of the subtask group is updated, wherein the updated status flag indicates that there is a subtask in the subtask group in an execution state.

13. The device according to claim 12, characterized in that The execution module is further specifically configured to, after determining the status flag of the subtask group, if the status flag indicates that there is a subtask in the subtask group that is in an execution state, wait for the subtask to be completed, and update the status flag of the subtask group, wherein the updated status flag indicates that there is no subtask in the subtask group that is in an execution state; An unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

14. The device according to claim 9, characterized in that The execution module is also specifically used for: For each subtask group, determining the status of each subtask in the subtask group; If there is no subtask in execution at present, an unexecuted subtask is selected from the subtask group for execution according to the business configuration rule.

15. The device according to claim 14, characterized in that The execution module is also specifically used for, for each subtask group, after determining the execution status of each subtask in the subtask group, if there is a subtask in the execution state, then waiting for the subtask to be executed, and selecting an unexecuted subtask from the subtask group for execution according to the business configuration rule.

16. The device according to any one of claims 9 to 15, characterized in that: It also includes a state feedback module for: For each main task, determining the execution status of each subtask in the main task; If all subtasks in the main task are completed, a message that the main task has been completed is fed back.

17. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 8.

19. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or the instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images