Task processing method and device and storage medium

Abstract

The invention discloses a task processing method and device and a storage medium, relates to the technical field of computers, and is used for solving the problem that the efficiency is low when tasks are directly processed according to task understanding in the general technology. The method comprises the steps that after task configuration information of a to-be-executed task is received, the to-be-executed task can be decomposed into a plurality of sub-tasks according to the task configuration information so as to further perform parallel processing on at least one first task and at least one second task, and a processing result corresponding to the to-be-executed task is obtained. Wherein the task configuration information is used for indicating a dependency relationship among a plurality of steps in the to-be-executed task, the plurality of sub-tasks are in one-to-one correspondence with the plurality of steps and comprise at least one first task and at least one second task, the dependency relationship does not exist among different first tasks, and the second task depends on part of the first tasks.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a task processing method, apparatus, and storage medium. Background Art

[0002] With the development of computer technology, in the production and operation of enterprises, the amount of data to be processed is continuously increasing and the processing logic of tasks is becoming more and more complex, resulting in longer processing times for some tasks in enterprises. For example, the processing of various business data of enterprises and the calculation of employees' salaries in enterprises.

[0003] In the general technology, tasks are generally processed directly according to the processing logic of the tasks to obtain the processing results of the tasks. However, due to the large amount of data to be processed and the relatively complex processing logic for some tasks, the processing efficiency of this method of directly processing tasks according to the processing logic of the tasks is relatively low. Summary of the Invention

[0004] This application provides a task processing method, apparatus, and storage medium, which are used to improve the problem of relatively low efficiency when directly processing tasks according to the processing logic of tasks in the general technology, and can improve the efficiency of task processing.

[0005] To achieve the above object, this application adopts the following technical solutions:

[0006] In a first aspect, a task processing method is provided, including: after receiving the task configuration information of the task to be executed, decomposing the task to be executed into multiple subtasks according to the task configuration information, so as to further perform parallel processing on at least one first task and at least one second task respectively to obtain the processing result corresponding to the task to be executed. Among them, the task configuration information is used to indicate the dependency relationship between multiple steps in the task to be executed, and the multiple subtasks correspond one-to-one with the multiple steps, including at least one first task and at least one second task. There is no dependency relationship between different first tasks, and the second task depends on some first tasks.

[0007] Optionally, the task configuration information further includes the task execution time; before decomposing the task to be executed into multiple subtasks according to the task configuration information, the task processing method further includes: determining that the current time is consistent with the task execution time.

[0008] Optionally, the task processing method further includes: updating the execution progress of the task to be executed when the current time is consistent with the task execution time, the first task is executed, or the second task is executed.

[0009] Optionally, the task configuration information further includes authentication information of the initiating object of the task to be executed, and the task processing method further includes: authenticating the authentication information of the initiating object to obtain an authentication result; the authentication result is used to indicate whether to process the task to be executed or not.

[0010] In a second aspect, a task processing system is provided, including: a task management module, a message queue module, a parallel processing module, and a delay scheduling module; the delay scheduling module is configured to schedule the task management module to process the task to be executed when it is determined that the current time is consistent with the task execution time; the task management module is configured to decompose the task to be executed into multiple subtasks and send at least one task request message to the message queue module; at least one task request message corresponds to at least one first task or at least one second task one by one; the parallel processing module is configured to receive at least one task request message from the message queue module and parallelly process the tasks corresponding to at least one task request message.

[0011] In a third aspect, a task processing device is provided, including: a receiving unit, a decomposing unit, and a processing unit; the receiving unit is configured to receive task configuration information of the task to be executed; the task configuration information is used to indicate the dependency relationship between multiple steps in the task to be executed; the decomposing unit is configured to decompose the task to be executed into multiple subtasks according to the task configuration information; the multiple subtasks correspond to the multiple steps one by one, including at least one first task and at least one second task; there is no dependency relationship between different first tasks, and the second task depends on some first tasks; the processing unit is configured to parallelly process at least one first task and at least one second task respectively to obtain a processing result corresponding to the task to be executed.

[0012] Optionally, the task processing device further includes: a determining unit; the determining unit is configured to determine that the current time is consistent with the task execution time.

[0013] Optionally, the task processing device further includes: an updating unit; the updating unit is configured to update the execution progress of the task to be executed when the current time is consistent with the task execution time, the first task is completed, or the second task is completed.

[0014] Optionally, the task processing device further includes: an authentication unit; the authentication unit is configured to authenticate the authentication information of the initiating object to obtain an authentication result; the authentication result is used to indicate whether to process the task to be executed or not.

[0015] Fourthly, a task processing device is provided, which includes a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory through a bus; when the task processing device runs, the processor executes the computer execution instructions stored in the memory, so that the task processing device executes the task processing method described in the first aspect.

[0016] The task processing device may be a network device or a part of a network device, such as a chip system in a network device. The chip system is used to support the network device to implement the functions involved in the first aspect and any possible implementation manner thereof. For example, it is used to obtain, determine, and send the data and / or information involved in the above task processing method. The chip system includes a chip and may also include other discrete devices or circuit structures.

[0017] Fifthly, a computer-readable storage medium is provided. The computer-readable storage medium includes computer execution instructions. When the computer execution instructions run on a computer, the computer is enabled to execute the task processing method described in the first aspect.

[0018] Sixthly, a computer program product is further provided. The computer program product includes computer instructions. When the computer instructions run on the task processing device, the task processing device is enabled to execute the task processing method described in the first aspect as above.

[0019] It should be noted that the above computer instructions may be stored in whole or in part on the first computer-readable storage medium. Among them, the first computer-readable storage medium may be packaged together with the processor of the task processing device or separately packaged with the processor of the task processing device. This application does not make any limitation in this regard.

[0020] The descriptions of the second aspect, the third aspect, the fourth aspect, the fifth aspect, and the sixth aspect in this application may refer to the detailed description of the first aspect; and the beneficial effects of the second aspect, the third aspect, the fourth aspect, the fifth aspect, and the sixth aspect may refer to the analysis of the beneficial effects of the first aspect, which will not be elaborated here.

[0021] In this application, the names of the above task processing devices do not constitute a limitation to the devices or functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of each device or functional module are similar to those of this application and fall within the scope of the claims of this application and their equivalent technologies.

[0022] These aspects or other aspects of this application will be more clearly understood in the following description.

[0023] The technical solutions provided by this application at least bring the following beneficial effects:

[0024] Based on any of the above aspects, the present application provides a task processing method. After receiving the task configuration information of the task to be executed, the task processing device can decompose the task to be executed into multiple subtasks according to the task configuration information, so as to further perform parallel processing on at least one first task and at least one second task respectively to obtain the processing result corresponding to the task to be executed. Among them, the task configuration information is used to indicate the dependency relationship between multiple steps in the task to be executed. The multiple subtasks correspond one by one to the multiple steps, including at least one first task and at least one second task. There is no dependency relationship between different first tasks, and the second task depends on some first tasks.

[0025] Based on this, the present application can decompose the task to be executed into different stages according to the dependency relationship between each step in the task to be executed, and can perform parallel processing on the subtasks within each stage. Compared with the method of directly processing according to the task processing logic, the present application can further decompose the task to be executed, perform parallel processing on the steps that can be executed in parallel in the task to be executed, and reduce the problem of low task processing efficiency caused by the large amount of data to be processed in the task and the complex processing logic, resulting in directly processing the task according to the processing logic. Therefore, the present application can be used to improve the problem of low processing efficiency when directly processing tasks according to the processing logic in the prior art, and improve the processing efficiency of tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic structural diagram of a task processing system provided by an embodiment of the present application;

[0027] Figure 2 It is a schematic structural diagram of a task processing device provided by an embodiment of the present application;

[0028] Figure 3 It is a schematic hardware structure diagram of a task processing device provided by an embodiment of the present application;

[0029] Figure 4 It is a schematic flowchart of a task processing method provided by an embodiment of the present application;

[0030] Figure 5 It is a schematic flowchart of a task processing method provided by an embodiment of the present application;

[0031] Figure 6 It is a schematic flowchart of a task processing method provided by an embodiment of the present application;

[0032] Figure 7 It is a schematic flowchart of a task processing method provided by an embodiment of the present application;

[0033] Figure 8Schematic diagram of a task processing method provided by an embodiment of the present application;

[0034] Figure 9 Structural schematic diagram of a task processing device provided by an embodiment of the present application. Detailed implementation manners

[0035] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0036] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present related concepts in a specific manner.

[0037] In order to facilitate the clear description of the technical solutions in the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and roles are distinguished by using words such as "first" and "second". Those skilled in the art can understand that the words such as "first" and "second" do not limit the quantity and execution order.

[0038] With the development of computer technology, the amount of data processed during task processing is increasing, and the processing logic of tasks is becoming more and more complex, resulting in longer processing time for some tasks. For example, the salary calculation task of the human resources system of large enterprise groups.

[0039] Currently, it is possible to introduce a request set to implement user front-end configuration of the request set, and then submit it to the back-end server for execution, that is, a way of configuring once and executing multiple times can be achieved. This way can realize that the user does not need to monitor the execution of the task after configuring the request set, enabling the user to process other tasks while processing the current task, optimizing the user experience, but there is still a problem of too long task processing time.

[0040] For example, a relatively mature method currently is to introduce a request set in the e-business suite (EBS) system developed by Oracle to separate user configuration and execution. However, with the continuous increase in data volume and the increasing complexity of task processing logic, when facing complex tasks, there is still a problem of excessive task execution time when only using request sets to execute tasks. Moreover, since the human resources system developed by Oracle is not open source, it is necessary to purchase it from Oracle.

[0041] Compared with the method of task scheduling and processing through Oracle EBS, spark, and akka in the prior art, to solve the problem of long task processing time in the prior art, this application provides a parallel processing method for distributed tasks, which combines xxl-job and kafka to achieve parallel processing of timed tasks. Based on xxl-job, it realizes timed scheduling of tasks, and based on kafka, it realizes parallel processing of tasks. Since both xxl-job and kafka are open source processing platforms with good scalability, when new tasks need to be processed, only the decomposition logic and processing logic need to be configured. Subsequently, kafka can be used to achieve task scheduling and display of task processing results. Moreover, kafka itself can support million-level data query and processing, and can also maintain good processing effects when dealing with complex tasks. In addition, this application uses a kafka cluster and an xxl-job cluster to process tasks, improving the stability and processing efficiency of task processing.

[0042] An embodiment of this application provides a task processing method. After receiving the task configuration information of the task to be executed, the task processing device can decompose the task to be executed into multiple subtasks according to the task configuration information, so as to further perform parallel processing on at least one first task and at least one second task respectively to obtain the processing result corresponding to the task to be executed. Among them, the task configuration information is used to indicate the dependency relationship between multiple steps in the task to be executed. The multiple subtasks correspond one-to-one with the multiple steps, including at least one first task and at least one second task. There is no dependency relationship between different first tasks, and the second task depends on some first tasks.

[0043] Based on this, the present application can decompose the task to be executed into different stages according to the dependency relationships between the steps in the task to be executed, and can process the subtasks within each stage in parallel. Compared with the method of directly processing according to the task processing logic, the present application can further decompose the task to be executed, process the steps that can be executed in parallel in the task to be executed in parallel, and reduce the problem of low task processing efficiency caused by the large amount of data to be processed in the task and the relatively complex processing logic. Therefore, the present application can be used to improve the problem of low processing efficiency when directly processing tasks according to the processing logic in the prior art, and improve the processing efficiency of tasks.

[0044] This task processing method is applicable to a task processing system. Figure 1 The structural schematic diagram of a task processing system is shown. As Figure 1 shown, the task processing system 100 includes: a task processing device 101 and an information collection device 102.

[0045] Among them, the task processing device 101 and the information collection device 102 can be communicatively connected.

[0046] Optionally, the connection manner between the task processing device 101 and the information collection device 102 can be a wired connection or a wireless connection.

[0047] In some embodiments, the task processing device 101 can be used to implement the task processing function, and can be a functional module on the information collection device 102 or an entity device independently provided from the information collection device 102.

[0048] It is easy to understand that when the task processing device 101 is a functional module on the information collection device 102, the interaction manner between the task processing device 101 and the information collection device 102 is the interaction between internal modules of the information collection device 102. In this case, the interaction process between the two is the same as the "interaction process between the two in the case where the task processing device 101 is an independently provided entity device".

[0049] For ease of understanding, Figure 1 take the case where "the task processing device 101 and the information collection device 102 are independently provided" as an example for description.

[0050] Figure 1 The task processing device 101 in can receive the task configuration information of the task to be executed from the information collection device 102, and can decompose the task to be executed according to the configuration information of the task to be executed from the information collection device 102 to obtain a plurality of subtasks, and process the plurality of subtasks to obtain the processing result corresponding to the task to be executed.

[0051] Figure 1 The information collection device 102 therein may be configured with a database or connected to a device configured with a database, for collecting configuration information of tasks to be executed, and may send the collected information to the task processing device 101.

[0052] In some embodiments, when the task processing device 101 and the information collection device 102 are independent physical devices, the task processing device 101 and the information collection device 102 may be an independent server or other forms of physical devices. The physical device may be a server in a server cluster (composed of multiple servers), may also be a chip in a physical device, may also be a system on a chip in a physical device, and may also be implemented by a virtual machine deployed on a physical machine. The embodiments of the present application do not limit this.

[0053] In some embodiments, the task processing device 101 and the information collection device 102 may be terminals for realizing human-computer interaction through a display page. The terminal may be a handheld device with a wireless connection function, or a wireless terminal of other processing devices connected to a wireless modem, or may also be a wired terminal. For example, intelligent devices such as mobile phones, personal computers (PCs), desktop computers, tablet computers, laptop computers, netbooks, personal digital assistants (PDAs), etc. The embodiments of the present application do not limit this.

[0054] Combined with Figure 1 the task processing system 100 shown in Figure 2 as shown in Figure 2 is a schematic structural diagram of another task processing device 101 provided by the embodiments of the present application. As

[0055] The task management module 201 is used to decompose the tasks to be executed into multiple subtasks and send at least one task request message to the message queue module 202.

[0056] The message queue module 202 is used to receive at least one task request message sent by the task management module 201 and send at least one task request message to the parallel processing module 203.

[0057] The parallel processing module 203 is used to receive at least one task request message from the message queue module 202 and parallel process the tasks corresponding to at least one task request message.

[0058] The delay scheduling module 204 is configured to schedule the task management module 201 to process the to-be-executed task when it is determined that the current time is consistent with the task execution time.

[0059] Combined with Figure 1 , the task processing device 101 and the information collection device 102 in the task processing system may include Figure 3 the components included in the task processing device shown. Below, taking Figure 3 the task processing device shown as an example, the hardware structures of the task processing device 101 and the information collection device 102 are introduced.

[0060] As Figure 3 shown, it is a schematic diagram of a hardware structure of the task processing device provided by an embodiment of the present application. The task processing device includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, the memory 22, and the communication interface 23 may be connected through the bus 24.

[0061] The processor 21 is the control center of the task processing device, which may be a single processor or a collective term for multiple processing elements. For example, the processor 21 may be a general-purpose central processing unit (CPU), or other general-purpose processors. Among them, the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0062] As an embodiment, the processor 21 may include one or more CPUs, such as Figure 3 the CPU 0 and CPU 1 shown in

[0063] The memory 22 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0064] In one possible implementation, the memory 22 can exist independently of the processor 21. The memory 22 can be connected to the processor 21 through the bus 24 and is used to store instructions or program codes. When the processor 21 calls and executes the instructions or program codes stored in the memory 22, the task processing method provided in the following embodiments of the present invention can be implemented.

[0065] In another possible implementation, the memory 22 can also be integrated with the processor 21.

[0066] The communication interface 23 is used for the task processing device to connect to other devices through a communication network. The communication network can be an Ethernet, a wireless access network, a wireless local area network (WLAN), etc. The communication interface 23 can include a receiving unit for receiving data and a sending unit for sending data.

[0067] The bus 24 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 3 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0068] The task processing method provided in the embodiments of the present application will be introduced in detail below with reference to the accompanying drawings. As Figure 4 shown, the task processing method includes: S401 - S403.

[0069] S401. The task processing device receives the task configuration information of the task to be executed.

[0070] Among them, the task configuration information can be used to indicate the dependency relationship between multiple steps in the task to be executed.

[0071] Optionally, the task configuration information can include the delayed execution time of the task and the authentication information of the user who requests the task to be executed.

[0072] In one possible way, after receiving the task configuration information of the task to be executed, the task processing device can store the task configuration information of the task to be executed in the configured data storage module. The task processing device can audit the processing result of the task to be executed according to the task configuration information stored in the data storage module.

[0073] In a possible implementation, the task processing device may be configured with an information collection module and input modules such as a keyboard and a mouse. The information collection module may be used to collect task configuration information of a to-be-executed task input by a user through the input modules. Based on this, the task processing device may process the to-be-executed task according to the task configuration information of the to-be-executed task input by the user in the information collection module.

[0074] The task processing device may receive task configuration information sent by a user terminal according to the information collection module.

[0075] Exemplarily, in a salary calculation system, the task configuration information may include different departments of an enterprise, salary items of enterprise employees, salary slips of enterprise employees, etc.

[0076] S402. The task processing device decomposes the to-be-executed task into multiple subtasks according to the task configuration information.

[0077] Among them, the multiple subtasks may correspond to multiple steps one by one, and may include at least one first task and at least one second task. There may be no dependency relationship between different first tasks, and the second task may depend on at least one first task.

[0078] In a possible implementation, when the task processing device receives task configuration information of multiple to-be-executed tasks, the task processing device may sort according to the task execution times of the multiple to-be-executed tasks, and the task processing device may preferentially process the to-be-executed tasks with earlier task execution times.

[0079] In a possible implementation, when all the received to-be-executed tasks by the task processing device are atomic tasks, the task processing device may determine atomic tasks that can be executed in parallel according to the task configuration information corresponding to each atomic task.

[0080] In a possible implementation, the task processing device may decompose the to-be-executed task according to the task configuration information, decompose the to-be-executed task into multiple stages, and each stage includes multiple atomic tasks. An atomic task is used to represent the smallest task that can be executed.

[0081] In a possible implementation, after the task processing device decomposes the to-be-executed task into multiple subtasks, it may push the multiple subtasks to a kafka message queue respectively, and then push the message corresponding to each subtask to an idle server for processing through a kafka processing platform.

[0082] Kafka is a distributed data processing platform that can be used to record tasks, allocate tasks to servers for execution, etc. Kafka has the advantages of high parallel processing efficiency, automatic peak shaving, and good stability, which can ensure the orderly, stable, and safe execution of tasks. At the same time, it supports the hot-pluggable real-time expansion of business processing during peak business periods, ensuring the normal execution of the task processing system.

[0083] Exemplarily, it is assumed that there are 10 first subtasks, and the task processing device has a total of 3 task processing units. The task processing device can send the 10 first subtasks to the Kafka message queue, and the Kafka processing platform pushes the first subtasks to the 3 task processing units. When any of the subtasks is completed, the Kafka processing platform can continue to push other first subtasks to the task processing unit.

[0084] In a possible way, the task processing device parses the task configuration information of the task to be executed. When the execution time of the task to be executed does not match the current time, the task processing device can send the task to be executed to XXL-Job, and XXL-Job realizes the scheduled execution of the task to be executed.

[0085] XXL-Job is a distributed task scheduling platform that supports the delayed execution and scheduled execution of tasks. After configuring multiple tasks to be executed, it can schedule multiple tasks to be executed, and can meet the configuration of execution time, execution parameters, etc. It can also realize the viewing of detailed task configuration information and task status.

[0086] S403. The task processing device processes at least one first task and at least one second task in parallel respectively to obtain the processing results corresponding to the tasks to be executed.

[0087] Optionally, the task configuration information may further include the task data identifier corresponding to the task to be executed.

[0088] In a possible way, the task processing device can perform parallel processing on at least one first task based on the task data input by the user through the input module or the task data determined from the database module according to the task data identifier corresponding to the task to be executed, to obtain at least one first processing result corresponding one-to-one to at least one first task. And the task processing device can perform parallel processing on at least one second task based on at least one first processing result to obtain at least one second processing result corresponding one-to-one to at least one second task.

[0089] In a possible way, the task processing device can summarize the second processing results to obtain the processing results corresponding to the tasks to be executed.

[0090] In a possible way, when the task processing device is applied to the salary calculation service, after receiving the salary calculation parameters and salary basic data configured by the user, the task processing device can store the salary calculation parameters and salary basic data in the database. Then, the task processing device can execute tasks in real time or with a delay according to the user configuration, decompose the tasks into sub-tasks that can be calculated in parallel, and push the sub-tasks to the message server kafka. Then, the salary calculation module of the task processing device pulls the kafka tasks and calculates the salary data, and at the same time updates the status of the main task. After obtaining the salary data, the user can check the salary data and determine the final salary calculation result to further realize the payment of the salary.

[0091] The technical solutions provided above have at least the following beneficial effects: As can be seen from S401 - S403, after receiving the task configuration information of the task to be executed, the task processing device can decompose the task to be executed into multiple sub-tasks according to the task configuration information, so as to further perform parallel processing on at least one first task and at least one second task respectively to obtain the processing result corresponding to the task to be executed. Among them, the task configuration information is used to indicate the dependency relationship between multiple steps in the task to be executed. The multiple sub-tasks correspond one by one to the multiple steps, including at least one first task and at least one second task. There is no dependency relationship between different first tasks, and the second task depends on some first tasks.

[0092] Based on this, the present application can decompose the task to be executed into different stages according to the dependency relationship between each step in the task to be executed, and can parallelly process the sub-tasks within each stage. Compared with the method of directly processing according to the task processing logic, the present application can further decompose the task to be executed, parallelly process the steps that can be executed in parallel in the task to be executed, and reduce the problem of low task processing efficiency caused by the large amount of data to be processed by the task and the relatively complex processing logic, resulting in directly processing the task according to the processing logic. Therefore, the present application can be used to improve the problem of low processing efficiency in the general technology when directly processing tasks according to the processing logic, and improve the processing efficiency of tasks.

[0093] In an embodiment, the task configuration information further includes the task execution time. Combining Figure 4 , before S402, that is, before decomposing the task to be executed into multiple sub-tasks according to the task configuration information, the task processing method provided by the present application further includes: S501.

[0094] S501. The task processing device determines that the current time is consistent with the task execution time.

[0095] In a possible way, the task processing device can push the task to be executed to a configured delayed task scheduling platform. When the current time is consistent with the task execution time, the task processing device can decompose the task to be executed.

[0096] In a possible way, the task processing device can be configured with a timer, and the task processing device can implement timed processing and delayed processing of tasks through the timer.

[0097] In a possible way, the task processing device can receive the task configuration information of the task to be executed at any time, and the task processing device can implement the scheduling of the task to be executed through xxl-job.

[0098] In a possible way, when the task to be executed needs to be executed regularly, after the task processing device decomposes the task to be executed into multiple subtasks and processes the multiple subtasks, it can modify the task execution time of the task to be executed to achieve the regular execution of the task to be executed.

[0099] Exemplarily, it is preset that the employee salary calculation task needs to be executed on the 10th day of each month. After the task processing device obtains the task result of the employee salary calculation task for this month, it can modify the task execution time of the employee salary calculation task to the 10th day of the next month to achieve the automatic execution of the employee salary calculation task on the 10th day of each month.

[0100] Exemplarily, as Figure 5 shown, for tasks that need to be executed immediately and tasks that need to be executed regularly, the specific process includes:

[0101] After the task processing device receives a task that needs to be executed immediately, the task processing device can record the task data and task configuration information of the task, generate a task request of type M according to the task configuration information, then divide the M task request into multiple stage tasks S, and further generate subtasks P for each stage task S, so as to sequentially push multiple subtasks P of each stage to the server.

[0102] When the task processing device receives a task that requires a plan request, the task processing device can record the task data and task configuration information of the task, generate a task request of type M according to the task configuration information, and push the task request M to xxl-job. When the task execution time is consistent with the current time, the task processing device receives the task request M pushed by xxl-job, then divides the M task request into multiple stage tasks S, and further generates subtasks P for each stage task S, so as to sequentially push multiple subtasks P of each stage to the server.

[0103] The task processing device can monitor the execution status of the scheduled request task. When the scheduled request task is completed, the task processing device can regenerate the task request M and modify the execution time of the task request M to achieve the periodic execution (scheduled execution) of the request task. When the scheduled request task is not completed, the task processing device can generate a push message for the next subtask in the scheduled request task and push the push message for the next subtask to the server.

[0104] The technical solution provided above has at least the following beneficial effects: As can be seen from S501, before the task processing device decomposes the task to be executed into multiple subtasks, it can determine whether the current time is consistent with the task execution time. If the current time is consistent with the task execution time, the task to be executed is decomposed. Therefore, this application can be used to implement the timed processing of the task to be executed, reduce the user waiting time, and optimize the user experience.

[0105] In another embodiment, to facilitate the display of the task execution progress to the user, the task processing method provided in this application further includes: S601.

[0106] S601. The task processing device updates the execution progress of the task to be executed when the current time is consistent with the task execution time, the first task is completed, or the second task is completed.

[0107] In a possible way, when the current time is consistent with the task execution time, the task processing device can update the task progress of the task to be executed to the start of task execution. When the first task is completed, the task processing device can update the task progress of the task to be executed to 50% completion of the task. When both the first and second tasks are completed, the task processing device can update the task progress of the task to be executed to the completion of task execution.

[0108] Optionally, the task processing device can also represent the execution progress of the task to be executed through the task status of the task to be executed. The task status can include not executed, started execution, in execution, execution completed, and task execution error, etc.

[0109] In a possible way, after the task processing device updates the execution progress of the task to be executed, it can send the task progress of the task to be executed to the terminal that configures the task to be executed, which is convenient for monitoring the task to be executed.

[0110] When each first task or second task is completed, the task processing device can determine the execution progress of the task to be executed according to the number of the first task or the second task.

[0111] Exemplarily, it is preset that there are 5 first tasks and 5 second tasks in total. When 3 first tasks are completed, the task processing device can determine the execution progress as 30% of the task execution completion. When 5 first tasks are completed, the task processing device can determine the execution progress of the to-be-executed task as 50% of the task execution completion. When all 5 first tasks and 5 second tasks are completed, the task processing device can determine the execution progress of the to-be-executed task as the task execution completion.

[0112] Exemplarily, it is preset that there are 3 first tasks and 3 second tasks in total. When 3 first tasks and 2 second tasks are completed and the remaining one second task is executed incorrectly, the task processing device can determine the task execution progress as 90% of the task execution completion, and one second task is executed abnormally.

[0113] The technical solution provided above has at least the following beneficial effects: As can be seen from S601, the task processing device can update the execution progress of the to-be-executed task when the current time is consistent with the task execution time, the first task is completed, or the second task is completed. Therefore, the present application can update the execution progress of the task processing device in real time, which is convenient for monitoring the to-be-executed task and realizing the orderly execution of the to-be-executed task.

[0114] In another embodiment, the task configuration information may include the authentication information of the initiator of the to-be-executed task. After S401 above, that is, after the task processing device receives the task configuration information of the to-be-executed task, the task processing method provided by the present application further includes: S701.

[0115] S701. The task processing device authenticates the authentication information of the initiator and obtains an authentication result.

[0116] Among them, the authentication result can be used to indicate whether to process the to-be-executed task or not.

[0117] Optionally, the authentication information may be an identity credential or data representing the initiator of the to-be-executed task. The authentication information may include authentication information such as a username and password, a dynamic token, and permission information.

[0118] In a possible way, after receiving the task configuration information, the task processing device can verify the authentication information of the initiator of the configured to-be-executed task to obtain the authentication result of the to-be-executed task configuration information. When the to-be-executed task authentication passes, the task processing device can implement the scheduling of the to-be-executed task through xxl-job. When the to-be-executed task authentication fails, the task processing device can discard the to-be-executed task or send the task configuration information of the to-be-executed task to the terminal of the initiator of the to-be-executed task.

[0119] The technical solutions provided above bring at least the following beneficial effects: As can be seen from S701, the task processing device can authenticate the authentication information of the initiating object, obtain the authentication result, and determine whether to execute the task to be executed according to the authentication result. Therefore, authenticating the task to be executed before executing it can improve the security of the task processing device, prevent data leakage, and ensure the safe and stable operation of the task processing device.

[0120] Exemplarily, as Figure 6 shown, it is a schematic flowchart of a task processing method provided by this application. After the user submits task configuration information, the task processing device can verify the user's authentication information through the authentication information in the database module and return the verification result. When the verification passes, the task processing device can store the configuration information of the task to be executed in the database module. When the verification fails, the task processing device can return the task configuration information of the task to be executed.

[0121] After the verification passes, the task processing device can decompose the task to be executed into multiple subtasks through the parallel manager and request the task data of the multiple subtasks from the database module. After receiving the task data of the multiple subtasks sent by the data, the task processing device can verify the received task data (such as matching manager verification and exclusive verification, etc.) to determine the integrity of the task data.

[0122] For tasks that need to be executed immediately (such as non-payroll calculation tasks and salary calculation tasks, etc.) and tasks that need to be manually started (such as tasks with abnormal execution), the task processing device can send their corresponding subtasks to the kafka processing module configured in the task processing device. The kafka processing module calls the execution program from the consumer server, determines the processing result of each subtask according to the execution program, and sends the processing result to the database module.

[0123] For tasks that need to be executed according to a plan (i.e., executed regularly), after the task processing device receives the user's plan request, it can schedule the execution program through the scheduling service module to achieve the planned execution of the task. When the task execution time matches the current time, the task processing device can start the task to be executed by the scheduling service module, decompose the task to be executed into multiple subtasks through the parallel manager, and then obtain the task data corresponding to the multiple subtasks from the database module. After the database module returns the task data, the task processing device verifies the task data. After the verification passes, the task processing device sends its corresponding subtasks to the kafka processing module configured in the task processing device. The kafka processing module calls the execution program from the consumer server, determines the processing result of each subtask according to the execution program, and sends the processing result to the database module.

[0124] Whenever a subtask starts execution or is completed, the task processing device can update the execution status of the task, generate log data of the task, and store the log data of the task in the database module.

[0125] Combined with the above example, when the task processing device is applied to a salary calculation system, as Figure 7 shown, the present application provides a schematic flowchart of a task processing method. After receiving a task request and task information sent by an operating user, the task processing device can store the task request and the corresponding task information in a database.

[0126] For tasks that need to be executed immediately or scheduled tasks sent by xxl-job, the task processing device can decompose the tasks according to the task information and send the decomposed subtasks to the kafka cluster. Among them, the task information may include a task identifier, a user identifier, and task configuration information. The kafka cluster contains multiple partitions, and each partition includes a leader server and multiple follower servers. The leader is responsible for reading and writing task data requests within the same partition, and the follower is used to synchronize the task data in the leader. When the leader fails, the follower can become the new leader.

[0127] After receiving the subtasks, the kafka cluster can send the task data to the standard manager (topic-standaed) service cluster, the customized management (topic-customize) service cluster, and the salary calculation (topic-cal) manager service cluster according to the configuration information of the subtasks. After obtaining the task data, each server cluster can execute multiple subtasks in parallel through multiple configured servers, and after processing the results of the multiple subtasks through the redis cluster, store the processing results in the database cluster. The database cluster stores the processing results in the file server. At the same time, each server cluster can send the processing logs of the multiple subtasks to the log service cluster through the kafka cluster, and the log (topic-logs) manager service cluster stores the processing logs of the multiple subtasks in the file server.

[0128] After receiving the processing results of multiple subtasks, the database cluster can generate a task result report and store the task result report in the file server. The task processing device can request the task configuration information of the user and the administrator to control access to the file server, and can implement cleaning the processing logs in the file server every three months through the xxl-job cluster and creating a new log file for storing the processing logs.

[0129] As Figure 8As shown in the figure, the present application provides a schematic diagram of a task processing method. The manager in each service cluster can implement parallel processing of tasks through multiple actor models. Each actor can include multiple sub-actors. The actor can send multiple subtasks to a thread pool, and the thread pool sends the multiple subtasks to multiple sub-actors to implement parallel processing of tasks. The task processing device can combine the actor reuse mechanism with the thread pool to ensure the execution efficiency and stability of tasks.

[0130] In a possible way, the task processing device can adopt multiple replicas of the kafka cluster, increase consumer services, configure and control the number of consumers, and push messages in the acknowledge character (ACK) mode to ensure data security and the stable operation of the task processing device.

[0131] In a possible way, the operating user can view the task results through the function interface in the task processing device.

[0132] The above mainly introduces the solution provided by the embodiments of the present application from the perspective of the method. To implement the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that the present application can be implemented in the form of hardware or a combination of hardware and computer software in combination with the units and algorithm steps of each example described in the embodiments disclosed herein. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0133] The embodiments of the present application can divide the function modules of the task processing device according to the above method examples. For example, each function module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software function module. Optionally, the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0134] As Figure 9 shown, it is a schematic structural diagram of a task processing device provided by an embodiment of the present application. The task processing device can be used to execute Figure 4 the task processing method shown. Figure 9 The task processing device shown includes: a receiving unit 901, a decomposing unit 902, and a processing unit 903.

[0135] A receiving unit 901, configured to receive task configuration information of a to-be-executed task; the task configuration information is used to indicate the dependency relationship between multiple steps in the to-be-executed task. For example, in combination with Figure 4 , the receiving unit 901 can be used to execute S401.

[0136] A decomposition unit 902, configured to decompose the to-be-executed task into multiple subtasks according to the task configuration information; the multiple subtasks correspond to the multiple steps one by one, including at least one first task and at least one second task; there is no dependency relationship between different first tasks, and the second task depends on some of the first tasks. For example, in combination with Figure 4 , the decomposition unit 902 can be used to execute S402.

[0137] A processing unit 903, configured to perform parallel processing on at least one first task and at least one second task respectively to obtain a processing result corresponding to the to-be-executed task. For example, in combination with Figure 4 , the processing unit 903 can be used to execute S403.

[0138] Optionally, the task processing device further includes: a determination unit 904.

[0139] The determination unit 904 is configured to determine that the current moment is consistent with the task execution moment. For example, the determination unit 904 can be used to execute S501.

[0140] Optionally, the task processing device further includes: an update unit 905.

[0141] The update unit 905 is configured to update the execution progress of the to-be-executed task when the current moment is consistent with the task execution moment, the first task is executed, or the second task is executed. For example, the update unit 905 can be used to execute S601.

[0142] Optionally, the task processing device further includes: an authentication unit 906.

[0143] The authentication unit 906 is configured to authenticate the authentication information of the initiating object to obtain an authentication result; the authentication result is used to indicate whether to process the to-be-executed task or not. For example, the authentication unit 906 can be used to execute S701.

[0144] An embodiment of the present application further provides a computer-readable storage medium, where the computer-readable storage medium includes computer-executable instructions, and when the computer-executable instructions run on a computer, the computer is caused to execute the task processing method provided in the foregoing embodiment.

[0145] The embodiments of the present application further provide a computer program. This computer program can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, this computer program can implement the task processing method provided in the above embodiments.

[0146] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. The computer-readable medium includes computer-readable storage media and communication media, where the communication media includes any medium that facilitates the transmission of a computer program from one place to another. The storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0147] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0148] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical functional division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated. The components displayed as units can be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, in each embodiment of the present invention, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the common technology, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0150] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A task processing method, characterized in that, including: receiving task configuration information of a to-be-executed task; the task configuration information is used to indicate the dependency relationship between multiple steps in the to-be-executed task; decomposing the to-be-executed task into multiple subtasks according to the task configuration information; the multiple subtasks correspond one-to-one to the multiple steps, and include at least one first task and at least one second task; there is no dependency relationship between different first tasks, and the second task depends on some first tasks; performing parallel processing on the at least one first task and the at least one second task respectively to obtain a processing result corresponding to the to-be-executed task.

2. The task processing method according to claim 1, wherein the task configuration information further includes a task execution time; before decomposing the to-be-executed task into multiple subtasks according to the task configuration information, it further includes: determining that the current time is consistent with the task execution time.

3. The task processing method according to claim 2, wherein further including: updating the execution progress of the to-be-executed task when the current time is consistent with the task execution time, the first task is completed, or the second task is completed.

4. The task processing method according to claim 1, wherein the task configuration information further includes authentication information of the initiator of the to-be-executed task; the method further includes: authenticating the authentication information of the initiator to obtain an authentication result; the authentication result is used to indicate whether to process the to-be-executed task or not.

5. A task processing system, characterized in that, including: a task management module, a message queue module, a parallel processing module, and a delay scheduling module; the delay scheduling module is used to schedule the task management module to process the to-be-executed task when it is determined that the current time is consistent with the task execution time; the task management module is used to decompose the to-be-executed task into multiple subtasks and send at least one task request message to the message queue module; the at least one task request message corresponds one-to-one to the at least one first task or corresponds one-to-one to the at least one second task; the parallel processing module is used to receive the at least one task request message from the message queue module and perform parallel processing on the tasks corresponding to the at least one task request message.

6. A task processing device, characterized in that, including: a receiving unit, a decomposing unit, and a processing unit; the receiving unit is used to receive task configuration information of a to-be-executed task; the task configuration information is used to indicate the dependency relationship between multiple steps in the to-be-executed task; the decomposing unit is used to decompose the to-be-executed task into multiple subtasks according to the task configuration information; the multiple subtasks correspond one-to-one to the multiple steps, and include at least one first task and at least one second task; there is no dependency relationship between different first tasks, and the second task depends on some first tasks; the processing unit is used to perform parallel processing on the at least one first task and the at least one second task respectively to obtain a processing result corresponding to the to-be-executed task.

7. The task processing device according to claim 6, wherein the task configuration information further includes a task execution time; it further includes: a determining unit; the determining unit is used to determine that the current time is consistent with the task execution time.

8. The task processing device according to claim 7, wherein further including: an updating unit; The update unit is configured to update the execution progress of the task to be executed when the current moment is consistent with the task execution moment, the first task is executed completely, or the second task is executed completely.

9. A task processing device, characterized in that, It includes a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory through a bus; when the task processing device runs, the processor executes the computer execution instructions stored in the memory, so that the task processing device executes the task processing method according to any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer execution instructions, and when the computer execution instructions run on a task processing device, the task processing device is caused to execute the task processing method according to any one of claims 1-4.

Images