Task processing method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, in particular to a task processing method and device, equipment and a medium, and is used for solving the problem that a task fails due to the fact that the task cannot be retry when equipment for processing a flow is abnormal. The method comprises the steps of receiving a task notification; wherein the task notification is published after each task in the batch tasks is added to a message queue of a corresponding type according to the task type of the task when the publishing equipment receives the batch tasks; if the published theme corresponding to the task notification is the predetermined theme, obtaining a to-be-processed task from the task notification, and processing the to-be-processed task; and if the to-be-processed task fails to be processed and the processing failure reason of the to-be-processed task is a network reason, recording the state of the to-be-processed task as a failure state, and when the processing failure reason of the to-be-processed task is eliminated, sending the to-be-processed task to the processing equipment for processing the abnormal task. Therefore, the processing equipment can process the to-be-processed task in the failure state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a task processing method and device, equipment and a medium. BACKGROUND

[0002] In the financial field and real-time transaction systems such as e-commerce, batch processing and online processing are common processing modes. Batch processing is usually used for batch processing of a large amount of data, while online processing is used for real-time processing of single or small transaction requests. In some applications, batch processing data needs to be synchronized to the online processing system in real time to achieve higher data real-time performance.

[0003] The traditional mechanism of batch processing to online processing is implemented by using asynchronous threads. Batch tasks are decomposed into multiple small tasks, which are put into an asynchronous thread queue. Once an exception occurs in the device processing the flow, the retry cannot be automatically initiated, resulting in task failure. SUMMARY

[0004] Embodiments of the present application provide a task processing method, device, equipment and medium, which are used to solve the problem that when an exception occurs in a device processing a flow, the task cannot be retried, resulting in task failure.

[0005] In a first aspect, the present application provides a task processing method, comprising:

[0006] receiving a task notification; wherein the task notification is published by a publishing device when a batch task is received, and each task in the batch task is added to a message queue of a corresponding type according to the task type of the task, and then published;

[0007] if the publishing topic corresponding to the task notification is a predetermined topic, obtaining a to-be-processed task from the task notification, and processing the to-be-processed task;

[0008] if the to-be-processed task processing fails and the to-be-processed task processing failure reason is a network reason, recording the state of the to-be-processed task as a failure state, and when the to-be-processed task processing failure reason is eliminated, sending the to-be-processed task to a processing device processing an abnormal task, so that the processing device can process the to-be-processed task in the failure state.

[0009] The method can publish a message queue for each task in the batch task when receiving the batch task, perform task processing when the publishing theme is a predetermined theme, record a state of the task as a failure state when the task fails and the failure reason is a network reason, and send the task in the failure state to a processing device for processing abnormal tasks after the failure reason is eliminated, and the processing device can process the task in the failure state. Thus, the problem that a task fails due to the abnormality of a device in a processing flow and the task cannot be retried is solved, and the success rate of the task is improved.

[0010] In a possible embodiment, before sending the task to be processed to the processing device for processing abnormal tasks, the method further includes:

[0011] determining that the number of retries of the task to be processed is less than a retry threshold corresponding to the task to be processed, wherein the retry threshold corresponding to the task to be processed is recorded in the task notification.

[0012] In a possible embodiment, after processing the task to be processed, the method further includes:

[0013] If the task to be processed fails in processing and the processing failure reason of the task to be processed is a business reason, the task to be processed is added to a target file, and the target file is displayed to a manager.

[0014] In a possible embodiment, after processing the task to be processed, the method further includes:

[0015] If the task to be processed succeeds in processing, a state of the task to be processed is recorded as a success state.

[0016] In a possible embodiment, after recording the state of the task to be processed as the failure state, the method further includes:

[0017] If the task to be processed is part of a target task, processing of a first subtask in the target task is suspended, and processing of a second subtask in the target task is continued, wherein the first subtask is a subtask in the target task that has a dependency relationship with the task to be processed, and the second subtask is a subtask in the target task that has no dependency relationship with the task to be processed.

[0018] After recording the state of the task to be processed as the success state, the method further includes:

[0019] If the task to be processed is part of a target task and states of other subtasks of the target task are success states, a state of the target task is determined as a success state.

[0020] In a possible embodiment, the obtaining of the to-be-processed task from the task notification comprises:

[0021] The database comprises a plurality of tasks and a plurality of key-value pairs.

[0022] In a second aspect, the present application provides a task processing apparatus, comprising:

[0023] The receiving module is configured to receive a task notification, wherein the task notification is published by a publishing device after adding each task in a batch of tasks to a message queue of a corresponding type according to a task type of the task.

[0024] The obtaining module is configured to obtain a to-be-processed task from the task notification if a publishing topic corresponding to the task notification is a predetermined topic, and process the to-be-processed task.

[0025] The exception processing module is configured to record a state of the to-be-processed task as a failure state if the to-be-processed task fails to be processed and a reason for the failure of the to-be-processed task is a network reason, and send the to-be-processed task to a processing device for processing an abnormal task when the reason for the failure of the to-be-processed task is eliminated, so that the processing device can process the to-be-processed task in the failure state.

[0026] In a possible embodiment, the exception processing module is further configured to:

[0027] The exception processing module is configured to determine that a retry number of the to-be-processed task is less than a retry threshold corresponding to the to-be-processed task, wherein the retry threshold corresponding to the to-be-processed task is recorded in the task notification.

[0028] In a possible embodiment, the exception processing module is further configured to:

[0029] The exception processing module is configured to add the to-be-processed task to a target file and display the target file to a management personnel if the to-be-processed task fails to be processed and a reason for the failure of the to-be-processed task is a business reason.

[0030] In a possible embodiment, the apparatus further comprises:

[0031] The processing success module is configured to record a state of the to-be-processed task as a success state if the to-be-processed task is successfully processed.

[0032] In a possible embodiment, the exception processing module is further configured to:

[0033] If the to-be-processed task is part of a target task, a state of the target task is determined as a pause state, and other subtasks in the target task are paused from being processed; wherein the other subtasks in the target task are subtasks in the target task that have a dependency relationship with the to-be-processed task.

[0034] The processing success module is further configured to:

[0035] If the to-be-processed task is part of a target task, and a state of other subtasks of the target task is a success state, a state of the target task is determined as a success state.

[0036] In a possible embodiment, the obtaining module is specifically configured to:

[0037] obtain, from a database, a to-be-processed task to which a task identifier in the key-value pair corresponding to the to-be-processed task in the task notification belongs; wherein the database includes a plurality of tasks and a plurality of key-value pairs.

[0038] In a third aspect, the present application provides an electronic device, comprising:

[0039] a memory configured to store program instructions;

[0040] a processor configured to invoke the program instructions stored in the memory, and perform steps included in the method according to the obtained program instructions.

[0041] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions, when executed by a computer, cause the computer to execute the method according to any one of the first aspect.

[0042] In a fifth aspect, the present application provides a computer program product, the computer program product includes computer program code, and when the computer program code runs on a computer, causes the computer to execute the method according to any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A schematic structural diagram of a cluster is provided for an embodiment of the present application.

[0044] Figure 2 A flowchart of a task processing method is provided for an embodiment of the present application.

[0045] Figure 3 A schematic diagram of a first task processing is provided for an embodiment of the present application.

[0046] Figure 4A second task processing schematic diagram provided for the embodiments of the present application;

[0047] Figure 5 A schematic diagram of interaction between a subscription device and a database provided for the embodiments of the present application;

[0048] Figure 6 Another cluster structure schematic diagram provided for the embodiments of the present application;

[0049] Figure 7 A structure diagram of a task processing apparatus provided for the embodiments of the present application;

[0050] Figure 8 A structure diagram of an electronic device provided for the embodiments of the present application. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without any creative work fall within the scope of protection of the present application. The embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily without conflict. And, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from the one here.

[0052] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, but not to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover the non-exclusive protection. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to the process, method, product or device. "Multiple" in the present application can mean at least two, for example, can be two, three or more, and the embodiments of the present application are not limited.

[0053] Exemplary embodiments of the present application are described herein with reference to the accompanying drawings, which are cited by way of example only. The present application is therefore not limited to the embodiments described herein, but encompasses all possible embodiments within the scope of the present application. As such, various changes and modifications can be suggested to one skilled in the art, and it is intended that the present application encompass such changes and modifications as fall within the scope of the appended claims. Also, for the sake of brevity, descriptions of functions and constructions that are well known to one skilled in the art can be omitted. It should be noted that, in the embodiments of the present application, some software, components, models, etc. that are available in the industry can be mentioned, which should be considered as exemplary, and the purpose is merely to illustrate the feasibility of the implementation of the technical solutions of the present application, but does not mean that the applicant has or must have used the scheme.

[0054] In the technical solutions of the present application, the acquisition, transmission, storage, and use of data comply with relevant national laws and regulations.

[0055] Before introducing the task processing method provided by the embodiments of the present application, in order to facilitate understanding, first, the technical background of the embodiments of the present application is introduced in detail.

[0056] In the financial field and real-time transaction systems such as electronic commerce, batch processing and online processing are common processing modes. Batch processing is usually used to process a large amount of data in batches, and online processing is used to process single or small transaction requests in real time.

[0057] In some applications, batch processing data needs to be synchronized to the online processing system in real time to achieve higher data real-time performance and performance. However, if the device appears abnormal, the processing flow cannot automatically initiate a retry, resulting in a high task failure rate.

[0058] To solve the above technical problems, the present application proposes the following technical concept: using a publish and subscribe mode for batch task processing, and when the task processing fails and the failure reason of the task processing is network, the task is added to the processing device for processing abnormal tasks, and the processing device processes the to-be-processed tasks in the failure state, thereby improving the success rate of the task.

[0059] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0060] The embodiments of the present application provide a cluster, which includes a plurality of devices, which can be respectively a publishing device for task publishing, a processing device for processing abnormal tasks, and a plurality of subscription devices for processing tasks of different business types, for example,Figure 1 As shown, the cluster includes a publishing device 10, a processing device 11, a subscription device 12, a subscription device 13, a subscription device 14, and a database 15. Subscription device 12 is used to process tasks of the first business type, subscription device 13 is used to process tasks of the second business type, and subscription device 14 is used to process tasks of the third business type.

[0061] In detail, after receiving batch tasks, the publishing device 10 identifies the task type of each task in the batch and generates a message queue for each service type. The message queue uses a task identifier to represent the data of the task itself, combined with... Figure 1 , Figure 3 and Figure 4 As shown, the message queues of publishing device 10 include a first message queue, a second message queue, and a third message queue. The first message queue includes tasks of a first business type, the second message queue includes tasks of a second business type, and the third message queue includes tasks of a third business type. For each business type message queue, taking the first business type message queue as an example, according to the order of tasks in the first business type message queue, publishing device 10 publishes task notifications for tasks belonging to the first business type with the first business type as the topic. The task notification includes a task identifier belonging to the first business type and related task information. Subscribing devices 12 through 14 all receive this task notification. Subscribing devices 13 and 14 have not subscribed to the first business type topic, so they do not process it. Subscribing device 12 subscribes to the first business type topic, so after receiving the task notification for a task belonging to the first business type, it processes the task. Similarly, the processing of tasks of the second and third business types follows the same procedure.

[0062] The batch tasks can be stored in database 15. Each task and task identifier in database 15 are stored as key-value pairs. When storing the task identifier, task-related information such as task status, processing progress, number of retries, and retry threshold can also be stored. The record format can be as shown in the table below:

[0063] Unique identification of a task Task status Processing progress Number of retries Retry threshold

[0064] For the specific processing of subscription devices, combined with Figure 2 As shown in the figure, this application provides a task processing method, including:

[0065] S200: Receive task notification; where the task notification is issued by the publishing device after receiving a batch of tasks, adding each task in the batch to the corresponding type of message queue according to the task type.

[0066] In detail, due to the characteristics of the publish-subscribe mode, in combination with Figure 1 and Figure 3 , the publishing device 10 sends a task notification to the subscribing device 12, the subscribing device 13 and the subscribing device 14.

[0067] S201: If the publishing topic corresponding to the task notification is a predetermined topic, the subscribing device acquires the to-be-processed task from the task notification and processes the to-be-processed task.

[0068] The subscribing device sets a topic that needs to be processed by itself, for example, the predetermined topic of the subscribing device 12 is topic 1, the predetermined topic of the subscribing device 13 is topic 2, and the predetermined topic of the subscribing device 14 is topic 3.

[0069] In combination with Figure 3 , when the subscribing device 12 receives the task notification, if the publishing topic corresponding to the task notification is topic 1, the subscribing device 12 processes the task, and if the publishing topic corresponding to the task notification is topic 2, which is different from the topic 1 set by the subscribing device 12, the subscribing device 12 does not process the task. The subscribing device 13 and the subscribing device 14 have the same processing process. Of course, the predetermined topics of different subscribing devices are different.

[0070] In some embodiments, the embodiments of the present application provide a manner of acquiring a to-be-processed task from a task notification, specifically, acquiring, from a database, a to-be-processed task corresponding to a task identifier in a key-value pair to which a task identifier belongs; wherein the database includes a plurality of tasks and a plurality of key-value pairs.

[0071] In detail, as shown in Figure 1 and Figure 5 , the database 15 records tasks and corresponding task identifiers, the subscribing device 12 acquires the task identifier from the task notification after receiving the task notification, and then sends a message to the database to request to acquire the to-be-processed task in the database, the request including the task identifier of the to-be-processed task, the database 15 receives the request, and according to the task identifier in the request, checks the key-value pair stored by the database 15, thereby obtaining the specific content of the to-be-processed task, and sends the specific content of the to-be-processed task to the subscribing device. After the subscribing device 12 receives the specific content of the to-be-processed task, the subscribing device 12 processes the to-be-processed task.

[0072] S202: If the to-be-processed task processing fails and the reason for the to-be-processed task processing failure is a network problem, the state of the to-be-processed task is recorded as a failure state, and when the reason for the to-be-processed task processing failure is eliminated, the to-be-processed task in the failure state is sent to a processing device for processing abnormal tasks, so that the processing device can process the to-be-processed task in the failure state.

[0073] In this context, "network issues" can be understood as a situation where the subscribing device loses connection with other devices in the cluster. Normally, devices in a cluster periodically send heartbeat signals to ensure the network of each device in the cluster is functioning correctly. If a device fails to send a heartbeat signal periodically, it indicates a network problem with that device. Therefore, if a subscribing device periodically sends a heartbeat signal during the processing of a pending task and receives no response, the task is considered to have failed due to network issues. For example, the failure reason can be represented by an error code, with error codes starting with 1 to indicate a network problem.

[0074] When processing devices add tasks to the message queue, they are categorized according to business type, just like publishing devices. Figure 3 As shown, the processing device divides the abnormal task message queues into three types based on service type. The first abnormal task message queue includes abnormal tasks of the first service type, the second abnormal task message queue includes abnormal tasks of the second service type, and the third abnormal task message queue includes abnormal tasks of the third service type. Combined with... Figure 3 As shown, when the processing device receives a failed task, it adds the task to the first abnormal task message queue according to the business type. The processing device then returns the task to the original subscribing device according to the order of tasks in the abnormal task message queue for the corresponding business type, allowing the original subscribing device to continue processing the task. For example, the first abnormal task message queue in the processing device contains tasks 1, 2, 3, and 4. The original subscribing device for task 1 is subscribing device 12, for task 2 it is subscribing device 12, for task 3 it is subscribing device 13, and for task 4 it is subscribing device 14. When the processing device receives a task after task 4, it receives task 5. The original subscribing device for task 5 is the same as the subscribing device for the task to be processed, for example, subscribing device 12.

[0075] When a task fails to process, the device sends Task 1 from the exception task message queue to the subscribing device 12, which then reprocesses Task 1. Next, it sends Task 2 from the exception task message queue to the subscribing device 12, which reprocesses Task 2. Then, it sends Task 3 from the exception task message queue to the subscribing device 13, which reprocesses Task 3. Finally, it sends Task 4 from the exception task message queue to the subscribing device 14, which reprocesses Task 4. Finally, it sends Task 5 from the exception task message queue to the subscribing device 12, which reprocesses Task 5.

[0076] After the subscription device 12 records the status of the pending task as failed, it notifies the database 15 of this information, and the database 15 records the status of the pending task as failed.

[0077] When the subscription device 12 reprocesses task 5, it modifies the status of task 5 to "processing" and sends the "processing" message to the database 15. The database 15 records the status of task 5 as "processing".

[0078] When the processing device processes the abnormal task message queue, this application embodiment also provides a scheme in which the processing device directly processes the tasks according to the order of the tasks in the abnormal task message queue. For example, after processing task 1, task 2 is processed, task 3 is processed, task 4 is processed, task 5 is processed, and so on.

[0079] Combination Figure 4 As shown, when the processing device adds tasks to the message queue, they are all added to the same message queue, without being categorized according to business type. When processing device 110 receives a task in a failed state, it adds the task to the abnormal task message queue. Processing device 110 will then return the task to the original subscribed device according to the order of tasks in the abnormal task message queue, and continue processing the task using the original subscribed device.

[0080] For example, taking the process of a bank providing users with a balance via SMS during a transaction as an example, since the number of such transactions is relatively large, it needs to be processed in batches, i.e., forming batch tasks. Each batch task can be categorized according to the business type, forming multiple message queues according to the business type. Then, the task notification is published according to the queuing order of the tasks in each message queue. After receiving the task notification, the subscribing device determines that the subject is one it has subscribed to and processes the task, i.e., sends the balance to the communication system. The communication system provides the balance to the user via SMS. After the communication system completes the task, it sends feedback to the subscribing device. After receiving this feedback, the subscribing device determines that the task status is complete. If the subscribing device does not receive heartbeat feedback within a preset time, it determines that the task status is failed, with the reason for failure being network issues. The preset time is the time when feedback is received. The task is then sent to the processing device for handling abnormal tasks. The processing device adds the task to the abnormal task message queue. When the processing device processes the task, it processes the tasks one by one according to the task order in the abnormal task message queue.

[0081] To avoid the problem of unlimited retries, the method further includes the following step before sending the task to be processed to the processing device for handling abnormal tasks:

[0082] The retry number of the to-be-processed task is determined to be less than the retry threshold corresponding to the to-be-processed task, wherein the retry threshold corresponding to the to-be-processed task is recorded in the task notification.

[0083] In detail, the subscription device records the retry number of the task, that is, the number of times of receiving the same task sent by the processing device, before sending the to-be-processed task to the processing device for processing the abnormal task, it is determined whether the retry number of the to-be-processed task is less than the retry threshold corresponding to the to-be-processed task, if the retry number of the to-be-processed task is less than the retry threshold corresponding to the to-be-processed task, the to-be-processed task is sent to the processing device for processing the abnormal task, if the retry number of the to-be-processed task is equal to or greater than the retry threshold corresponding to the to-be-processed task, the to-be-processed task is added to the target file, and the target file is displayed to the administrator.

[0084] In order to solve the problem of unlimited retries, the embodiment of the application provides another implementation scheme, after the processing device receives the to-be-processed task, the number of times of receiving the to-be-processed task is recorded, and the number of times is increased by one, that is, the retry number, it is determined in the processing device whether the retry number of the to-be-processed task is less than the retry threshold corresponding to the to-be-processed task, if the retry number of the to-be-processed task is less than the retry threshold corresponding to the to-be-processed task, the to-be-processed task is added to the abnormal task message queue, if the retry number of the to-be-processed task is equal to or greater than the retry threshold corresponding to the to-be-processed task, the to-be-processed task is added to the target file, and the target file is displayed to the administrator.

[0085] In some embodiments, after processing the to-be-processed task, the method further comprises:

[0086] If the to-be-processed task fails and the reason for the to-be-processed task processing failure is a business reason, the to-be-processed task is added to the target file, and the target file is displayed to the administrator.

[0087] The error code of the business reason starts with 2, and the business reason is, for example, password error, transaction account state exception, etc.

[0088] At the same time, the state of the to-be-processed task is recorded as a failure state, and the information is notified to the database 15, and the database 15 records the state of the to-be-processed task as a failure.

[0089] In some embodiments, after processing the to-be-processed task, the method further comprises:

[0090] If the to-be-processed task is successfully processed, the state of the to-be-processed task is recorded as a success state.

[0091] At the same time, the state of the to-be-processed task is recorded as a success state, and the information is notified to the database 15, and the database 15 records the state of the to-be-processed task as a success state.

[0092] In some embodiments, after recording the state of the to-be-processed task as a failure state, the method further comprises:

[0093] If the to-be-processed task is a part of the target task, the first subtask in the target task is suspended and the second subtask in the target task is continued to be processed, wherein the first subtask is a subtask in the target task having a dependency relationship with the to-be-processed task, and the second subtask is a subtask in the target task having no dependency relationship with the to-be-processed task.

[0094] After recording the state of the to-be-processed task as the success state, the method further comprises:

[0095] If the to-be-processed task is a part of the target task and the states of other subtasks of the target task are success states, the state of the target task is determined as a success state.

[0096] In detail, the case that the target task is divided into multiple subtasks can be recorded in a database, the database records task-related information of the multiple subtasks, the state of each subtask is obtained from the task-related information, when the state of the to-be-processed task is determined by the subscription device, the state of the to-be-processed task is fed back to the database, the database detects whether the to-be-processed task is a subtask of another task, if yes, the related case of each subtask of the target task is fed back to the subscription device, wherein the related case of each subtask of the target task includes the state of each subtask and the dependency relationship therebetween, after the subscription device receives the related case, it is judged whether the state of the to-be-processed task can change the state of the target task.

[0097] Specifically, if the state of the to-be-processed task is a success state, it is judged whether the states of other subtasks of the target task are all success states, if yes, the state of the target task is determined as a success state, and the information is fed back to the database, after the database receives the feedback information, the state of the target task is modified as a success state, if not all success, the state of the target task is determined as processing, and it is judged whether a first subtask in the target task having a dependency relationship with the to-be-processed task is suspended, if yes, it is informed that the first subtask can continue to be processed, and the above information is sent to the database, after the database receives the message, the related case of the multiple subtasks of the target task is updated, at the same time, the subscription device can inform the subscription device processing the first subtask to continue to process the first subtask.

[0098] If the state of the to-be-processed task is a failure state, the information is fed back to the database, after the database receives the feedback information, the state of the to-be-processed task of the target task is updated as a failure state, and the first subtask and the second subtask in the target task are found, the subscription device processing the first subtask is informed to suspend processing, and the suspension processing case of the first subtask is fed back to the database, after the database receives the feedback information, the feedback information is recorded.

[0099] Another cluster is provided by the embodiments of the present application, which includes a plurality of devices, for example, a publishing device for task publishing, a plurality of processing devices for processing abnormal tasks of different business types, and a plurality of subscription devices for processing tasks of different business types. For example, in combination with Figure 6 As shown in FIG. 1, the cluster includes a publishing device 10, a processing device 110, a processing device 111, a processing device 112, a subscription device 12, a subscription device 13, a subscription device 14, and a database 15. The subscription device 12 is used for processing tasks of a first business type, the subscription device 13 is used for processing tasks of a second business type, and the subscription device 14 is used for processing tasks of a third business type. The processing device 110 is used for processing abnormal tasks of the first business type, the processing device 111 is used for processing abnormal tasks of the second business type, and the processing device 112 is used for processing abnormal tasks of the third business type.

[0100] The specific processing procedures of the publishing device, the subscription device, the database, and the processing device are as shown above. When the processing device is working, it is in a manner as shown in Figure 3 Each processing device corresponds to processing abnormal tasks of one business type.

[0101] In detail, the subscription device receives a task notification. If the publishing topic corresponding to the task notification is a predetermined topic, the subscription device acquires a to-be-processed task from the task notification and processes the to-be-processed task. If the to-be-processed task fails to be processed and the reason for the to-be-processed task failing to be processed is a network problem, the subscription device records a state of the to-be-processed task as a failure state and, when the reason for the to-be-processed task failing to be processed is eliminated, sends the to-be-processed task in the failure state to the processing device 110 for processing abnormal tasks if the business type of the to-be-processed task is the first business type. If the business type of the to-be-processed task is the second business type, the to-be-processed task in the failure state is sent to the processing device 111 for processing abnormal tasks. If the business type of the to-be-processed task is the third business type, the to-be-processed task in the failure state is sent to the processing device 112 for processing abnormal tasks.

[0102] Since the processing device only processes abnormal tasks of a single business type, when abnormal tasks are added to a message queue, there is no need to classify them according to business types. When the processing device receives a to-be-processed task in a failure state, the processing device directly adds the to-be-processed task to an abnormal task message queue. The processing device returns the task to the original subscription device according to the order of tasks in the abnormal task message queue, and the original subscription device continues to process the task.

[0103] Based on the same inventive concept, the embodiments of the present application provide a task processing device, which is shown in FIG. 1 and includes Figure 7

[0104] ​The receiving module 710 is configured to receive a task notification; wherein the task notification is published by a publishing device when the publishing device receives a batch of tasks, and adds each task in the batch of tasks to a message queue of a corresponding type according to a task type of the task.

[0105] The obtaining module 720 is configured to, if the publishing topic corresponding to the task notification is a predetermined topic, obtain a to-be-processed task from the task notification, and process the to-be-processed task.

[0106] The exception processing module 730 is configured to, if the to-be-processed task fails to be processed and a reason for the to-be-processed task failing to be processed is a network reason, record a state of the to-be-processed task as a failure state, and when the reason for the to-be-processed task failing to be processed is eliminated, send the to-be-processed task to a processing device that processes an exception task, so that the processing device can process the to-be-processed task in the failure state.

[0107] Optionally, the exception processing module 730 is further configured to:

[0108] determine that a retry number of the to-be-processed task is less than a retry threshold corresponding to the to-be-processed task, wherein the retry threshold corresponding to the to-be-processed task is recorded in the task notification.

[0109] Optionally, the exception processing module 730 is further configured to:

[0110] if the to-be-processed task fails to be processed and a reason for the to-be-processed task failing to be processed is a business reason, add the to-be-processed task to a target file, and show the target file to a manager.

[0111] Optionally, the apparatus further includes:

[0112] The processing success module is configured to, if the to-be-processed task is processed successfully, record a state of the to-be-processed task as a success state.

[0113] Optionally, the exception processing module 730 is further configured to:

[0114] if the to-be-processed task is part of a target task, determine a state of the target task as a pause state, and pause processing of other subtasks in the target task; wherein the other subtasks in the target task are subtasks in the target task that have a dependency relationship with the to-be-processed task.

[0115] The processing success module is further configured to:

[0116] if the to-be-processed task is part of a target task, and a state of other subtasks of the target task is a success state, determine a state of the target task as a success state.

[0117] Optionally, the acquisition module 720 is specifically configured to:

[0118] acquire, from a database, a task identifier in the key-value pair corresponding to the to-be-processed task in the task notification belongs to a to-be-processed task; the database includes a plurality of tasks and a plurality of key-value pairs.

[0119] It should be noted that the division of each module of the above apparatus is only a logical functional division, and all or part of the modules can be integrated into one physical entity, or can be physically separated. And these modules can all be implemented in the form of software through a processing element; or all can be implemented in the form of hardware; or part of the modules can be implemented in the form of software through a processing element, and part of the modules can be implemented in the form of hardware. The processing element herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in the processing element or an instruction in the form of software.

[0120] Based on the same inventive concept, an electronic device is provided, Figure 8 An electronic device structure schematic diagram provided by some embodiments is shown, as Figure 8 As shown, the electronic device includes a processor 810, a memory 820, a communication interface 830, and a communication bus 840, wherein the processor 810, the communication interface 830, and the memory 820 complete mutual communication through the communication bus 840.

[0121] The memory 820 stores a computer program, and when the program is executed by the processor 810, the processor 810 executes the data analysis report construction method according to any one of the embodiments.

[0122] The above processor 810 can be a general-purpose processor, including a central processing unit CPU, a network processor NP, etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0123] The memory 820 is connected with the processor 810 through the system bus 840 and completes mutual communication, and the memory 820 is used for storing computer program instructions.

[0124] The system bus 840 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, or the like. The system bus 840 can be divided into an address bus, a data bus, a control bus, or the like. For ease of representation, only one thick line is used to represent the buses in the figure, but it does not mean that there is only one bus or only one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory can include random access memory (RAM), and can also include non-volatile memory.

[0125] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium, and a computer program product comprising computer program code which, when executed on a computer, causes the computer to perform the task processing method according to any one of the preceding embodiments.

[0126] Based on the same inventive concept, the embodiments of the present application also provide a computer program product, which comprises computer program code which, when executed on a computer, causes the computer to perform the task processing method according to any one of the preceding embodiments. Since the problem-solving principle of the above computer program product is similar to that of the data analysis report construction method, the implementation of the above computer program product can be referred to the implementation of the method, and the repeated parts will not be described again.

[0127] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions described in the flowcharts and / or block diagrams.Figure 1 apparatuses that implement the functions specified in the flowchart Figure 1 block or blocks.

[0129] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart Figure 1 apparatuses that implement the functions specified in the flowchart Figure 1 block or blocks.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart Figure 1 apparatuses that implement the functions specified in the flowchart Figure 1 block or blocks.

[0131] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the apparent, such modifications and variations are to be included within the scope of the application and the equivalent of such.

Claims

1. A task processing method characterized by, The method comprises the following steps: receiving a task notification; wherein the task notification is published by a publishing device after adding each task in a batch task to a message queue of a corresponding type according to the task type of the task when the batch task is received; if the publishing topic corresponding to the task notification is a predetermined topic, obtaining a to-be-processed task from the task notification, and processing the to-be-processed task; if the to-be-processed task fails to be processed and the reason for the failure of the to-be-processed task to be processed is a network reason, recording the state of the to-be-processed task as a failure state, and when the reason for the failure of the to-be-processed task to be processed is eliminated, sending the to-be-processed task in the failure state to a processing device for processing abnormal tasks, so that the processing device can process the to-be-processed task in the failure state.

2. The task processing method according to claim 1, characterized by, Before sending the to-be-processed task to the processing device for processing abnormal tasks, the method further comprises: determining that the number of retries of the to-be-processed task is less than the retry threshold corresponding to the to-be-processed task; wherein the retry threshold corresponding to the to-be-processed task is recorded in the task notification.

3. The task processing method of claim 1, wherein, After processing the to-be-processed task, the method further comprises: if the to-be-processed task fails to be processed and the reason for the failure of the to-be-processed task to be processed is a business reason, adding the to-be-processed task to a target file, and displaying the target file to a manager.

4. The task processing method of claim 1, wherein, After processing the to-be-processed task, the method further comprises: if the to-be-processed task is successfully processed, recording the state of the to-be-processed task as a success state.

5. The task processing method according to claim 4, characterized by, After recording the state of the to-be-processed task as a failure state, the method further comprises: if the to-be-processed task is part of a target task, suspending processing of a first subtask in the target task and continuing processing of a second subtask in the target task; wherein the first subtask is a subtask in the target task that has a dependency relationship with the to-be-processed task; and the second subtask is a subtask in the target task that has no dependency relationship with the to-be-processed task. After recording the state of the to-be-processed task as a success state, the method further comprises: if the to-be-processed task is part of a target task and the states of other subtasks of the target task are success states, determining that the state of the target task is a success state.

6. The task processing method according to any one of claims 1 to 5, characterized by, Obtaining a to-be-processed task from the task notification comprises: obtaining a to-be-processed task corresponding to a task identifier in a key-value pair from a database; wherein the database comprises a plurality of tasks and a plurality of key-value pairs.

7. A task processing apparatus characterized by comprising: The method comprises the following steps: a receiving module configured to receive a task notification; wherein the task notification is published by a publishing device after adding each task in a batch task to a message queue of a corresponding type according to the task type of the task when the batch task is received; an obtaining module configured to, if the publishing topic corresponding to the task notification is a predetermined topic, obtain a to-be-processed task from the task notification, and process the to-be-processed task; The abnormality processing module is configured to: if the to-be-processed task fails to be processed and a reason for the failure of the to-be-processed task to be processed is a network problem, record a state of the to-be-processed task as a failure state, and when the reason for the failure of the to-be-processed task to be processed is eliminated, send the to-be-processed task to a processing device that processes abnormal tasks, so that the processing device can process the to-be-processed task in the failure state.

8. The task processing apparatus according to claim 7, characterized by The abnormality processing module is further configured to: determine that a number of retries of the to-be-processed task is less than a retry threshold corresponding to the to-be-processed task, wherein the retry threshold corresponding to the to-be-processed task is recorded in the task notification.

9. The task processing apparatus according to claim 7, wherein The abnormality processing module is further configured to: if the to-be-processed task fails to be processed and a reason for the failure of the to-be-processed task to be processed is a business problem, add the to-be-processed task to a target file, and show the target file to a manager.

10. The task processing apparatus according to claim 7, characterized by The apparatus further includes: a processing success module configured to: if the to-be-processed task is successfully processed, record a state of the to-be-processed task as a success state.

11. The task processing apparatus according to claim 10, wherein The abnormality processing module is further configured to: if the to-be-processed task is part of a target task, determine a state of the target task as a pause state, and pause processing of other subtasks in the target task, wherein the other subtasks in the target task are subtasks in the target task that have a dependency relationship with the to-be-processed task. The processing success module is further configured to: if the to-be-processed task is part of a target task and a state of other subtasks of the target task is a success state, determine a state of the target task as a success state.

12. The task processing apparatus according to any one of claims 7 to 11, characterized by The obtaining module is specifically configured to: obtain, from a database, a to-be-processed task to which a task identifier in a key-value pair corresponding to a to-be-processed task in the task notification belongs, wherein the database includes a plurality of tasks and a plurality of key-value pairs.

13. An electronic device, comprising: including: a memory configured to store program instructions; a processor configured to invoke the program instructions stored in the memory and perform steps included in the method according to any one of claims 1-6 according to the obtained program instructions.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program includes program instructions, which, when executed by a computer, cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product, characterised in that, The computer program product includes computer program code, which, when executed on a computer, causes the computer to perform the method according to any one of claims 1-6.

Images