Method and system for multi-service task processing based on general dispatch service

Abstract

This invention provides a multi-service task processing method and system based on a general scheduling service, comprising: a first service party calling a task distribution method in a first task distribution service instance to submit first task data; the first service party encapsulating the first task data to obtain a first standard task message and sending the first standard task message into a first general task queue; a task scheduling service retrieving the first standard task message from the first general task queue; the first task distribution service instance calling a first task execution method in an internally referenced first task distribution notification to execute the first standard task message; and a second service party using a method similar to that used by the first service party; aiming to solve the problems of redundancy, resource waste, complex maintenance, and poor scalability in task processing systems under multi-service scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, and in particular to a method and system for processing multiple business tasks based on a general scheduling service. Background Technology

[0002] In today's internet and enterprise applications, systems typically need to handle a variety of asynchronous background tasks, such as sending emails, generating reports, processing images, and synchronizing data. Traditional solutions involve building a separate processing system for each type of task, including independent message queues (such as RabbitMQ or Kafka topics) and dedicated consumer services.

[0003] However, this traditional architecture has the following problems:

[0004] 1) System redundancy and high maintenance costs: Each new business type requires the development, deployment and maintenance of a completely new set of queue production and consumption code, resulting in a large number of code with duplicate functions but different business functions in the code library, and high maintenance and upgrade costs.

[0005] 2) Low resource utilization: Each independent consumer service requires a certain amount of computing resources (CPU, memory). Due to the uneven workload of different services, some services may experience resource shortages while others are idle and wasted, making it impossible to achieve resource sharing and flexible allocation.

[0006] 3) Poor system maintainability: Maintenance personnel need to monitor and manage a large number of scattered queues and services simultaneously, making fault localization difficult. When a problem occurs in the system, troubleshooting needs to be carried out across multiple systems, which is extremely complex.

[0007] 4) Insufficient scalability: The system has poor scalability and cannot achieve unified pooling and flexible scheduling of resources. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide a multi-service task processing method and system based on a general scheduling service, which aims to solve the problems of redundancy, resource waste, complex maintenance and poor scalability of task processing systems in multi-service scenarios.

[0009] In a first aspect, embodiments of the present invention provide a multi-service task processing method based on a general scheduling service, the method comprising:

[0010] The first business party calls the task distribution method in the first task distribution service instance and submits the first task data;

[0011] The first business party encapsulates the first task data to obtain a first standard task message and sends the first standard task message into a first general task queue.

[0012] The second business party calls the task distribution method in the second task distribution service instance and submits the second task data;

[0013] The second business party encapsulates the second task data to obtain the second standard task message, and sends the second standard task message into the second general task queue;

[0014] The task scheduling service retrieves the first standard task message from the first general task queue and the second standard task message from the second general task queue;

[0015] The first task distribution service instance calls the first task execution method in the internally referenced first task distribution notification to execute the first standard task message;

[0016] The second task distribution service instance calls the second task execution method in the internally referenced second task distribution notification to execute the second standard task message.

[0017] Furthermore, the first business party encapsulates the first task data to obtain a first standard task message, including:

[0018] The unified access layer receives task processing request parameters sent by the first service provider; the task processing request parameters include the first task data.

[0019] The unified access layer encapsulates the first task data according to a preset task message protocol to obtain the first standard task message.

[0020] The first service provider receives the first standard task message sent by the unified access layer.

[0021] Furthermore, the preset task message protocol includes: task ID, task name, task status, and remarks.

[0022] Furthermore, after sending the first standard task message into the first general task queue, the method further includes:

[0023] The first general task queue will queue the first standard task messages according to order or priority.

[0024] Furthermore, before the first business party calls the task distribution method in the first task distribution service instance, the method further includes:

[0025] When the application is initialized, the first business party creates or injects the first task distribution service instance and registers the first task distribution notification corresponding to the first business party with the first task distribution service instance.

[0026] Secondly, embodiments of the present invention provide a multi-service task processing system based on a general scheduling service, the system comprising:

[0027] The first calling module is used by the first business party to call the task distribution method in the first task distribution service instance and submit the first task data.

[0028] The first encapsulation module is used to encapsulate the first task data by the first business party to obtain a first standard task message, and send the first standard task message into a first general task queue.

[0029] The second calling module is used by the second business party to call the task distribution method in the second task distribution service instance and submit the second task data.

[0030] The second encapsulation module is used by the second business party to encapsulate the second task data to obtain the second standard task message, and then send the second standard task message into the second general task queue.

[0031] The acquisition module is used for the task scheduling service to acquire the first standard task message from the first general task queue and to acquire the second standard task message from the second general task queue;

[0032] The first execution module is used by the first task distribution service instance to call the first task execution method in the internally referenced first task distribution notification to execute the first standard task message;

[0033] The second execution module is used by the second task distribution service instance to call the second task execution method in the internally referenced second task distribution notification to execute the second standard task message.

[0034] Furthermore, the first encapsulation module is specifically used for:

[0035] The unified access layer receives task processing request parameters sent by the first service provider; the task processing request parameters include the first task data.

[0036] The unified access layer encapsulates the first task data according to a preset task message protocol to obtain the first standard task message.

[0037] The first service provider receives the first standard task message sent by the unified access layer.

[0038] Furthermore, the preset task message protocol includes: task ID, task name, task status, and remarks.

[0039] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described above.

[0040] Fourthly, embodiments of the present invention provide a computer-readable medium having processor-executable non-volatile program code that causes the processor to perform the method described above.

[0041] This invention provides a multi-service task processing method and system based on a general scheduling service, comprising: a first service party calling a task distribution method in a first task distribution service instance to submit first task data; the first service party encapsulating the first task data to obtain a first standard task message and sending the first standard task message into a first general task queue; a second service party calling a task distribution method in a second task distribution service instance to submit second task data; the second service party encapsulating the second task data to obtain a second standard task message and sending the second standard task message into a second general task queue; a task scheduling service retrieving the first standard task message from the first general task queue and the second standard task message from the second general task queue; the first task distribution service instance calling a first task execution method in an internally referenced first task distribution notification to execute the first standard task message; and the second task distribution service instance calling a second task execution method in an internally referenced second task distribution notification to execute the second standard task message. This aims to solve the problems of redundancy, resource waste, complex maintenance, and poor scalability in task processing systems under multi-service scenarios.

[0042] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0043] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0044] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a multi-service task processing method based on a general scheduling service provided in Embodiment 1 of the present invention;

[0046] Figure 2 This is a signaling diagram of the task processing flow provided in Embodiment 1 of the present invention;

[0047] Figure 3 This is a schematic diagram of the overall architecture provided in Embodiment 2 of the present invention;

[0048] Figure 4 This is a schematic diagram of the task message protocol provided in Embodiment 2 of the present invention;

[0049] Figure 5 This is a schematic diagram of a multi-service task processing system based on a general scheduling service provided in Embodiment 3 of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] To facilitate understanding of this embodiment, the embodiments of the present invention will be described in detail below.

[0052] Example 1:

[0053] Figure 1 The flowchart is provided for a multi-service task processing method based on a general scheduling service, as shown in Embodiment 1 of the present invention.

[0054] Reference Figure 1 The method includes the following steps:

[0055] Step S101: The first business party calls the task distribution method in the first task distribution service instance and submits the first task data;

[0056] Step S102: The first business party encapsulates the first task data to obtain the first standard task message, and sends the first standard task message into the first general task queue.

[0057] Step S103: The second business party calls the task distribution method in the second task distribution service instance and submits the second task data;

[0058] Step S104: The second business party encapsulates the second task data to obtain the second standard task message, and sends the second standard task message into the second general task queue.

[0059] Step S105: The task scheduling service obtains a first standard task message from the first general task queue and a second standard task message from the second general task queue.

[0060] Here, the task scheduling service serves as a core scheduling component, providing functions such as task enqueuing and scheduling execution.

[0061] Step S106: The first task distribution service instance calls the first task execution method in the internally referenced first task distribution notification to execute the first standard task message;

[0062] In step S107, the second task distribution service instance calls the second task execution method in the internally referenced second task distribution notification to execute the second standard task message.

[0063] Here, new business units only need to create a new Task Dispatch Service instance and implement the common methods in the taskDispatch Notify base class, without modifying the core scheduling framework, which greatly reduces code duplication and redundant development.

[0064] Furthermore, step S102 includes the following steps:

[0065] Step S201: The unified access layer receives the task processing request parameters sent by the first service provider; the task processing request parameters include the first task data.

[0066] Step S202: The unified access layer encapsulates the first task data according to the preset task message protocol to obtain the first standard task message.

[0067] Step S203: The first service provider receives the first standard task message sent by the unified access layer.

[0068] Furthermore, the preset task message protocol includes: task ID, task name, task status, and remarks.

[0069] Furthermore, after sending the first standard task message into the first general task queue, the method also includes the following steps:

[0070] Step S301: The first general task queue will queue the first standard task messages according to order or priority.

[0071] Furthermore, before the first business party calls the task distribution method in the first task distribution service instance, the method also includes the following steps:

[0072] Step S401: When the application is initialized, the first business party creates or injects a first task distribution service instance and registers the first task distribution notification corresponding to the first business party with the first task distribution service instance.

[0073] Reference Figure 2 During the task submission phase, business service A distributes the task to TDS instance A; TDS instance A is bound to general task queue A and task Dispatch Notify implementation A; the task enters the general task queue A (dedicated queue) of TDS instance A.

[0074] During the task execution phase, scheduler A retrieves a task from the general task queue A, and scheduler A calls the TDS instance A to process the task. TDS instance A calls the task execution method of its internally referenced task Dispatch Notify implementation A to execute the task and returns the execution result to TDS instance A.

[0075] In the independent operation description, the process of TDS instance B is completely independent. It uses a general task queue B, and TDS instance B calls the task execution method of B, which is referenced internally, to execute tasks, and adopts an independent scheduling cycle.

[0076] Compared with the prior art, this application has the following specific advantages:

[0077] 1) Complete decoupling and reduced maintenance costs: The traditional "centralized routing + type resolution" model is replaced by the "service instantiation + policy injection" model, and the generation and execution of tasks are completely separated through a unified protocol and task scheduling service.

[0078] New service integration only requires creating a new Task Dispatch Service instance and implementing the common methods in the task DispatchNotify base class, without modifying the core scheduling framework, greatly reducing code duplication and redundant development.

[0079] 2) It has extremely strong security and isolation: the task processing of each business line is isolated from each other. The processor failure of one business will not affect the central scheduler or the processor of other businesses, and errors will not spread through the central processor.

[0080] 3) Enhanced observability and maintainability: All tasks flow through a unified path, facilitating centralized monitoring, logging, and tracing, significantly reducing the complexity of system maintenance. Specifically, this is reflected in:

[0081] Unified monitoring points: By implementing unified monitoring points in key methods such as dispatch (task submission) and performTask (task execution) of the Task Dispatch Service, task metrics (such as throughput, time consumption, success rate, etc.) of all businesses can be collected.

[0082] Global Link Tracing: A globally unique trace identifier (trace_id) can be generated when a task is submitted. This identifier is continuously transmitted and recorded in the log throughout the entire task flow, enabling complete tracing of the entire link status of a task from submission, queuing to completion, and quickly locating which link the delay or failure occurred.

[0083] 4) Excellent scalability: The system can be easily scaled horizontally. For example, by increasing the number of instances of a specific taskDispatch Notify implementation class, execution requests can be distributed to multiple instances to alleviate processing bottlenecks caused by the complexity of a business logic; system-level scaling can be achieved by deploying new, independent Task Dispatch Service instances, dedicated queues, and corresponding task Dispatch Notify implementations for new business units, allowing new business lines to join without competing for any resources with existing businesses.

[0084] 5) Improve development efficiency: It provides developers with a standardized task access paradigm, enabling them to focus on the development of the business logic itself, rather than the construction of the underlying queue infrastructure.

[0085] Example 2

[0086] Figure 3 This is a schematic diagram of the overall system architecture provided in Embodiment 2 of the present invention.

[0087] Reference Figure 3 Business service A calls the dispatch method of its Task Dispatch Service instance A to submit task data. Business service A then encapsulates the task data and sends it to the general task queue A.

[0088] Business service B calls the dispatch method of its Task Dispatch Service instance B to submit task data. Business service B then encapsulates the task data and sends it to the general task queue B.

[0089] The scheduler executor A retrieves task messages from the general task queue A. The Task Dispatch Service instance A calls the performTask method of the task Dispatch Notify implementation A, which it internally references, to execute the task.

[0090] The scheduler B retrieves a task message from the general task queue B. The Task Dispatch Service instance B calls the performTask method of the task Dispatch Notify implementation B, which it internally references, to execute the task.

[0091] Since each Task Dispatch Service instance is bound to a specific business handler during initialization (e.g., instance A is bound to HandlerA, and instance B is bound to HandlerB), it can automatically execute the correct business logic without specifying the type or performing routing checks in the message body.

[0092] The system includes: a unified access layer, used to receive task processing requests from different business parties. This module defines and follows a preset task message protocol, encapsulating request parameters from different businesses into standardized task messages. The preset task message protocol includes at least, but is not limited to, the following fields, as detailed in [reference needed]. Figure 4 :

[0093] taskId: Task ID.

[0094] taskName: Task name.

[0095] taskStatus: Task status.

[0096] Remark: Notes / Notes.

[0097] General Task Queue: A central message queue used to receive and store all standard task messages generated by the unified access layer that conform to the protocol. Tasks from all services are queued in this queue according to order or priority, achieving unified task management and traffic peak shaving.

[0098] Task Dispatch Service: As a core scheduling component, it provides functions such as task enqueueing and scheduled execution. At its core, it does not contain any specific business logic, but holds references to the business processor interface.

[0099] Business Processor Interface: A general convention that defines the task execution method. This interface declares a unified task processing method (e.g., performTask(TaskCorn taskCorn)) and establishes a unified specification that all specific task execution components must follow.

[0100] Multiple business services: When the application is initialized, each business service creates or injects a dedicated TaskDispatchService instance and registers its own Task Dispatch Notify implementation with that instance.

[0101] Thus, although the code logic of each TaskDispatchService instance is completely generic, it is "specialized" into a task scheduler that handles a certain type of business through composition.

[0102] Example 3

[0103] Figure 5 This is a schematic diagram of a multi-service task processing system based on a general scheduling service provided in Embodiment 3 of the present invention.

[0104] Reference Figure 5 The system includes:

[0105] The first calling module is used by the first business party to call the task distribution method in the first task distribution service instance and submit the first task data.

[0106] The first encapsulation module is used by the first business party to encapsulate the first task data to obtain the first standard task message and send the first standard task message into the first general task queue.

[0107] The second calling module is used by the second business party to call the task distribution method in the second task distribution service instance and submit the second task data.

[0108] The second encapsulation module is used by the second business party to encapsulate the second task data, obtain the second standard task message, and send the second standard task message into the second general task queue.

[0109] The acquisition module is used for the task scheduling service to acquire a first standard task message from a first general task queue and to acquire a second standard task message from a second general task queue.

[0110] The first execution module is used by the first task distribution service instance to call the first task execution method in the internally referenced first task distribution notification to execute the first standard task message.

[0111] The second execution module is used by the second task distribution service instance to call the second task execution method in the internally referenced second task distribution notification to execute the second standard task message.

[0112] Furthermore, the first encapsulation module is specifically used for:

[0113] The unified access layer receives task processing request parameters sent by the first service provider; the task processing request parameters include the first task data.

[0114] The unified access layer encapsulates the first task data according to the preset task message protocol to obtain the first standard task message;

[0115] The first service provider receives the first standard task message sent by the unified access layer.

[0116] Furthermore, the preset task message protocol includes: task ID, task name, task status, and remarks.

[0117] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the multi-service task processing method based on a general scheduling service provided in the above embodiments.

[0118] This invention also provides a computer-readable medium having processor-executable non-volatile program code, on which a computer program is stored, and which, when run by a processor, executes the steps of the multi-service task processing method based on a general scheduling service described above.

[0119] The computer program product provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0121] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0122] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0124] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-service task processing method based on a general scheduling service, characterized in that, The method includes: The first business party calls the task distribution method in the first task distribution service instance and submits the first task data; The first business party encapsulates the first task data to obtain a first standard task message and sends the first standard task message into a first general task queue. The second business party calls the task distribution method in the second task distribution service instance and submits the second task data; The second business party encapsulates the second task data to obtain the second standard task message, and sends the second standard task message into the second general task queue; The task scheduling service retrieves the first standard task message from the first general task queue and the second standard task message from the second general task queue; The first task distribution service instance calls the first task execution method in the internally referenced first task distribution notification to execute the first standard task message; The second task distribution service instance calls the second task execution method in the internally referenced second task distribution notification to execute the second standard task message.

2. The multi-service task processing method based on general scheduling service according to claim 1, characterized in that, After encapsulating the first task data, the first business party obtains a first standard task message, including: The unified access layer receives task processing request parameters sent by the first service provider; the task processing request parameters include the first task data. The unified access layer encapsulates the first task data according to a preset task message protocol to obtain the first standard task message. The first service provider receives the first standard task message sent by the unified access layer.

3. The multi-service task processing method based on general scheduling service according to claim 2, characterized in that, The preset task message protocol includes: task ID, task name, task status, and remarks.

4. The multi-service task processing method based on general scheduling service according to claim 1, characterized in that, After sending the first standard task message into the first general task queue, the method further includes: The first general task queue will queue the first standard task messages according to order or priority.

5. The multi-service task processing method based on general scheduling service according to claim 1, characterized in that, Before the first business party calls the task distribution method in the first task distribution service instance, the method further includes: When the application is initialized, the first business party creates or injects the first task distribution service instance and registers the first task distribution notification corresponding to the first business party with the first task distribution service instance.

6. A multi-service task processing system based on a general scheduling service, characterized in that, The system includes: The first calling module is used by the first business party to call the task distribution method in the first task distribution service instance and submit the first task data. The first encapsulation module is used to encapsulate the first task data by the first business party to obtain a first standard task message, and send the first standard task message into a first general task queue. The second calling module is used by the second business party to call the task distribution method in the second task distribution service instance and submit the second task data. The second encapsulation module is used by the second business party to encapsulate the second task data to obtain the second standard task message, and then send the second standard task message into the second general task queue. The acquisition module is used for the task scheduling service to acquire the first standard task message from the first general task queue and to acquire the second standard task message from the second general task queue. The first execution module is used by the first task distribution service instance to call the first task execution method in the internally referenced first task distribution notification to execute the first standard task message; The second execution module is used by the second task distribution service instance to call the second task execution method in the internally referenced second task distribution notification to execute the second standard task message.

7. The multi-service task processing system based on a general scheduling service according to claim 6, characterized in that, The first encapsulation module is specifically used for: The unified access layer receives task processing request parameters sent by the first service provider; the task processing request parameters include the first task data. The unified access layer encapsulates the first task data according to a preset task message protocol to obtain the first standard task message. The first service provider receives the first standard task message sent by the unified access layer.

8. The multi-service task processing system based on a general scheduling service according to claim 7, characterized in that, The preset task message protocol includes: task ID, task name, task status, and remarks.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method described in any one of claims 1 to 5.

10. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the method described in any one of claims 1 to 5.

Images