Distributed Task Scheduling Method, Device, Electronic Device and Storage Medium

Abstract

This application relates to the field of data processing technologies, and particularly to a distributed task scheduling method, apparatus, electronic device, and storage medium, which are applied to a distributed system. The distributed system includes multiple service nodes. The method includes: receiving a transaction request sent by a client and obtaining the task volume that each service node can handle; comparing the task volume to be processed corresponding to the transaction request with the task volumes that the multiple service nodes can handle; determining a corresponding load balancing strategy according to the comparison result, and distributing each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing strategy; wherein, the load balancing strategy includes a concurrency degree priority strategy and a message capacity priority strategy. In this way, batch transactions with high concurrency and large transaction volumes can be effectively split, the flexibility of the deployment strategy can be improved, and thus the processing efficiency can be improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a distributed task scheduling method, device, electronic device, and storage medium. Background Art

[0002] With the continuous optimization of enterprise online banking, the enterprise online banking business has flourished. While the business volume has reached new highs, the process of enterprise online banking transactions has become increasingly complex. More and more online banking transactions need to be completed through asynchronous processes, and the demand for daily batch and end-of-day batch is also increasing. Moreover, the vast majority of batches are periodic batches scheduled in minutes, which brings a lot of pressure to development and operation and maintenance. As a result, the enterprise online banking system has put forward higher requirements for the scheduling and processing efficiency of batch operations.

[0003] In the prior art, a distributed task scheduling system can be used to schedule tasks to be processed in a time-sharing and slicing manner, that is, to schedule tasks regularly and quantitatively based on the time dimension. For example, when a customer submits a batch transaction task, the system will add a new timed task. Further, the system uses a timed scheduling framework to send requests on time and periodically to activate job execution.

[0004] However, when the system receives a large number of tasks to be executed concurrently, the occupancy rate of the Central Processing Unit (CPU) is relatively high, resulting in low operating efficiency. When the number of tasks is so large that the system cannot bear it, it cannot meet the current performance requirements, and the flexibility of the deployment strategy is relatively low. Summary of the Invention

[0005] This application provides a distributed task scheduling method, device, electronic device, and storage medium, which can effectively split batch transactions with high concurrency and large transaction volumes, improve the flexibility of the deployment strategy, and thus improve the processing efficiency.

[0006] In a first aspect, this application provides a distributed task scheduling method, which is applied to a distributed system. The distributed system includes multiple service nodes. The method includes:

[0007] Receiving a transaction request sent by a client and obtaining the amount of tasks that each service node can process;

[0008] Comparing the amount of tasks to be processed corresponding to the transaction request with the amount of tasks that the multiple service nodes can process;

[0009] Determining a corresponding load balancing strategy according to the comparison result, and allocating each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing strategy;

[0010] Among them, the load balancing strategy includes the concurrency degree priority strategy and the message capacity priority strategy; the concurrency degree priority strategy is used to evenly divide the tasks to be processed based on the number of service nodes; the message capacity priority strategy is used to divide the amount of tasks to be processed based on the benchmark task amount and allocate them to service nodes.

[0011] Optionally, determining the corresponding load balancing strategy according to the comparison result includes:

[0012] If the amount of tasks that each service node can process is greater than the preset threshold, determine that the load balancing strategy is the concurrency degree priority strategy;

[0013] If the amount of tasks that any one service node can process is less than the preset threshold, determine that the load balancing strategy is the message capacity priority strategy.

[0014] Optionally, allocating each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing strategy includes:

[0015] If it is determined that the load balancing strategy is the concurrency degree priority strategy, evenly divide the tasks to be processed corresponding to the transaction request based on the total number of current service nodes, and allocate the evenly divided tasks to be processed to the corresponding service nodes for processing;

[0016] If it is determined that the load balancing strategy is the message capacity priority strategy, use the minimum value of the amount of tasks that the service node can process as the benchmark task amount, and divide the tasks to be processed corresponding to the transaction request based on the benchmark task amount, and allocate the divided tasks to be processed to the corresponding service nodes for processing.

[0017] Optionally, obtaining the amount of tasks that each service node can process includes:

[0018] Obtain the deployment location corresponding to each service node, determine the transmission path for sending tasks to the service node according to the deployment location, and determine the amount of tasks that the service node can process according to the transmission path.

[0019] Optionally, allocating each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing strategy includes:

[0020] Obtain the message queue corresponding to each service node based on the load balancing strategy, where the message queue includes tasks to be processed carrying the transaction type, and the transaction type is used to indicate the processing time and / or processing order of the tasks to be processed;

[0021] Allocate the message queue to corresponding service nodes for processing, so that the service nodes determine a thread pool corresponding to the to-be-processed task according to the transaction type, and the thread pool meets the requirements of the processing time and / or processing order of the to-be-processed task; and process the to-be-processed task according to the thread pool corresponding to the to-be-processed task.

[0022] Optionally, the method further includes:

[0023] When adding M service nodes, obtain the processable task volume corresponding to the M service nodes;

[0024] Send a recycling instruction to the multiple service nodes for recycling the remaining to-be-processed tasks of the transaction request;

[0025] Re-determine a load balancing strategy based on the remaining to-be-processed task volume corresponding to the transaction request and the processable task volume corresponding to the M service nodes, and allocate the remaining to-be-processed tasks to corresponding service nodes for processing based on the re-determined load balancing strategy.

[0026] Optionally, the method further includes:

[0027] At every preset period, check the logs generated when each service node processes each task corresponding to the transaction request;

[0028] Judge whether there is abnormal information in the logs;

[0029] If there is, remove the service node corresponding to the log, recycle the to-be-processed tasks in this service node, and allocate the to-be-processed tasks to service nodes other than those with abnormal information for processing.

[0030] In a second aspect, the present application further provides a distributed task scheduling device, which is applied to a distributed system. The distributed system includes multiple service nodes, and the device includes:

[0031] An obtaining module, configured to receive a transaction request sent by a client and obtain the processable task volume of each service node;

[0032] A comparison module, configured to compare the to-be-processed task volume corresponding to the transaction request with the processable task volumes of the multiple service nodes;

[0033] A processing module, configured to determine a corresponding load balancing strategy according to the comparison result, and allocate each task corresponding to the transaction request to corresponding service nodes for processing based on the load balancing strategy;

[0034] Among them, the load balancing strategy includes the concurrency degree priority strategy and the message capacity priority strategy; the concurrency degree priority strategy is used to evenly divide the tasks to be processed based on the number of service nodes; the message capacity priority strategy is used to divide the amount of tasks to be processed based on the benchmark task amount and allocate them to service nodes.

[0035] In a third aspect, the present application further provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0036] The memory stores computer-executable instructions;

[0037] The processor executes the computer-executable instructions stored in the memory to implement the method described in any one of the first aspects.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the distributed task scheduling method described in any one of the first aspects.

[0039] In summary, the present application provides a distributed task scheduling method, device, electronic device and storage medium, which can receive a transaction request sent by a client, and then obtain the amount of tasks that each service node can process; further, compare the amount of tasks to be processed corresponding to the transaction request with the amount of tasks that multiple service nodes can process; and determine the corresponding load balancing strategy according to the comparison result, and further allocate each task corresponding to the transaction request to the corresponding service node for processing; among them, the load balancing strategy includes the concurrency degree priority strategy and the message capacity priority strategy; the concurrency degree priority strategy is used to evenly divide the tasks to be processed based on the number of service nodes; the message capacity priority strategy is used to divide the amount of tasks to be processed based on the benchmark task amount and allocate them to service nodes. In this way, batch transactions with high concurrency and large transaction volume can be effectively split, the flexibility of the deployment strategy can be improved, and the processing efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0041] Figure 1 It is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0042] Figure 2 It is a schematic flowchart of a distributed task scheduling method provided by an embodiment of the present application;

[0043] Figure 3Schematic diagram of the architecture of a distributed task scheduling system provided by an embodiment of the present application;

[0044] Figure 4 Schematic diagram of the architecture of the deployment location of a service node provided by an embodiment of the present application;

[0045] Figure 5 Schematic diagram of the structure of a service node provided by an embodiment of the present application;

[0046] Figure 6 Deployment architecture diagram of a distributed task scheduling system provided by an embodiment of the present application;

[0047] Figure 7 Schematic diagram of the structure of a distributed task scheduling device provided by an embodiment of the present application;

[0048] Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application.

[0049] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0050] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with the present application. On the contrary, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0051] To facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. For example, the first device and the second device are only used to distinguish different devices, and do not limit their sequence. Those skilled in the art can understand that the terms "first" and "second" do not limit the quantity and execution order, and the terms "first" and "second" do not necessarily limit to be different.

[0052] It should be noted that in this application, words such as "exemplary" or "for example" are used to give examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0053] In this application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0054] To facilitate the understanding of the embodiments of this application, the following briefly explains the vocabulary involved in the embodiments of this application.

[0055] Distributed system: Refers to a software system built on a network, which has a variety of general physical and logical resources, can dynamically allocate tasks, and the distributed physical and logical resources exchange information through a computer network. There is a distributed operating system in the system that manages computer resources in a global manner, with high cohesion and transparency.

[0056] The following introduces the embodiments of this application with reference to the accompanying drawings. Figure 1 FIG. is a schematic diagram of an application scenario provided for the embodiments of this application. The distributed task scheduling method provided by this application can be applied to an application scenario as Figure 1 shown. This application scenario includes: a terminal device 101, a distributed system 102, a first node server 103, a second node server 104, and a third node server 105.

[0057] Specifically, the terminal device 101 can send a transaction request to the distributed system 102. Further, the distributed system 102 receives the transaction request and obtains the corresponding amount of tasks to be processed in the transaction request. Correspondingly, the distributed system 102 obtains the processable task amounts of the first node server 103, the second node server 104, and the third node server 105. Further, the distributed system 102 determines in which way to allocate each task corresponding to the transaction request to which node servers for processing based on the amount of tasks to be processed and the processable task amounts of each node server, so as to improve the processing efficiency and achieve load balancing.

[0058] It can be understood that the number of node servers is multiple. The number of the above node servers is only an example for illustration, and the embodiments of the present application do not make specific limitations thereto.

[0059] Among them, the terminal device can also be referred to as a terminal (Terminal), a user equipment (User Equipment, abbreviated as UE), a mobile station (Mobile Station, abbreviated as MS), a mobile terminal (Mobile Terminal, abbreviated as MT), etc. The terminal device can be a mobile phone (Mobile Phone), a smart TV, a wearable device, a smart speaker, a smart security device, a smart gateway, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (Virtual Reality, abbreviated as VR) terminal device, an augmented reality (Augmented Reality, abbreviated as AR) terminal device, a wireless terminal in industrial control (Industrial Control), a wireless terminal in self-driving (Self-Driving), a wireless terminal in remote medical surgery (Remote Medical Surgery), a wireless terminal in smart grid (Smart Grid), a wireless terminal in transportation safety (Transportation Safety), a wireless terminal in smart city (Smart City), a wireless terminal in smart home (Smart Home), and so on.

[0060] It should be noted that in this embodiment, the number and type of terminal devices are not specifically limited. Figure 1 The number of terminal devices shown is only for illustration.

[0061] In the prior art, a distributed task scheduling system can be used to schedule tasks to be processed in a time-sharing and time-slicing manner, that is, to schedule tasks regularly and quantitatively based on the time dimension. For example, when a customer submits a batch transaction task, the system will newly add a timing task. Further, the system uses a timing scheduling framework to send requests on time and periodically to activate the job to run.

[0062] However, when the above method is used and the system receives a large number of tasks for concurrent execution simultaneously, the CPU occupancy rate is relatively high, resulting in low operating efficiency. When the number of tasks is so large that the system cannot bear it, the current performance requirements cannot be met. Moreover, when new task jobs need to be added, new job logic needs to be added, and the overall compilation, packaging, and deployment need to be carried out again, and the deployment flexibility is not high, and the flexibility of the deployment strategy is relatively low.

[0063] Therefore, the present application provides a distributed task scheduling method, which is applied to a distributed system. The distributed system includes multiple service nodes, and can achieve scheduling among the service nodes of the distributed system, ensure the consistency among the service nodes of the distributed system while allocating resources, and can automatically sense the load balancing strategy according to the operating load of its own service node and the bearing capacity of the associated service nodes, and then select an appropriate load balancing strategy to send tasks to each service node for processing.

[0064] The technical solution of the present application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0065] Figure 2 It is a schematic flowchart of a distributed task scheduling method provided by an embodiment of the present application; as Figure 2 shown, the method of this embodiment can be applied to a distributed system. The distributed system includes multiple service nodes, and the method may include:

[0066] S201. Receive a transaction request sent by a client, and obtain the amount of tasks that each service node can process.

[0067] In this step, each service node is independent of each other. There is no key service node among the service nodes managed by the distributed system, and the service nodes do not communicate directly with each other, but rely on the database (cluster) and distributed cache to achieve the allocation of system resources, so as to achieve multi-channel job concurrency, load balancing, fault transfer, and system expansion, etc.

[0068] Exemplarily, in Figure 1 the application scenario, the distributed system 102 can receive a transaction request sent by the terminal device 101. Further, the distributed system 102 also needs to obtain the amount of tasks that the first node server 103, the second node server 104, and the third node server 105 can process.

[0069] S202. Compare the amount of tasks to be processed corresponding to the transaction request with the amount of tasks that the multiple service nodes can process.

[0070] In this step, by comparing the volume of tasks to be processed corresponding to the transaction request with the sum of the volumes of tasks that can be processed by multiple service nodes, and finding the minimum value of the volumes of tasks that can be processed by multiple service nodes, the volume of tasks that each service node needs to receive is determined, so as to make the allocation reasonable and achieve load balancing, thereby improving the processing efficiency.

[0071] Exemplarily, in Figure 1 the application scenario of, the distributed system 102 compares the volume of tasks to be processed corresponding to the transaction request received from the terminal device 101 with the sum of the volumes of tasks that can be processed by the first node server 103, the second node server 104, and the third node server 105, and finds the minimum value of the volumes of tasks that can be processed corresponding to the first node server 103, the second node server 104, and the third node server 105. For example, the volume of tasks to be processed is 30, the volume of tasks that can be processed corresponding to the first node server 103 is 20, the volume of tasks that can be processed corresponding to the second node server 104 is 15, and the volume of tasks that can be processed corresponding to the third node server 105 is 10. Since the volume of tasks to be processed is 30, which is less than the sum of the volumes of tasks that can be processed by each node server, which is 45, and the minimum value of the volumes of tasks that can be processed by each node server is 10, it can be determined that the volume of tasks that each node server needs to receive is 10.

[0072] S203. Determine the corresponding load balancing policy according to the comparison result, and allocate each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing policy.

[0073] Among them, the load balancing policy includes a concurrency degree priority policy and a message capacity priority policy; the concurrency degree priority policy is used to evenly divide the tasks to be processed based on the number of service nodes; the message capacity priority policy is used to divide the volume of tasks to be processed based on the reference task volume and allocate them to the service nodes.

[0074] In the embodiments of the present application, the number of service nodes may be the number of service nodes in the system that can receive the evenly divided volume of tasks to be processed, and this number of service nodes may also be set in advance manually; the reference task volume may refer to the volume of tasks that each service node in the system can receive and process, and this task volume may also be set in advance manually. The embodiments of the present application do not limit the specific values of the number of service nodes and the reference task volume.

[0075] Preferably, two load balancing strategies are designed in the embodiments of the present application, and each strategy can be dynamically adjusted according to the adaptive result. Among them, the concurrency degree priority strategy means that the received tasks are split with a fixed concurrency degree. For example, for 3000 tasks, if the set concurrency degree (the preset number of service nodes) is 3, then each message contains 1000 tasks, that is, each service node processes 1000 tasks, and the number of such messages is fixed. The message capacity priority strategy means that the number of tasks contained in each message is fixed. For example, each message (the task sent to the service node) can only contain 30 tasks. If there is a remainder when the total number of tasks is divided by the message capacity, the remaining tasks are put into a separate message and sent to the service node for processing.

[0076] Exemplarily, in Figure 1 the application scenario, taking the amount of tasks to be processed as 29, the amount of tasks that can be processed corresponding to the first node server 103 as 20, the amount of tasks that can be processed corresponding to the second node server 104 as 15, and the amount of tasks that can be processed corresponding to the third node server 105 as 10 as an example, the distributed system 102 compares the amount of tasks to be processed, which is 29, with the amount of tasks that can be processed corresponding to each node server and the sum of the amounts of tasks that can be processed corresponding to each node server, and determines that the amount of tasks to be processed, which is 29, is less than the sum of the amounts of tasks that can be processed by each node server, which is 45, and the minimum value of the amounts of tasks that can be processed by each node server is 10. Therefore, it can be determined to select the message capacity priority strategy, send 10 tasks to be processed to the first node server 103, send 10 tasks to be processed to the second node server 104, and send 9 tasks to be processed to the third node server 105. Correspondingly, each node server receives the tasks to be processed corresponding to it and processes them.

[0077] Therefore, the method provided by the embodiments of the present application can dynamically adjust the load balancing strategy according to the amount of tasks to be processed and the amount of tasks that can be processed by multiple service nodes, and can also effectively split batch transactions with high concurrency and large transaction volume, improving the processing efficiency and flexibility.

[0078] Exemplarily, Figure 3 is a schematic architecture diagram of a distributed task scheduling system provided by an embodiment of the present application. As Figure 3 shown, the application architecture of this distributed task scheduling system is divided into four layers from bottom to top, which are: the data layer, the service layer, the application layer, and the development layer.

[0079] Among them, the data layer refers to the data foundation on which the distributed task scheduling system framework runs, consisting of two parts, namely the Extensible Markup Language (XML) repository and the Relational Database (RDB). The XML repository is described in the form of XML metadata, including the definition of scheduler behavior, the definition of tasks, the scheduling policies of tasks, etc.; the data saved in the RDB includes: the persistent data of tasks, the scheduling information of tasks, the data for ensuring the operation of the distributed system, etc.

[0080] The service layer refers to the core mechanism for the operation of the distributed task scheduling system framework, including the Scheduler, Repository, Job Store, Thread Pool, Message Queue (MQ), Customer Information Control System (CICS), Remote Procedure Call (RPC), Plugin, communication basic components, etc.

[0081] The application layer refers to the service modules provided by the distributed task scheduling system framework, including the task definition module, scheduling request module, data operation module, message sending module, data synchronization module, data caching module, etc.

[0082] The development layer refers to the assembly packaging for developing applications using the distributed task scheduling system framework, including the Integrated Development Environment (IDE), plugin release, application release, etc.

[0083] It can be understood that the most core function of the distributed task scheduling system framework is to implement the scheduler and achieve scheduling among the service nodes of the distributed system. While allocating resources, it is also necessary to ensure the consistency among the service nodes of the distributed system. At the same time, it also provides an application programming interface and a solution for quickly releasing applications.

[0084] Optionally, determine the corresponding load balancing strategy according to the comparison result, including:

[0085] If the task volume that each service node can handle is greater than the preset threshold, determine the load balancing strategy as the concurrency degree priority strategy;

[0086] If the task volume that any one service node can handle is less than the preset threshold, determine the load balancing strategy as the message capacity priority strategy.

[0087] In the embodiments of the present application, the preset threshold may refer to the threshold set by the system for determining whether each service node has the ability to process a large number of tasks. The preset threshold can also be modified manually. The embodiments of the present application do not limit the specific value of the preset threshold. For example, the preset threshold is 1000.

[0088] Exemplarily, in Figure 1 the application scenario, taking the amount of tasks to be processed as 3000, the amount of tasks that can be processed by the first node server 103 as 1005, the amount of tasks that can be processed by the second node server 104 as 2000, and the amount of tasks that can be processed by the third node server 105 as 1020 as an example, the distributed system 102 determines through judgment that the amounts of tasks that can be processed by the first node server 103, the second node server 104, and the third node server 105 are all greater than the preset threshold 1000, then it can be determined that the selected load balancing strategy is the concurrency degree priority strategy.

[0089] Optionally, taking the amount of tasks to be processed as 70, the amount of tasks that can be processed by the first node server 103 as 40, the amount of tasks that can be processed by the second node server 104 as 1001, and the amount of tasks that can be processed by the third node server 105 as 30 as an example, the distributed system 102 determines through judgment that the amounts of tasks that can be processed by the first node server 103 and the third node server 105 are less than the preset threshold 1000, then it can be determined that the selected load balancing strategy is the message capacity priority strategy.

[0090] Therefore, the present application can select an appropriate load balancing strategy based on the amount of tasks that can be processed by each service node and the preset threshold to issue tasks, reduce the CPU occupancy rate of each service node, so that each task can be reasonably allocated and the task load is balanced.

[0091] Optionally, allocating each task corresponding to the transaction request to a corresponding service node for processing based on the load balancing strategy includes:

[0092] If it is determined that the load balancing strategy is the concurrency degree priority strategy, then the tasks to be processed corresponding to the transaction request are evenly divided based on the total number of current service nodes, and the evenly divided tasks to be processed are allocated to the corresponding service nodes for processing;

[0093] If it is determined that the load balancing strategy is the message capacity priority strategy, then the minimum value of the amounts of tasks that can be processed by the service nodes is used as the benchmark task amount, and the tasks to be processed corresponding to the transaction request are divided based on the benchmark task amount, and the divided tasks to be processed are allocated to the corresponding service nodes for processing.

[0094] Exemplarily, in Figure 1 In the application scenario, taking the number of tasks to be processed as 3,000, the number of tasks that can be processed corresponding to the first node server 103 is 1,005, the number of tasks that can be processed corresponding to the second node server 104 is 2,000, and the number of tasks that can be processed corresponding to the third node server 105 is 1,020 as an example, if the distributed system 102 determines that the selected load balancing strategy is the concurrency degree priority strategy, then the 3,000 tasks to be processed corresponding to the transaction request are evenly divided based on the total number of current service nodes, which is 3. Each node server should process 1,000 tasks. Further, the 1,000 tasks to be processed are respectively allocated to the first node server 103, the second node server 104, and the third node server 105 for processing.

[0095] In another embodiment, in Figure 1 In the application scenario, taking the number of tasks to be processed as 70, the number of tasks that can be processed corresponding to the first node server 103 is 40, the number of tasks that can be processed corresponding to the second node server 104 is 1,001, and the number of tasks that can be processed corresponding to the third node server 105 is 30 as an example, if the distributed system 102 determines that the selected load balancing strategy is the message capacity priority strategy, then taking the number of tasks that can be processed corresponding to the third node server 105, which is 30, as the reference task quantity, and dividing the tasks to be processed, which is 70, based on the reference task quantity. That is, the first node server 103 can process 30 tasks, the second node server 104 can process 30 tasks, and the third node server 105 can process 10 tasks. Further, the divided tasks to be processed are allocated to the corresponding node servers for processing.

[0096] It should be noted that when dividing the tasks to be processed into the corresponding service nodes, there is no order among each service node. The above division can also be that the first node server 103 processes 30 tasks, the second node server 104 processes 10 tasks, and the third node server 105 processes 30 tasks. That is, the number of tasks allocated to each service node is fixed. If there is a remainder when the total number of tasks / the number of service nodes, then the remaining tasks are separately allocated to any one service node for processing. The above number of service nodes is only an example, and actually there are multiple service nodes.

[0097] Therefore, the present application can select an appropriate load balancing strategy according to the running load and bearing capacity of the service nodes. When using the concurrency degree priority strategy, the processing rate can be improved, that is, it is not necessary to distribute the tasks to be processed to too many service nodes for processing, saving the distribution time.

[0098] Optionally, obtaining the number of tasks that each service node can process includes:

[0099] Obtain the deployment location corresponding to each service node, determine the transmission path for sending tasks to the service node according to the deployment location, and determine the amount of tasks that the service node can process according to the transmission path.

[0100] In this step, when deploying service nodes to corresponding applications, the status of each service node is equal, and it does not rely on a central service node for system-level decision-making. The application module containing specific business logic can be in the same program space as the service node or can be distributed elsewhere on the network.

[0101] It can be understood that the local service node is deployed to achieve second-level scheduling, and other service nodes at other deployment locations in the distributed system can achieve minute-level scheduling for task processing.

[0102] Exemplarily, Figure 4 is a schematic architecture diagram of the deployment location of a service node provided by an embodiment of the present application. As Figure 4 shown, there are multiple ways to carry each service node in the distributed system. It can rely on an executable program (Executable Program, abbreviated as EXE) (i.e., Node 1), or in a local service (i.e., Node 4), or can use a hosted Web application (i.e., Node 3) or a World Wide Web (abbreviated as Web) site (i.e., Node 2) for hosting. Among them, each node corresponds to its own server. Since each service node has different processing capabilities due to different available system resources, the distributed system can identify the processing capabilities of each service node and allocate tasks to each service node based on this, so as to achieve load balancing. Since there is no key service node, any service node can be smoothly processed when it is accessed or unloaded.

[0103] It should be noted that the entire distributed system is composed of a series of nodes, and information sharing can be achieved between nodes through messages.

[0104] Exemplarily, by obtaining the deployment location corresponding to each service node, further, determine the transmission path for sending tasks to the service node according to the deployment location. For example, the deployment location of Node 4 is on the local server, so it can be determined that the amount of tasks that Node 4 can process is more than that of other service nodes.

[0105] Therefore, the embodiment of the present application can determine the processing capacity of the service node according to the deployment location of the service node, that is, the amount of tasks that the service node can process, improve the accuracy of obtaining the amount of tasks that the service node can process, and make the processing task allocation more reasonable.

[0106] Optionally, allocating each task corresponding to the transaction request to a corresponding service node based on the load balancing policy includes:

[0107] Obtaining a message queue corresponding to each service node based on the load balancing policy, where the message queue includes to-be-processed tasks carrying transaction types, and the transaction types are used to indicate the processing time and / or processing order of the to-be-processed tasks;

[0108] Allocating the message queue to a corresponding service node for processing, so that the service node determines a thread pool corresponding to the to-be-processed task according to the transaction type, and the thread pool meets the requirements of the processing time and / or processing order of the to-be-processed task; and processing the to-be-processed task according to the thread pool corresponding to the to-be-processed task.

[0109] In the embodiments of the present application, the transaction type may include at least one of the following: real-time transaction, non-real-time transaction, and sequential transaction. Among them, a real-time transaction is a transaction with a processing time less than a preset time; a non-real-time transaction is a transaction without restrictions on the processing time; a sequential transaction is a transaction processed in sequence according to the received time and / or the received order.

[0110] For example, transaction 1 is a real-time transaction, and the preset time can be set according to actual needs. For example, if it is 10s, it is required that the processing time for completing transaction 1 is less than 10s; among them, the processing time may refer to the time it takes for transaction 1 to receive the feedback processing result; transaction 2 is a non-real-time transaction, because there is no requirement for the processing time of transaction 2, so it can be processed within a certain time; transactions 3 and 4 are sequential transactions, and it is required to process the above transactions in sequence according to the received time of transactions 3 and 4 and / or the received order.

[0111] It should be noted that the received time of transactions 3 and 4 and / or the received order may represent three relationships. For example, the received time and / or the received order may represent: the received time exists alone, the received time and the received order exist simultaneously, and the received order exists alone. When the processing time of the transaction arrives, the timer will notify the scheduler to schedule tasks for processing.

[0112] Exemplarily, Figure 5 is a schematic structural diagram of a service node provided by an embodiment of the present application; as Figure 5 shown, in the embodiments of the present application, the service node uses a thread pool as a hub to implement the processing and monitoring of tasks. The following introduces the node:

[0113] A task is used to describe a job, including the context of the task, an executable scheduler, and related parameters, etc.

[0114] A Scheduler refers to a container that implements scheduling functions. Thread pools, timers, tasks, etc. are all loaded in the container. Among them, a service node can configure one or more containers. Multiple tasks can be registered in one container, but one container can only have and must have one timer and one thread pool.

[0115] A Thread Pool refers to an indicator of the node's processing capacity. The size of the thread pool is defined according to the available resources of the server where the node is located.

[0116] A Timer is a thread timing device used to trigger scheduling. The timer counts time based on a time base point and will notify the scheduler for scheduling when it reaches the time to execute a task.

[0117] A Message Queue (MQ) refers to a local or network message queue interface that a service node can use. The implementation of the interface depends on different message queues. The framework comes with interface implementations for Microsoft Message Queuing (MSMQ for short) and Message Middleware (Tonglink / Q). Among them, the Tonglink / Q architecture consists of three major parts: server nodes, monitoring and management centers, and development interfaces.

[0118] A Remote Procedure Call (RPC) refers to an RPC interface resource identifier that a service node can use and follows the.NetRemoting standard.

[0119] A Monitor is used to monitor the running status of other service nodes and declare the running status of the current service node to the system.

[0120] An XML Processor refers to a set of operations on XML content.

[0121] A Plugin Manager refers to managing plugins within a service node.

[0122] A Cache refers to a distributed cache, which is managed and controlled by a server. Multiple client nodes store data, which can further improve the data reading rate.

[0123] An ADO Store for task storage support refers to database support for persisting tasks.

[0124] Specifically, in Figure 1In the application scenario, the distributed system 102 can obtain the message queues corresponding to the first node server 103, the second node server 104, and the third node server 105 based on the selected load balancing strategy. Each message queue includes a to-be-processed task carrying a transaction type. For example, taking the message queue corresponding to the first node server 103 as an example, this message queue contains that transactions 1-3 are real-time transactions, transactions 4-9 are non-real-time transactions, and transactions 10-20 are sequential transactions. Further, the distributed system 102 allocates this message queue to the first node server 103 for processing, so that the first node server 103 determines the thread pool corresponding to the to-be-processed task according to the above three transaction types, and each thread pool meets the requirements of the processing time and / or processing order of the to-be-processed tasks of transactions 1-20. Further, transactions 1-20 are processed according to the thread pool corresponding to the to-be-processed task.

[0125] Therefore, the embodiments of the present application can select thread pools corresponding to different types of transactions to process transactions, which can ensure that transactions of different transaction types are processed in a timely and efficient manner, improve the efficiency and flexibility of processing transactions, and thus improve system performance.

[0126] Optionally, the method further includes:

[0127] When M service nodes are newly added, obtain the processable task volume corresponding to the M service nodes;

[0128] Send a recycling instruction to the multiple service nodes for recycling the remaining to-be-processed tasks of the transaction request;

[0129] Based on the remaining to-be-processed task volume corresponding to the transaction request and the processable task volume corresponding to the M service nodes, re-determine the load balancing strategy, and based on the re-determined load balancing strategy, allocate the remaining to-be-processed tasks to the corresponding service nodes for processing.

[0130] Wherein, M is a positive integer greater than 1, and the embodiments of the present application do not limit the specific value of M.

[0131] In this step, when a new service node is accessed in the distributed system, it can be quickly identified and tasks can be assigned to it for execution, so as to achieve the horizontal expansion of the distributed system. After the new service node is accessed, it is necessary to recycle the remaining task volume (that is, the unprocessed task volume) in the to-be-processed tasks issued to the original service nodes, and re-allocate them to the new service node or the original service nodes. Or, the newly received task volume can also be directly assigned to the new service node for processing. The embodiments of the present application do not make specific limitations on this.

[0132] Exemplarily, in Figure 1In the application scenario, if the distributed system 102 identifies that 2 new node servers are added, it will obtain the amount of tasks that can be processed corresponding to the 2 node servers; further, it will send a recycling instruction to the first node server 103, the second node server 104, and the third node server 105 to recycle the tasks that have not been completed by the first node server 103, the second node server 104, and the third node server 105; further, based on the amount of tasks that have not been completed, the amount of tasks that can be processed corresponding to the 2 service nodes, and the amount of tasks that the first node server 103, the second node server 104, and the third node server 105 can currently process, it will re-determine the load balancing policy, and based on the re-determined load balancing policy, it will allocate the tasks that have not been completed to the corresponding service nodes for processing.

[0133] Therefore, in the embodiment of the present application, when a new service node is identified to be accessed, the load policy can be re-selected or new tasks can be sent to the new service node for processing, improving the flexibility of processing and the real-time performance of processing tasks.

[0134] Optionally, the method further includes:

[0135] Every preset period, check the logs generated when each service node processes each task corresponding to the transaction request;

[0136] Judge whether there is abnormal information in the log;

[0137] If there is, remove the service node corresponding to the log, recycle the tasks to be processed in the service node, and allocate the tasks to be processed to the service nodes other than those with abnormal information for processing.

[0138] In the embodiment of the present application, the preset period can refer to the time period set by the system for monitoring whether there is an abnormality in task processing. For example, the preset period can be one week or one day, and the embodiment of the present application does not make a specific limitation on this; the abnormal information can refer to the log information generated when the faulty service node fails and cannot process tasks or processes tasks abnormally, and the embodiment of the present application does not make a specific limitation on the specific content of the abnormal information.

[0139] Exemplarily, in Figure 1In the application scenario, every other week, the distributed system 102 checks the logs generated when the first node server 103, the second node server 104, and the third node server 105 process each task corresponding to the transaction request. Further, it determines whether there is abnormal information in the logs. For example, if there is abnormal information in the log corresponding to the third node server 105, the third node server 105 is removed, and the tasks to be processed in the third node server 105 are recycled. Further, the tasks to be processed are assigned to the first node server 103 and the second node server 104 for processing.

[0140] Therefore, the embodiment of the present application can monitor the system. When a service node in the system fails, it can be quickly identified, and then the tasks in the service node are recycled and the service node is removed from the system, thereby realizing fault transfer and improving system performance.

[0141] It should be noted that the embodiment of the present application can also manually intervene in the execution and suspension of tasks, and view the system logs, task logs, and exception logs generated by each service node, improving the flexibility of task processing.

[0142] Combined with the above embodiments, Figure 6 This is a deployment architecture diagram of a distributed task scheduling system provided by an embodiment of the present application. As Figure 6 shown, tasks can be sent to the distributed task scheduling system framework through a load balancing strategy. A framework can contain multiple distributed task scheduling systems. The distributed task scheduling system framework performs batch task processing according to its own processing mechanism, that is, it can be sent to different distributed task scheduling systems for business processing. Specifically, the applications in each client send transaction requests to the local distributed task scheduling system (F5). Further, the distributed task scheduling system can also associate with other distributed task scheduling systems for task processing. Among them, a communication infrastructure framework (Windows Communication Foundation, abbreviated as WCF) is deployed in each system for business processing. Further, the framework can select a corresponding load balancing strategy according to the running load of its own framework and the bearing capacity of the associated systems, and then send the tasks corresponding to the transaction requests to the message queue (i.e., TLQ) of each distributed task scheduling system based on the load balancing strategy, and remotely call or locally call the corresponding business logic for business processing. For example, call the transfer business logic for batch transfer, and then feedback the processing result to the background service.

[0143] In the foregoing embodiments, the distributed task scheduling method provided by the embodiments of the present application is introduced. To implement the various functions in the method provided by the embodiments of the present application, the electronic device as the execution subject may include a hardware structure and / or a software module, and implement the various functions in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module. Whether a certain function among the various functions is executed in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module depends on the specific application and design constraints of the technical solution.

[0144] For example, Figure 7 is a schematic structural diagram of a distributed task scheduling device provided by an embodiment of the present application. As Figure 7 shown, the device includes: an acquisition module 710, a comparison module 720, and a processing module 730; wherein, the acquisition module 710 is configured to receive a transaction request sent by a client and acquire the task volume that each service node can process;

[0145] The comparison module 720 is configured to compare the task volume to be processed corresponding to the transaction request with the task volume that the multiple service nodes can process;

[0146] The processing module 730 is configured to determine a corresponding load balancing policy according to the comparison result, and distribute each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing policy;

[0147] Wherein, the load balancing policy includes a concurrency degree priority policy and a message capacity priority policy; the concurrency degree priority policy is used to evenly divide the tasks to be processed based on the number of service nodes; the message capacity priority policy is used to divide the task volume to be processed based on a reference task volume and distribute it to service nodes.

[0148] Optionally, the processing module 730 includes a determination unit and a processing unit; the determination unit is configured to:

[0149] If the task volume that each service node can process is greater than a preset threshold, determine that the load balancing policy is the concurrency degree priority policy;

[0150] If the task volume that any one service node can process is less than the preset threshold, determine that the load balancing policy is the message capacity priority policy.

[0151] Optionally, the processing unit is configured to:

[0152] If it is determined that the load balancing policy is the concurrency degree priority policy, evenly divide the tasks to be processed corresponding to the transaction request based on the total number of current service nodes, and distribute the evenly divided tasks to the corresponding service nodes for processing;

[0153] If it is determined that the load balancing policy is the message capacity priority policy, the minimum value of the task volume that can be processed by the service node is used as the reference task volume, and the pending tasks corresponding to the transaction request are divided based on the reference task volume, and the divided pending tasks are assigned to the corresponding service node for processing.

[0154] Optionally, the obtaining module 710 is specifically configured to:

[0155] Obtain the deployment location corresponding to each service node, determine the transmission path for sending tasks to the service node according to the deployment location, and determine the task volume that the service node can process according to the transmission path.

[0156] Optionally, the processing module 730 is specifically configured to:

[0157] Obtain the message queue corresponding to each service node based on the load balancing policy, where the message queue includes pending tasks carrying transaction types, and the transaction types are used to indicate the processing time and / or processing order of the pending tasks;

[0158] Allocate the message queue to the corresponding service node for processing, so that the service node determines the thread pool corresponding to the pending task according to the transaction type, and the thread pool meets the requirements of the processing time and / or processing order of the pending task; and process the pending task according to the thread pool corresponding to the pending task.

[0159] Optionally, the device further includes an updating module, and the updating module is used for:

[0160] When M service nodes are newly added, obtain the task volume that can be processed by the M service nodes;

[0161] Send a recycling instruction to the multiple service nodes for recycling the remaining pending tasks of the transaction request;

[0162] Based on the remaining pending task volume corresponding to the transaction request and the task volume that can be processed by the M service nodes, re-determine the load balancing policy, and allocate the remaining pending tasks to the corresponding service node for processing based on the re-determined load balancing policy.

[0163] Optionally, the device further includes a monitoring module, and the monitoring module is used for:

[0164] Check the logs generated when each service node processes each task corresponding to the transaction request every preset period;

[0165] Judge whether there is abnormal information in the log;

[0166] If it exists, remove the service node corresponding to the log, recycle the tasks to be processed in the service node, and assign the tasks to be processed to service nodes other than those with exception information for processing.

[0167] For the specific implementation principle and effects of the distributed task scheduling device provided in the embodiments of the present application, reference can be made to the relevant descriptions and effects corresponding to the above embodiments, and details will not be elaborated here.

[0168] The embodiments of the present application also provide a schematic structural diagram of an electronic device. Figure 8 As a schematic structural diagram of an electronic device provided in the embodiments of the present application, as Figure 8 shown, the electronic device may include: a processor 802 and a memory 801 communicatively connected to the processor; the memory 801 stores a computer program; the processor 802 executes the computer program stored in the memory 801, so that the processor 802 executes the method described in any of the above embodiments.

[0169] Among them, the memory 801 and the processor 802 may be connected through a bus 803.

[0170] The embodiments of the present application also provide a computer-readable storage medium, which stores computer program execution instructions. When the computer execution instructions are executed by a processor, they are used to implement the method described in any of the foregoing embodiments of the present application.

[0171] The embodiments of the present application also provide a chip for running instructions. The chip is used to execute the method described in any of the foregoing embodiments executed by an electronic device in any of the foregoing embodiments of the present application.

[0172] The embodiments of the present application also provide a computer program product, which includes a computer program. When the computer program is executed by a processor, it can implement the method described in any of the foregoing embodiments executed by an electronic device in any of the foregoing embodiments of the present application.

[0173] In several embodiments provided in the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of devices or modules can be in electrical, mechanical or other forms.

[0174] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place or distributed over multiple network units. Some or all of the modules can be selected according to actual needs to implement the solution of this embodiment.

[0175] In addition, the functional modules in each embodiment of this application can be integrated in a processing unit, or each module can exist physically alone, or two or more modules can be integrated in one unit. The units formed by the above modules can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0176] The integrated modules implemented in the form of software functional modules can be stored in a computer-readable storage medium. The above software functional modules are stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in each embodiment of this application.

[0177] It should be understood that the above processor can be a Central Processing Unit (CPU for short), or other general-purpose processors, Digital Signal Processors (DSP for short), Application Specific Integrated Circuits (ASIC for short), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the application can be directly implemented by the execution of the hardware processor, or can be implemented by the combination of the hardware and software modules in the processor.

[0178] The memory may include a high-speed random access memory (Random Access memory, RAM for short), and may also include a non-volatile memory (Non-volatile Memory, NVM for short), such as at least one disk memory, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disc, etc.

[0179] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0180] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a Static Random-Access Memory (SRAM), an Electrically Erasable Programmable Read Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk. The storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0181] An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a master control device.

[0182] As described above, it is only the specific implementation manner of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any change or replacement within the technical scope disclosed in the embodiments of this application should be covered by the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be subject to the protection scope of the claims.

Claims

1. A distributed task scheduling method, characterized in that, Applied to a distributed system, the distributed system including multiple service nodes, the method includes: Receiving a transaction request sent by a client and obtaining the task volume that each service node can process; Comparing the task volume to be processed corresponding to the transaction request with the task volume that the multiple service nodes can process; Determining a corresponding load balancing strategy according to the comparison result, and allocating each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing strategy; Wherein, the load balancing strategy includes a concurrency degree priority strategy and a message capacity priority strategy; the concurrency degree priority strategy is used to evenly divide the tasks to be processed based on the number of service nodes; the message capacity priority strategy is used to divide the task volume to be processed based on a benchmark task volume and allocate it to service nodes; the concurrency degree priority strategy means splitting the received tasks at a fixed concurrency degree; the message capacity priority strategy means that the number of tasks included in each message is fixed. If there is a remainder when the total number of tasks is divided by the message capacity, the remaining tasks are put into a separate message and sent to the service node for processing; Allocating each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing strategy includes: If it is determined that the load balancing strategy is the concurrency degree priority strategy, evenly divide the tasks to be processed corresponding to the transaction request based on the total number of current service nodes, and allocate the evenly divided tasks to be processed to the corresponding service nodes for processing; If it is determined that the load balancing strategy is the message capacity priority strategy, use the minimum value of the task volume that the service node can process as the benchmark task volume, and divide the tasks to be processed corresponding to the transaction request based on the benchmark task volume, and allocate the divided tasks to be processed to the corresponding service nodes for processing.

2. The method according to claim 1, characterized in that, Determining a corresponding load balancing strategy according to the comparison result includes: If the task volume that each service node can process is greater than a preset threshold, determine that the load balancing strategy is the concurrency degree priority strategy; If the task volume that any one service node can process is less than the preset threshold, determine that the load balancing strategy is the message capacity priority strategy.

3. The method according to claim 1, characterized in that Obtaining the task volume that each service node can process includes: Obtaining the deployment location corresponding to each service node, determining the transmission path for sending tasks to the service node according to the deployment location, and determining the task volume that the service node can process according to the transmission path.

4. The method according to claim 1, wherein Allocating each task corresponding to the transaction request to the corresponding service node for processing based on the load balancing strategy includes: Obtaining the message queue corresponding to each service node based on the load balancing strategy, the message queue including the tasks to be processed carrying the transaction type, and the transaction type being used to indicate the processing time and / or processing order of the tasks to be processed; Allocate the message queue to the corresponding service node for processing, so that the service node determines the thread pool corresponding to the to-be-processed task according to the transaction type, and the thread pool meets the requirements of the processing time and / or processing order of the to-be-processed task; and process the to-be-processed task according to the thread pool corresponding to the to-be-processed task.

5. The method according to claim 1, wherein The method further includes: When adding M service nodes, obtain the amount of tasks that can be processed corresponding to the M service nodes; Send a recycling instruction to the multiple service nodes for recycling the remaining to-be-processed tasks of the transaction request; Based on the remaining to-be-processed task amount corresponding to the transaction request and the amount of tasks that can be processed corresponding to the M service nodes, re-determine the load balancing strategy, and based on the re-determined load balancing strategy, allocate the remaining to-be-processed tasks to the corresponding service nodes for processing.

6. The method according to any one of claims 1-5, characterized in that The method further includes: At every preset period, check the logs generated when each service node processes each task corresponding to the transaction request; Judge whether there is abnormal information in the logs; If there is, remove the service node corresponding to the log, recycle the to-be-processed tasks in this service node, and allocate the to-be-processed tasks to the service nodes other than those with abnormal information for processing.

7. A distributed task scheduling device, characterized in that, Applied to a distributed system, the distributed system includes multiple service nodes, and the device includes: An obtaining module, configured to receive a transaction request sent by a client and obtain the amount of tasks that can be processed by each service node; A comparison module, configured to compare the to-be-processed task amount corresponding to the transaction request with the amount of tasks that can be processed by the multiple service nodes; A processing module, configured to determine the corresponding load balancing strategy according to the comparison result, and based on the load balancing strategy, allocate each task corresponding to the transaction request to the corresponding service node for processing; Wherein, the load balancing strategy includes a concurrency degree priority strategy and a message capacity priority strategy; the concurrency degree priority strategy is used to evenly divide the to-be-processed tasks based on the number of service nodes; the message capacity priority strategy is used to divide the to-be-processed task amount based on the benchmark task amount and allocate it to the service nodes; the concurrency degree priority strategy means that the received tasks are split with a fixed concurrency degree; the message capacity priority strategy means that the number of tasks included in each message is fixed. If the total number of tasks divided by the message capacity has a remainder, the remaining tasks are put into a separate message and sent to the service node for processing; The processing module is specifically configured to, if it is determined that the load balancing strategy is the concurrency degree priority strategy, evenly divide the to-be-processed tasks corresponding to the transaction request based on the total number of current service nodes, and allocate the evenly divided to-be-processed tasks to the corresponding service nodes for processing; if it is determined that the load balancing strategy is the message capacity priority strategy, use the minimum value of the amount of tasks that can be processed by the service node as the benchmark task amount, and based on the benchmark task amount, divide the to-be-processed tasks corresponding to the transaction request, and allocate the divided to-be-processed tasks to the corresponding service nodes for processing.

8. An electronic device, characterized in that, Includes: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the distributed task scheduling method according to any one of claims 1-6.

Images