Task scheduling method, device and equipment and computer readable storage medium
By obtaining service process status information and determining the target service process in the distributed task scheduling framework, the task reallocation problem caused by network partition is solved to ensure the consistency and accuracy of task scheduling.
Patent Information
- Application Number
- CN202310638803.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
In the distributed task scheduling framework, network partitioning causes master node exceptions, and the system mistakenly believes that slave nodes are the master nodes, resulting in task redistribution, resulting in problem of task scheduling exceptions and inconsistent processing results.
The target server obtains the service process status information of each server, determines the target service process, obtains the current task list from the main service node and assigns it, ensuring that only one target service process performs task allocation and avoids duplicate allocation.
Avoid task scheduling exceptions, ensure that there is only one processing result for the same task, and avoid the problem of inconsistent processing results.
Smart Images

Figure CN119065799A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a task scheduling method, apparatus, device, and computer-readable storage medium. Background Art
[0002] In order to achieve high-concurrency task processing, the currently commonly used task scheduling framework is a distributed task scheduling framework.
[0003] Usually, the network corresponding to the distributed task scheduling framework includes multiple nodes and sub-networks managed by each node. Among the multiple nodes, there is a master node and the rest are slave nodes. When the sub-network is normal, due to some reasons, the network is disconnected between the multiple nodes, resulting in the entire network being divided into several independent partitions. Taking the division into two partitions as an example, one of the two partitions has a master node, and the other partition does not have a master node. At this time, the master node or slave node in any partition will think that the master node or slave node in the other partition is abnormal.
[0004] Since only the master node can assign tasks, at this time, the system will also promote the slave node of another partition to the master node, and the system will have two master nodes. The two master nodes will reallocate the tasks to be processed. Each master node will assign the tasks to be processed to its own partition for processing, so that each partition will process tasks that were not originally in its own partition, causing the same task to be processed in two partitions respectively, resulting in abnormal task scheduling, and multiple processing results for the same task, and the multiple processing results are inconsistent. Summary of the invention
[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a task scheduling method, device, equipment and computer-readable storage medium to solve the problem of abnormal task scheduling and avoid the phenomenon that the same task has multiple processing results and the multiple processing results are inconsistent.
[0006] In a first aspect, an embodiment of the present disclosure provides a task scheduling method, characterized in that the method is applicable to a task scheduling system, the task scheduling system includes multiple servers, each server includes multiple service processes, each server corresponds to a service node, and the service node corresponding to the target server among the multiple servers is the main service node;
[0007] The method comprises:
[0008] The target server obtains status information of each service process on each server;
[0009] The target server determines the target service process according to the status information of each service process on each server;
[0010] The target server obtains a current task list from the main service node through the target service process, and allocates tasks in the current task list through the target service process.
[0011] In a second aspect, an embodiment of the present disclosure provides a task scheduling device, characterized in that the device is applicable to a task scheduling system, the task scheduling system includes multiple servers, each server includes multiple service processes, each server corresponds to a service node, and the service node corresponding to the target server among the multiple servers is the main service node;
[0012] The device comprises:
[0013] An acquisition module, used for the target server to acquire status information of each service process on each server;
[0014] A determination module, used for the target server to determine the target service process according to the status information of each service process on each server;
[0015] The allocation module is used for the target server to obtain the current task list from the main service node through the target service process, and allocate the tasks in the current task list through the target service process.
[0016] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
[0017] Memory;
[0018] Processor; and
[0019] Computer programs;
[0020] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect.
[0021] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described in the first aspect.
[0022] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method described in the first aspect is implemented.
[0023] The task scheduling method, device, equipment and computer-readable storage medium provided by the embodiment of the present disclosure obtain the status information of each service process on each server through the target server, the target server determines the target service process according to the status information of each service process on each server, the target server obtains the current task list from the main service node through the target service process, and allocates the tasks in the current task list through the target service process. Compared with the prior art in which the master node allocates tasks, when a network partition occurs, there will be a scheme in which multiple master nodes allocate tasks at the same time. The service nodes of the present disclosure do not allocate tasks, each service node is used to store the task list, the data of each service node is consistent, the target service process is determined through the status information of each service process, and the task is allocated through the target service process, which can ensure that only one target service process can allocate or schedule tasks, and tasks will not be allocated repeatedly, thus avoiding task scheduling anomalies. The same task will only have one processing result, thereby avoiding task processing anomalies. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0026] Figure 1 A flowchart of a task scheduling method provided by an embodiment of the present disclosure;
[0027] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;
[0028] Figure 3 A flowchart of a task scheduling method provided by another embodiment of the present disclosure;
[0029] Figure 4 A flowchart of a task scheduling method provided by another embodiment of the present disclosure;
[0030] Figure 5 A schematic diagram of the structure of a task scheduling device provided in an embodiment of the present disclosure;
[0031] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0032] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0034] In order to achieve high-concurrency task processing, the currently commonly used task scheduling framework is a distributed task scheduling framework.
[0035] Usually, the network corresponding to the distributed task scheduling framework includes multiple nodes and sub-networks managed by each node. Among the multiple nodes, there is a master node and the rest are slave nodes. When the sub-network is normal, due to some reasons, the network is disconnected between the multiple nodes, resulting in the entire network being divided into several independent partitions. Taking the division into two partitions as an example, one of the two partitions has a master node, and the other partition does not have a master node. At this time, the master node or slave node in any partition will think that the master node or slave node in the other partition is abnormal.
[0036] Since only the master node can assign tasks, at this time, the system will also promote the slave node of another partition to the master node, and the system will have two master nodes. The two master nodes will reallocate the tasks to be processed, and each master node will assign the tasks to be processed to its own partition for processing, so that each partition will process tasks that are not originally in its own partition, resulting in the same task being processed in two partitions respectively, causing task scheduling anomalies, and the same task has multiple processing results, and the multiple processing results are inconsistent. In response to this problem, the embodiment of the present disclosure provides a task scheduling method, which is introduced below in conjunction with a specific embodiment.
[0037] Figure 1 The task scheduling method flow chart provided in the embodiment of the present disclosure. The method can be applied to Figure 2 The application scenario shown includes a target server 21, a server 22, and a server 23, each of which includes multiple service processes, each of which corresponds to a service node, and the service node corresponding to the target server among the multiple servers is the main service node. It can be understood that the task scheduling method provided in the embodiment of the present disclosure can also be applied in other scenarios.
[0038] Combine the following Figure 2 The application scenario shown is Figure 1The task scheduling method shown in the figure is introduced, and the specific steps of the method are as follows:
[0039] S101: The target server obtains status information of each service process on each server.
[0040] like Figure 2 As shown, the task scheduling system includes multiple servers, each server includes multiple service processes, each server corresponds to a service node, and the service node corresponding to the target server among the multiple servers is the main service node. The target server 21 obtains the status information of each service process on each server. Optionally, the status information of each service process may include the heartbeat message of each service process, the identification information of the server to which each service process belongs, and the identification information of each service process. The task scheduling system uses etcd as a service node, and a consistency protocol (raft protocol) is used between each service node to ensure the consistency of data stored in each service node. Optionally, the identification information of each service process is uniquely identified by the server address and the process number.
[0041] S102: The target server determines a target service process according to status information of each service process on each server.
[0042] The target server 21 determines the target service process according to the status information of each service process on each server, and specifically, can determine the target service process according to whether each service process is connected to the main service node and whether it carries the identification information of the main service node lock.
[0043] S103: The target server obtains a current task list from the main service node through the target service process, and allocates tasks in the current task list through the target service process.
[0044] The target server 21 obtains the current task list from the main service node through the target service process, and allocates the tasks in the current task list through the target service process. The target service process can schedule the tasks to ensure the accuracy of the tasks.
[0045] The disclosed embodiment obtains the status information of each service process on each server through the target server, and the target server determines the target service process according to the status information of each service process on each server. The target server obtains the current task list from the main service node through the target service process, and allocates the tasks in the current task list through the target service process. Compared with the prior art in which the master node allocates tasks, when a network partition occurs, there will be a scheme in which multiple master nodes allocate tasks at the same time. The service nodes disclosed in the present invention do not allocate tasks, and each service node is used to store the task list. The data of each service node is consistent. The target service process is determined through the status information of each service process, and the task allocation is performed through the target service process. It can be ensured that only one target service process can allocate or schedule tasks, and tasks will not be allocated repeatedly, thus avoiding task scheduling anomalies. The same task will only have one processing result, thereby avoiding task processing anomalies.
[0046] On the basis of the above embodiment, after the tasks in the current task list are allocated by the target service process, the method also includes: when a network partition occurs in the task scheduling system, the task scheduling system includes at least multiple network partitions, and the target server determines the target network partition among the multiple network partitions based on the number of service nodes included in each network partition in the multiple network partitions, and the target network partition includes the largest number of service nodes; for each network partition among the multiple network partitions except the target network partition, suspending the service process in the network partition.
[0047] When network partition occurs in the task scheduling system, the task scheduling system includes at least multiple network partitions. The target server 21 will determine the network partition with the largest number of service nodes as the target network partition based on the number of service nodes included in each of the multiple network partitions. For example, there are two network partitions, one network partition A has 2 service nodes, and the other network partition B has 3 service nodes. The target server 21 will determine the network partition with the largest number of service nodes as the target network partition. The target network partition can be a normal network partition, and the service process in the target network partition is processed normally. For each network partition among the multiple network partitions except the target network partition, the service process in the network partition is suspended.
[0048] In some embodiments, if the service node is abnormal and the network is normal, since each service process has its own task list locally, you can choose not to perform additional intervention and only restore the service node; you can also downgrade the service process of the server corresponding to the abnormal service node and not perform task processing, and the service process of the server corresponding to the main service node will take over.
[0049] In the embodiment of the present disclosure, the target server determines a target network partition among the multiple network partitions according to the number of service nodes included in each network partition among the multiple network partitions, and the target network partition includes the largest number of service nodes. For each network partition among the multiple network partitions except the target network partition, the service process in the network partition is suspended, thereby ensuring that the service process in the target network partition with the largest number of service nodes is processed normally, avoiding repeated assignment of tasks, avoiding task scheduling exceptions, and the same task will only have one processing result.
[0050] Figure 3 A flowchart of a task scheduling method provided by another embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the method includes the following steps:
[0051] S301: The target server obtains status information of each service process on each server.
[0052] Specifically, the implementation process and principle of S301 and S101 are the same, and will not be repeated here.
[0053] In some embodiments, the status information of each service process includes at least one of the following: a heartbeat message of each service process, identification information of a server to which each service process belongs, and identification information of a service node to which each service process is connected.
[0054] The heartbeat messages of each service process can represent the activity of each service process. If the feedback heartbeat message has not timed out, the service process is relatively active.
[0055] S302: The target server determines whether the service process is connected to the primary service node according to the status information of each service process.
[0056] After acquiring the status information of each service process on each server, the target server 21 determines whether the service process is connected to the primary service node according to the status information of each service process.
[0057] In some embodiments, S302 includes but is not limited to S3021 and S3022:
[0058] S3021. The target server determines whether the identification information of the service nodes to which the respective service processes are connected matches the identification information of the primary service node.
[0059] In this step, the target server determines whether the identification information of the service nodes to which the respective service processes are connected matches the identification information of the primary service node.
[0060] S3022: For each service thread, if the target server determines that the identification information of the service node to which the service thread is connected matches the identification information of the primary service node, it determines that the service process is connected to the primary service node.
[0061] For each service thread, if the target server determines that the identification information of the service node to which the service thread is connected matches the identification information of the main service node, it is determined that the service process is connected to the main service node; if they do not match, it is determined that the service process is not connected to the main service node.
[0062] S303: If the target server determines that the service process is connected to the primary service node, it determines whether the service process carries identification information of the primary service node lock.
[0063] Since there may be multiple service processes connected to the main service node, the present disclosure adopts the main service node lock, which can be a distributed mutex lock, that is, the service process connected to the main service node needs to apply for the main service node lock from the main service node. Only one service process can apply for the main service node lock in the same cycle, and the service process that applies for the main service node lock will carry the identification information of the main service node lock. In this step, the target server determines whether the service process carries the identification information of the main service node lock.
[0064] S304: If the target server determines that the service process carries identification information of the primary service node lock, it determines that the service process is a target service process.
[0065] If the target server determines that the service process carries the identification information of the main service node lock, it determines that the service process is the target service process; if the target server determines that the service process does not carry the identification information of the main service node lock, it determines that the service process is not the target service process.
[0066] S305: The target server obtains a current task list from the main service node through the target service process, and allocates tasks in the current task list through the target service process.
[0067] Specifically, the implementation process and principle of S305 and S103 are the same, and will not be repeated here.
[0068] The disclosed embodiment obtains the status information of each service process on each server through the target server, and the target server determines whether the service process is connected to the main service node based on the status information of each service process. Further, if the target server determines that the service process is connected to the main service node, it determines whether the service process carries the identification information of the main service node lock. If the target server determines that the service process carries the identification information of the main service node lock, it determines that the service process is the target service process. Then, the target server obtains the current task list from the main service node through the target service process, and allocates the tasks in the current task list through the target service process. Since the target server determines whether the service process is connected to the main service node based on the status information of each service process. Furthermore, if the target server determines that the service process is connected to the main service node, it determines whether the service process carries the identification information of the main service node lock. If the target server determines that the service process carries the identification information of the main service node lock, it determines that the service process is the target service process. By allocating tasks through the target service process, it can be ensured that only one target service process can allocate or schedule tasks, and tasks will not be allocated repeatedly, thus avoiding task scheduling anomalies. The same task will only have one processing result, thereby avoiding task processing anomalies.
[0069] Figure 4 A flowchart of a task scheduling method provided by another embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the method includes the following steps:
[0070] S401: The target server obtains status information of each service process on each server.
[0071] Specifically, the implementation process and principle of S401 and S101 are the same, and will not be repeated here.
[0072] S402: The target server determines a target service process according to status information of each service process on each server.
[0073] Specifically, the implementation process and principle of S402 and S102 are the same, and will not be repeated here.
[0074] S403: The target server obtains multiple current active service processes and a current task list from the main service node through the target service process, wherein the active service process is a service process that feeds back a heartbeat message to the target server without timeout among the service processes on each server.
[0075] The target server obtains multiple current active service processes and a current task list from the main service node through the target service process. Specifically, the target server obtains multiple current active service processes and a current task list from the main service node through a task scheduling thread in the target service process. In some embodiments, each service process will periodically feedback a heartbeat message to the target server, and the service process that feedbacks a heartbeat message without timeout among the service processes is an active service process.
[0076] S404. When the multiple current active service processes or the current task list changes, for each task in the current task list, the target server matches the identification information of the server of the task with the identification information of the server to which each active service process in the multiple current active service processes belongs through the target service process, and determines the active service process corresponding to the maximum matching degree as the active service process corresponding to the task.
[0077] If the multiple current active service processes or the current task list changes, the target server reallocates the current task list through the target service process. Specifically, the identification information of the server of the task is matched with the identification information of the server to which each active service process in the multiple current active service processes belongs, and the active service process corresponding to the maximum matching degree is determined as the active service process corresponding to the task. The essence of this step is to assign the tasks of each server to the service process in the server for easy processing.
[0078] In some embodiments, for some cross-server tasks, these tasks are assigned to the service process in the target server corresponding to the main service node for processing.
[0079] S405. The target server assigns the task to the active service process corresponding to the task through the target service process, obtains task assignment information, and synchronizes the task assignment information to the service node corresponding to each server respectively, wherein the task assignment information is used to characterize the correspondence between the task and the active service process.
[0080] The target server assigns the task to the active service process corresponding to the task through the target service process, obtains task assignment information, and synchronizes the task assignment information to the service node corresponding to each server, where the task assignment information is used to characterize the correspondence between the task and the active service process. The task assignment information is synchronized to the service node corresponding to each server to ensure that the task assignment information stored in each service node is consistent, thereby ensuring the accuracy of task assignment.
[0081] S406. The target server obtains the task allocation information from the service node to which the active service process is connected through the task update thread in each active service process, obtains the pending tasks of each server according to the task allocation information, and stores the pending tasks of each server on a local storage device.
[0082] Each active service process has a task update thread. The target server obtains the task allocation information from the service node to which the active service process is connected through the task update thread in each active service process. The task update thread in each active service process obtains the pending tasks of each server according to the task allocation information, and stores the pending tasks of each server on the local storage device, which can ensure that the tasks can be processed normally even if an abnormality occurs in the service node.
[0083] S407: The target server processes the pending tasks of each server through the task execution threads in each active service process.
[0084] Furthermore, the target server processes the pending tasks of each server through the task execution threads in each active service process.
[0085] In the disclosed embodiment, the target server obtains the status information of each service process on each server, and the target server determines the target service process according to the status information of each service process on each server. The target server obtains multiple current active service processes and the current task list from the main service node through the target service process. The active service process is a service process that has not timed out in the feedback of heartbeat messages to the target server among the service processes on each server. Further, when the multiple current active service processes or the current task list change, for each task in the current task list, the target server matches the identification information of the server of the task with the identification information of the server to which each active service process in the multiple current active service processes belongs through the target service process, and determines the active service process corresponding to the maximum matching degree as the active service process corresponding to the task. The target server assigns the task to the active service process corresponding to the task through the target service process, obtains the task allocation information, and synchronizes the task allocation information to the service node corresponding to each server, and the task allocation information is used to characterize the corresponding relationship between the task and the active service process. Then, the target server obtains the task allocation information from the service node to which the active service process is connected through the task update thread in each active service process, obtains the pending tasks of each server according to the task allocation information, and stores the pending tasks of each server on the local storage device, and the target server processes the pending tasks of each server through the task execution thread in each active service process. Since the present disclosure allocates tasks through the target service process, it can be ensured that only one target service process can allocate or schedule tasks, and tasks will not be allocated repeatedly, thus avoiding task scheduling anomalies. The same task will only have one processing result, thus avoiding task processing anomalies. Storing the pending tasks of each server on the local storage device can ensure that even if an abnormality occurs in the service node, the task can be processed normally.
[0086] Figure 5 A schematic diagram of the structure of a task scheduling device provided in an embodiment of the present disclosure. The task scheduling device may be the target server as described in the above embodiment, or the task scheduling device may be a component or assembly in the target server. The device is applicable to a task scheduling system, and the task scheduling system includes multiple servers, each server includes multiple service processes, and each server corresponds to a service node. The service node corresponding to the target server in the multiple servers is the main service node. The task scheduling device provided in an embodiment of the present disclosure can execute the processing flow provided in an embodiment of the task scheduling method, such as Figure 5As shown, the task scheduling device 50 includes: an acquisition module 51, a determination module 52, and an allocation module 53; wherein the acquisition module 51 is used by the target server to acquire the status information of each service process on each server; the determination module 52 is used by the target server to determine the target service process according to the status information of each service process on each server; the allocation module 53 is used by the target server to acquire the current task list from the main service node through the target service process, and allocate the tasks in the current task list through the target service process.
[0087] Optionally, the status information of each service process includes at least one of the following: a heartbeat message of each service process, identification information of a server to which each service process belongs, and identification information of a service node to which each service process is connected.
[0088] Optionally, the determination module 52 is used for the target server to determine the target service process based on the status information of each service process on each server, and is specifically used for: the target server to judge whether the service process is connected to the main service node based on the status information of each service process; if the target server judges that the service process is connected to the main service node, then judge whether the service process carries the identification information of the main service node lock; if the target server judges that the service process carries the identification information of the main service node lock, then determine that the service process is the target service process.
[0089] Optionally, the determination module 52 is used by the target server to determine whether the service process has established a connection with the main service node based on the status information of the respective service processes, and is specifically used for: the target server to respectively determine whether the identification information of the service node to which the respective service processes are connected matches the identification information of the main service node; for each service thread, if the target server determines that the identification information of the service node to which the service thread is connected matches the identification information of the main service node, it determines that the service process is connected to the main service node.
[0090] Optionally, the current task list includes at least one task and identification information of a server of each task;
[0091] The allocation module 53 is used for the target server to obtain the current task list from the main service node through the target service process, and to allocate the tasks in the current task list through the target service process. Specifically, it is used for: the target server to obtain multiple current active service processes and the current task list from the main service node through the target service process, and the active service process is a service process in each service process on each server that has not timed out in feeding back a heartbeat message to the target server; when the multiple current active service processes or the current task list change, for each task in the current task list, the target server matches the identification information of the server of the task with the identification information of the server to which each active service process in the multiple current active service processes belongs through the target service process, and determines the active service process corresponding to the maximum matching degree as the active service process corresponding to the task; the target server allocates the task to the active service process corresponding to the task through the target service process, obtains task allocation information, and synchronizes the task allocation information to the service node corresponding to each server, and the task allocation information is used to characterize the corresponding relationship between the task and the active service process.
[0092] Optionally, the target server assigns the task to the active service process corresponding to the task through the target service process. After obtaining the task assignment information, the device also includes: a task acquisition module 54 and a processing module 55; the task acquisition module 54 is used for the target server to obtain the task assignment information from the service node connected to the active service process through the task update thread in each active service process, obtain the pending tasks of each server according to the task assignment information, and store the pending tasks of each server on the local storage device; the processing module 55 is used for the target server to process the pending tasks of each server through the task execution thread in each active service process.
[0093] Optionally, after the tasks in the current task list are allocated by the target service process, the device also includes: a second determination module 56 and a pause module 57; the second determination module 56 is used when a network partition occurs in the task scheduling system, the task scheduling system includes at least multiple network partitions, and the target server determines the target network partition among the multiple network partitions according to the number of service nodes included in each network partition in the multiple network partitions, and the target network partition includes the largest number of service nodes; the pause module 57 is used to suspend the service process in each network partition among the multiple network partitions except the target network partition.
[0094] Figure 5The task scheduling device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effect are similar and will not be described in detail here.
[0095] Figure 6 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 6 , which shows a structural schematic diagram of an electronic device 600 suitable for implementing the embodiments of the present disclosure. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0096] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the task scheduling method of the embodiment described in the present disclosure. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0097] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0098] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the task scheduling method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0099] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0100] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0101] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0102] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:
[0103] The target server obtains status information of each service process on each server;
[0104] The target server determines the target service process according to the status information of each service process on each server;
[0105] The target server obtains a current task list from the main service node through the target service process, and allocates tasks in the current task list through the target service process.
[0106] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.
[0107] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0109] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.
[0110] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0111] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0112] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0113] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0114] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.
Claims
1. A task scheduling method, characterized in that: The method is applicable to a task scheduling system, which includes multiple servers, each server includes multiple service processes, each server corresponds to a service node, and the service node corresponding to the target server among the multiple servers is the main service node; The method comprises: The target server obtains status information of each service process on each server; The target server determines the target service process according to the status information of each service process on each server; The target server obtains a current task list from the main service node through the target service process, and allocates tasks in the current task list through the target service process.
2. The method according to claim 1, characterized in that The status information of each service process includes at least one of the following: The heartbeat message of each service process, the identification information of the server to which each service process belongs, and the identification information of the service node to which each service process is connected.
3. The method according to claim 2, characterized in that The target server determines the target service process according to the status information of each service process on each server, including: The target server determines whether the service process is connected to the main service node according to the status information of each service process; If the target server determines that the service process is connected to the primary service node, it determines whether the service process carries the identification information of the primary service node lock; If the target server determines that the service process carries the identification information of the primary service node lock, the service process is determined to be the target service process.
4. The method according to claim 3, characterized in that The target server determines whether the service process has established a connection with the primary service node according to the status information of each service process, including: The target server determines whether the identification information of the service nodes to which the respective service processes are connected matches the identification information of the primary service node; For each service thread, if the target server determines that the identification information of the service node to which the service thread is connected matches the identification information of the primary service node, the target server determines that the service process is connected to the primary service node.
5. The method according to claim 1, characterized in that: The current task list includes at least one task and identification information of a server of each task; The target server obtains the current task list from the main service node through the target service process, and allocates the tasks in the current task list through the target service process, including: The target server obtains multiple current active service processes and a current task list from the main service node through the target service process, wherein the active service process is a service process that feeds back a heartbeat message to the target server without timeout among the service processes on each server; When the multiple current active service processes or the current task list changes, for each task in the current task list, the target server matches the identification information of the server of the task with the identification information of the server to which each active service process in the multiple current active service processes belongs through the target service process, and determines the active service process corresponding to the maximum matching degree as the active service process corresponding to the task; The target server assigns the task to the active service process corresponding to the task through the target service process, obtains task assignment information, and synchronizes the task assignment information to the service node corresponding to each server respectively. The task assignment information is used to characterize the correspondence between the task and the active service process.
6. The method according to claim 5, characterized in that The target server assigns the task to the active service process corresponding to the task through the target service process, and after obtaining the task assignment information, the method further includes: The target server obtains the task allocation information from the service node to which the active service process is connected through the task update thread in each active service process, obtains the pending tasks of each server according to the task allocation information, and stores the pending tasks of each server on a local storage device; The target server processes the pending tasks of each server through the task execution threads in each active service process.
7. The method according to claim 1, characterized in that After allocating the tasks in the current task list through the target service process, the method further includes: When a network partition occurs in the task scheduling system, the task scheduling system includes at least a plurality of network partitions, and the target server determines a target network partition among the plurality of network partitions according to the number of service nodes included in each of the plurality of network partitions, and the target network partition includes the largest number of service nodes; For each network partition among the multiple network partitions except the target network partition, suspend the service process in the network partition.
8. A task scheduling device, characterized in that: The device is applicable to a task scheduling system, wherein the task scheduling system comprises a plurality of servers, each server comprises a plurality of service processes, each server corresponds to a service node, and the service node corresponding to the target server among the plurality of servers is the main service node; The device comprises: An acquisition module, used for the target server to acquire status information of each service process on each server; A determination module, used for the target server to determine the target service process according to the status information of each service process on each server; The allocation module is used for the target server to obtain the current task list from the main service node through the target service process, and allocate the tasks in the current task list through the target service process.
9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
