Distributed scheduling method and apparatus, computer device, and storage medium

Abstract

The present disclosure provides a distributed scheduling method and device, computer equipment and storage medium. The method comprises: obtaining to-be-executed unit information, the to-be-executed unit information comprising a to-be-executed unit identifier; determining a target execution device corresponding to the to-be-executed unit identifier from a plurality of execution devices, wherein the plurality of execution devices are configured in a distributed architecture; and providing the to-be-executed unit information to the target execution device, which is used to schedule a to-be-executed unit to which the to-be-executed unit information belongs. The method can support scheduling of a large number of distributed timing or real-time tasks, has a simple structure and is easy to deploy, can realize horizontal expansion, and thus improves the effect of task scheduling.

Description

Technical Field

[0001] This disclosure relates to the field of Internet technology, and in particular to a distributed scheduling method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of internet technology, intelligent customer service open products have encountered many challenges in delivery and product updates. System design needs to meet the requirements of millions of distributed timers, a small number of timers, security, and domestic production. Under these circumstances, scheduling technology requirements include the following characteristics: resource consumption must be scalable (both more and less are acceptable); deployment needs to support horizontal scaling to support larger business volumes; operation and maintenance need to be simple, minimizing the introduction of third-party middleware, and decentralized deployment.

[0003] In related technologies, system design can be implemented using open-source scheduled task frameworks (Quartz), or open-source scheduling frameworks such as TBSchedule and Elasticjob. However, Quartz does not support millions (a large number) of scheduled tasks, leading to a sharp performance drop, and the system is prone to exceptions if the execution class is missing during upgrades. Elasticjob and TBSchedule, on the other hand, coordinate services through distributed applications such as Zookeeper, increasing system complexity and hindering private deployment and maintenance. Summary of the Invention

[0004] This disclosure proposes a distributed scheduling method, apparatus, computer device, and storage medium, aiming to at least partially solve one of the technical problems in the related art.

[0005] According to a first aspect, a distributed scheduling method is provided, comprising: obtaining information about units to be executed, the information about units to be executed including: an identifier of a unit to be executed; determining a target execution device corresponding to the identifier of a unit to be executed from a plurality of execution devices, wherein the plurality of execution devices are configured in a distributed architecture; and providing the information about units to be executed to the target execution device, the target execution device being used to schedule the units to be executed to which the information about units to be executed belongs.

[0006] According to a second aspect, a distributed scheduling device is provided, comprising: a first acquisition module for acquiring information about a unit to be executed, the information about the unit to be executed including: an identifier of the unit to be executed; a determination module for determining a target execution device corresponding to the identifier of the unit to be executed from a plurality of execution devices, wherein the plurality of execution devices are configured in a distributed architecture; and an execution module for providing the information about the unit to be executed to the target execution device, the target execution device being used to schedule the unit to be executed to which the information about the unit to be executed belongs.

[0007] According to a third aspect, a computer device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the distributed scheduling method provided in the embodiments of this disclosure.

[0008] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the distributed scheduling method provided in the embodiments of this disclosure is proposed.

[0009] In this embodiment, by acquiring information about the unit to be executed, including an identifier for the unit to be executed, and determining a target execution device corresponding to the identifier from a plurality of execution devices, wherein the plurality of execution devices are configured in a distributed architecture, and by providing the information about the unit to be executed to the target execution device, the target execution device is used to schedule the unit to be executed to which the information about the unit to be executed belongs. This method can support the scheduling of a large number of distributed timed or real-time tasks, and has a simple structure that is easy to deploy. It can also achieve horizontal scaling, thereby improving the task scheduling effect. This solves the technical problem in related technologies that scheduling systems do not support millions (a large number) of timed tasks and are not conducive to deployment.

[0010] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0011] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 This is a flowchart illustrating a distributed scheduling method according to an embodiment of the present disclosure;

[0013] Figure 2 This is a schematic diagram of the structure of a distributed scheduling system cluster provided according to an embodiment of this disclosure;

[0014] Figure 3 This is an operation flowchart of a task executor provided according to embodiments of this disclosure;

[0015] Figure 4 This is a flowchart illustrating a distributed scheduling method according to another embodiment of the present disclosure;

[0016] Figure 5 This is a flowchart illustrating a distributed scheduling method according to another embodiment of the present disclosure;

[0017] Figure 6 This is an operation flowchart of a task timer (Job) provided according to an embodiment of this disclosure;

[0018] Figure 7 This is a schematic diagram of a distributed scheduling device according to another embodiment of the present disclosure;

[0019] Figure 8 This is a schematic diagram of a distributed scheduling device according to another embodiment of the present disclosure; and

[0020] Figure 9 This is a block diagram of a computer device used to implement the distributed scheduling method of the embodiments of this disclosure. Detailed Implementation

[0021] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are used only to explain this disclosure, and should not be construed as limiting this disclosure. Rather, embodiments of this disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0022] In response to the technical problem that scheduling systems in related technologies do not support millions (a large number) of scheduled tasks and are not conducive to deployment, this embodiment provides a distributed scheduling method. The method will be described below with reference to specific embodiments.

[0023] It should be noted that the execution subject of the distributed scheduling method disclosed in this embodiment can be a distributed scheduling device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, which may include, but is not limited to, a terminal, a server, etc.

[0024] Figure 1 This is a flowchart illustrating a distributed scheduling method proposed in one embodiment of this disclosure.

[0025] like Figure 1 As shown, the distributed scheduling method includes:

[0026] S101: Obtain the information of the unit to be executed, which includes the identifier of the unit to be executed.

[0027] In this context, an execution unit can also be understood as a scheduled task that a computing device executes in real time or at regular intervals. For example, sending emails, paying for orders, sending SMS reminders, and any other possible tasks can be called execution units without any restrictions.

[0028] The execution unit that needs to be executed now can be called the unit to be executed. Furthermore, the unit to be executed can have corresponding information, which may include, for example, the task number (id), the parameter description of the task (solt form), the data structure (hashmap form), the attribute name (key), the object (value), and any other possible information, without any restrictions.

[0029] The information of the unit to be executed may also include the identifier of the unit to be executed. For example, the task number can be used as the identifier of the unit to be executed, and the unit to be executed can be identified by the identifier of the unit to be executed.

[0030] In practical applications, this distributed scheduling method can be executed using either a distributed or single-node scheduling system. Figure 2 This is a schematic diagram of the structure of a distributed scheduling system cluster provided according to an embodiment of this disclosure, such as... Figure 2 As shown, the scheduling system can include multiple nodes in a distributed architecture. These nodes can use a distributed or single remote dictionary server (Redis) for caching, and each node can execute the distributed scheduling method. Each node can include a node watcher, a task executor, and a task timer.

[0031] The task executor supports blocking queues and the blocking `pop` method, ensuring a first-in, first-out (FIFO) order (add task, retrieve task). The job timer supports a delayed queue sorted set (zset), ordered in reverse chronological order, and supports an atomic `remove` method (add job, retrieve job). Furthermore, this embodiment's scheduling system supports cache bucketing. When the data volume is too large, it distributes data according to the node ID, such as node1 linking to cache 1, node2 linking to cache 2, ensuring horizontal scalability.

[0032] Figure 3 This is an operation flowchart of the task executor (Task) provided according to embodiments of this disclosure, such as... Figure 3 As shown, the Task executor can first launch the Task manager, which may include a daemon process to obtain information about the units to be executed.

[0033] S102: Determine the target execution device corresponding to the unit identifier to be executed from among multiple execution devices, wherein the multiple execution devices are configured in a distributed architecture.

[0034] After obtaining the information of the unit to be executed as described above, this embodiment of the present disclosure further determines the target execution device corresponding to the identifier of the unit to be executed from a plurality of execution devices.

[0035] The device used to execute the scheduled task (i.e., the execution unit) can be called an execution device or executor. The Task manager of the Task executor can contain multiple execution devices. These multiple execution devices can be in the form of a distributed architecture, and multiple execution devices in multiple nodes can also be configured in the form of a distributed architecture.

[0036] The execution device corresponding to the identifier of the unit to be executed can be called the target execution device. That is to say, the target execution device can be determined from multiple execution devices based on the identifier of the unit to be executed, and the target execution device can be used to execute the unit to be executed.

[0037] S103: Provide the information of the unit to be executed to the target execution device, which is used to schedule the unit to be executed to which the information of the unit to be executed belongs.

[0038] After determining the target execution device as described above, this embodiment can further provide the information of the unit to be executed to the target execution device, which is used to schedule the unit to be executed to which the information of the unit to be executed belongs. That is to say, the target execution device is used to schedule the unit to be executed.

[0039] In this embodiment, by acquiring information about the unit to be executed, including an identifier for the unit to be executed, and determining a target execution device corresponding to the identifier from multiple execution devices (where the multiple execution devices are configured in a distributed architecture), and providing the information about the unit to be executed to the target execution device, the target execution device is used to schedule the unit to be executed. This method supports the scheduling of a large number of distributed timed or real-time tasks, has a simple structure that is easy to deploy, and can also achieve horizontal scaling, thereby improving the task scheduling effect. This solves the technical problem in related technologies where scheduling systems do not support millions (a large number) of timed tasks and are difficult to deploy.

[0040] Figure 4 This is a flowchart illustrating a distributed scheduling method proposed in another embodiment of this disclosure.

[0041] like Figure 4 As shown, the distributed scheduling method includes:

[0042] S401: Obtain multiple initial unit information corresponding to multiple execution units respectively.

[0043] In this embodiment of the disclosure, firstly, multiple initial unit information corresponding to multiple execution units is obtained.

[0044] Multiple execution units can be created by processes other than the process executing the unit to be executed, such as... Figure 3 As shown, other processes can create one or more execution units. Furthermore, each execution unit can have corresponding initial unit information, which may include, for example, a task number (id), a description of the execution unit's parameters (in solit format), a data structure (in hashmap format), attribute names (keys), objects (values), and any other possible information; there are no restrictions on this.

[0045] It should be noted that the process that creates the execution unit and the process that executes the unit to be executed can be different processes. Furthermore, the creation of the execution unit and the execution of the unit to be executed can be performed synchronously or asynchronously, without any restrictions.

[0046] S402: Serialize multiple initial unit information separately to obtain multiple execution unit information in serialized form.

[0047] Furthermore, in this embodiment of the present disclosure, multiple initial unit information are serialized to obtain multiple execution unit information in serialized form.

[0048] The initial unit information after serialization can be referred to as execution unit information. In some embodiments, for example, a lightweight data exchange format (JavaScript Object Notation, JSON) can be used for serialization, or other forms can be used for serialization, without limitation.

[0049] Furthermore, the information to be executed belongs to multiple execution unit information, and can be obtained from multiple execution unit information. That is to say, the obtained information to be executed is serialized information after serialization processing, which is beneficial to the transmission and persistence of information.

[0050] S403: Determine the number of target execution units, which are execution units in the execution state among multiple execution units.

[0051] Among the multiple execution units, the execution unit being executed can be referred to as the target execution unit. In this embodiment of the disclosure, the number of target execution units can be determined, that is, the number of execution units being executed can be determined.

[0052] S404: Determine the total number of devices among multiple actuators.

[0053] Furthermore, determine the total number of devices for multiple executors in the Task manager. This total number of devices can be set according to the actual application scenario and is not limited.

[0054] S405: When the number of units is less than the total number of devices, obtain the information of the unit to be executed.

[0055] Furthermore, the number of units is compared with the total number of devices. When the number of units is less than the total number of devices, the information of the unit to be executed is obtained. That is, if there is an idle device among the multiple execution devices, the information of the unit to be executed is obtained and executed. Thus, the embodiments of this disclosure can ensure that the unit to be executed can be executed smoothly without waiting, and can ensure the stable operation of the system.

[0056] In some embodiments, the Task manager can also be configured with a distributed cached blocking queue, and the multiple execution unit information includes corresponding execution unit identifiers, such as the task number of the execution unit. In this embodiment, after obtaining the serialized execution unit information, the multiple execution unit information can be further stored in the distributed cached blocking queue according to the multiple execution unit identifiers. Furthermore, in the operation of retrieving the information of the unit to be executed, the information can be read from the blocking queue in a loop. For example, the `brpop` method in Redis can be used to read the information of the unit to be executed from the blocking queue in a loop. This method can improve the efficiency of retrieving the information of the unit to be executed and ensure that the units to be executed are executed sequentially.

[0057] S406: Deserialize the serialized unit information to be executed to obtain the target unit information to be executed.

[0058] The information to be executed obtained above is in serialized form. Before executing the unit to be executed, the execution device can further deserialize the serialized information to be executed to obtain the target unit to be executed information.

[0059] S407: Determine the target execution device corresponding to the identifier of the unit to be executed. The target execution device belongs to multiple execution devices, which are configured in a distributed architecture.

[0060] For a detailed description of S407, please refer to the above embodiments, which will not be repeated here.

[0061] S408: Provide the target execution unit information to the target execution device.

[0062] During the process of providing the information of the unit to be executed to the target execution device, the deserialized target unit to be executed information can be provided to the target execution device. Thus, serialization facilitates the transmission and persistence of information, while deserialization restores the information to the form of an object, which can then be executed by the target execution device.

[0063] In this embodiment, by acquiring information about the unit to be executed, including an identifier for the unit to be executed, and determining a target execution device corresponding to the identifier from multiple execution devices (where the multiple execution devices are configured in a distributed architecture), and providing the information about the unit to be executed to the target execution device, the target execution device schedules the unit to be executed to which the information belongs. This supports the scheduling of a large number of distributed timed or real-time tasks, has a simple structure that is easy to deploy, and can be horizontally scaled, thereby improving the task scheduling effect. This solves the technical problem in related technologies where scheduling systems do not support millions (a large number) of timed tasks and are difficult to deploy. Furthermore, this embodiment ensures that the unit to be executed can be executed smoothly without waiting and ensures stable system operation. In addition, serialization facilitates information transmission and persistence, while deserialization restores the information to the form of an object, which can then be executed by the target execution device.

[0064] Figure 5 This is a flowchart illustrating a distributed scheduling method proposed in another embodiment of this disclosure.

[0065] like Figure 5 As shown, the distributed scheduling method includes:

[0066] S501: Obtain multiple target tasks, each of which has corresponding target task information.

[0067] In this embodiment of the disclosure, multiple target tasks are first obtained, and each target task has corresponding target task information.

[0068] The target task can be represented by a job, which is used to instruct the distributed scheduling system to execute the task at a scheduled time. For example, it can be used to schedule tasks such as sending emails, making payment orders, and sending SMS reminders as described in the above embodiments. Each execution unit can have a corresponding target task.

[0069] The target task information includes, for example, a globally unique execution ID (which can be generated based on the business number or use a universally unique identifier UUID as the execution ID), job parameters (in slot form), data structure (in hashmap form), key (id), and value (serialized object). The serialization can be in JSON form (caching the mapping relationship between id and object).

[0070] Figure 6 This is an operation flowchart of a task timer (Job) provided according to an embodiment of this disclosure, such as... Figure 6 As shown, in practical applications, the task timer first starts the job manager, which can include a daemon. The daemon can retrieve multiple target tasks, which can be in the form of a sorted set (zset). Further, during task execution, multiple nodes simultaneously attempt to remove tasks from the zset; only one node will succeed. Success returns 1, failure returns 0. When a node is 1, it indicates that the target task (job) has been retrieved by that node's daemon.

[0071] In some embodiments, in the operation of acquiring multiple target tasks, a reference time can be determined first.

[0072] The reference time is used to indicate the execution time of multiple target tasks. For example, the reference time can be from the current time to 10 seconds later; that is, multiple target tasks between the current time and the next 10 seconds can be obtained. In some embodiments, after each task is completed, the task can sleep for a predetermined time, such as 50 milliseconds.

[0073] Furthermore, multiple linked list positions are determined based on the reference time. In this embodiment of the disclosure, the linked list can be a circular linked list of a distributed cache (wheel list), which can store multiple tasks, and the linked list position is used to represent the position information of each task within the circular linked list.

[0074] In practical applications, this circular linked list can be divided into 24*60*60 data blocks based on the number of seconds in a day, with each data block storing a set of jobs (zset). In operations that determine the positions of multiple linked lists based on a reference time, the position of a data block can be used as the linked list position. For example, the position of the block from the current time to the block position 10 seconds in the future can be used as the linked list position. The block position can be represented as: current time (hours) * 60 * 60 + current time (minutes) * 60 + current time (seconds) + 1, or it can be determined in any other possible way without restriction.

[0075] Furthermore, based on multiple linked list positions, multiple candidate tasks are read from the circular linked list of the distributed cache, that is, multiple tasks (jobs) at multiple linked list positions in the circular linked list are determined as multiple candidate tasks.

[0076] Furthermore, it is determined whether the execution time of the candidate task meets the set conditions. These conditions may include, for example, tasks that need to be executed on the same day, tasks within a predetermined time (e.g., within 5 minutes), and any other possible conditions, without restriction. If the execution time of a candidate task meets the set conditions, then the candidate task is selected as the target task. That is, the task that meets the set conditions is selected from multiple candidate tasks as the target task. Therefore, this embodiment can quickly retrieve candidate tasks from a circular linked list and determine the target task from multiple candidate tasks by comparing them with the set conditions. Thus, the target task can meet the time requirements, which is beneficial for the reasonable execution of the task.

[0077] In some embodiments, before determining the reference time, embodiments of this disclosure may also obtain multiple candidate tasks, each of which has a corresponding multiple execution times.

[0078] like Figure 6 As shown, one or more scheduled tasks (jobs) created by other processes can be called candidate tasks, and candidate tasks can have corresponding execution times. It should be noted that the process that creates the candidate task and the process that obtains the target task can be different processes, and the creation of the candidate task and the acquisition of the target task can be performed synchronously or asynchronously, without restriction.

[0079] Furthermore, based on multiple execution times, multiple corresponding linked list positions are generated, namely: linked list positions in a circular linked list. These linked list positions can correspond to data blocks in the circular linked list, and the data blocks can be in list form to store the candidate tasks.

[0080] Furthermore, by referencing multiple linked list positions, multiple candidate tasks are written into a circular linked list. These candidate tasks (e.g., job1, job2, job3...) can be written into the data block of the circular linked list as a zest set. This enables asynchronous job write operations.

[0081] S502: Store multiple target tasks in a distributed cache delay queue based on multiple execution times.

[0082] After obtaining multiple target tasks, the target tasks are further stored in a delayed queue of a distributed cache based on multiple execution times.

[0083] In practical applications, multiple target tasks can be deserialized tasks. In this embodiment, multiple deserialized target tasks can be stored in a delay queue in memory, so that multiple target tasks can be executed sequentially according to the order of the delay queue.

[0084] S503: Construct multiple execution units corresponding to multiple target tasks, and configure the corresponding execution unit information according to the information of multiple target tasks.

[0085] Furthermore, multiple execution units (tasks) are constructed according to the multiple target tasks (jobs), and the corresponding execution unit information is configured according to the information of the multiple target tasks.

[0086] In some implementations, it can be determined whether the current time has reached the target execution time of the target task (job). If the current time has reached the target execution time, the target task to which the target execution time belongs is read from the delay queue. In practical applications, each target task in the delay queue can have a corresponding target execution time, which indicates the start time of the target task's execution. When the target execution time is reached, the corresponding target tasks are traversed from the delay queue, and the traversed target tasks can be referred to as the belonging target task.

[0087] Furthermore, corresponding execution units are constructed based on the target tasks, until all target tasks in the delay queue have been traversed. The execution time of a target task can be expressed as: execution time of each job - current time. Therefore, the task timer implemented in this disclosure can store target tasks in a circular list and traverse the target task jobs according to the circular list to construct the tasks to be executed, thus supporting a massive number of distributed scheduled tasks.

[0088] S504: Obtain multiple initial unit information corresponding to multiple execution units respectively.

[0089] S505: Serialize multiple initial unit information separately to obtain multiple execution unit information in serialized form, wherein the unit information to be executed belongs to multiple execution unit information.

[0090] S506: Determine the number of target execution units, which are execution units in the execution state among multiple execution units.

[0091] S507: Determine the total number of devices among multiple actuators.

[0092] S508: When the number of units is less than the total number of devices, obtain the information of the unit to be executed.

[0093] S509: Deserialize the serialized unit information to be executed to obtain the target unit information to be executed.

[0094] S510: Determine the target execution device corresponding to the identifier of the unit to be executed. The target execution device belongs to multiple execution devices, which are configured in a distributed architecture.

[0095] S511: Provide the target execution unit information to the target execution device.

[0096] For detailed descriptions of S504-S511, please refer to the above embodiments, which will not be repeated here.

[0097] In this embodiment, by acquiring the information of the unit to be executed, including the identifier of the unit to be executed, and determining the target execution device corresponding to the identifier of the unit to be executed from multiple execution devices, wherein the multiple execution devices are configured in a distributed architecture, and providing the information of the unit to be executed to the target execution device, the target execution device is used to schedule the unit to be executed to which the information of the unit to be executed belongs. This can support the scheduling of a large number of distributed timed or real-time tasks, and the structure is simple and easy to deploy. It can also achieve horizontal scaling, thereby improving the task scheduling effect. This solves the technical problem in related technologies that the scheduling system does not support millions (a large number) of timed tasks and is not conducive to deployment. In addition, the task timer implemented in this disclosure can store target tasks through a circular list, and traverse the target task job according to the circular list to construct the task to be executed, thus supporting a massive number of distributed timed tasks.

[0098] Figure 7 This is a schematic diagram of a distributed scheduling device according to another embodiment of the present disclosure.

[0099] like Figure 7 As shown, the distributed scheduling device 70 includes:

[0100] The first acquisition module 701 acquires information about the unit to be executed, which includes: the identifier of the unit to be executed;

[0101] The determining module 702 is configured to determine, from a plurality of execution devices, a target execution device corresponding to an identifier of a unit to be executed, wherein the plurality of execution devices are configured in a distributed architecture; and

[0102] The execution module 703 is used to provide the information of the unit to be executed to the target execution device, and the target execution device is used to schedule the unit to be executed to which the information of the unit to be executed belongs.

[0103] Optionally, Figure 8 This is a schematic diagram of a distributed scheduling device according to another embodiment of the present disclosure, such as... Figure 8 As shown, in some embodiments, the information of the unit to be executed is in serialized form, and the apparatus 70 further includes:

[0104] The deserialization module 704 is used to deserialize the serialized information of the unit to be executed to obtain the target information of the unit to be executed.

[0105] Specifically, the execution module 703 is used to provide the target execution unit information to the target execution device.

[0106] Alternatively, in some embodiments, such as Figure 8 As shown, the first acquisition module 701 includes:

[0107] The first determining submodule 7011 is used to determine the number of units of the target execution unit, where the target execution unit is the execution unit in the execution state among multiple execution units;

[0108] The second determining submodule 7012 is used to determine the total number of devices among the multiple execution devices;

[0109] The first acquisition submodule 7013 is used to acquire information about the unit to be executed when the number of units is less than the total number of devices.

[0110] Alternatively, in some embodiments, such as Figure 8 As shown, the device 70 further includes: a second acquisition module 705, used to acquire multiple initial unit information corresponding to multiple execution units respectively; and a serialization module 706, used to serialize the multiple initial unit information to obtain multiple execution unit information in serialized form. The first acquisition module 701 is specifically used to: acquire the unit information to be executed from the multiple execution unit information.

[0111] Optionally, in some embodiments, the multiple execution unit information includes corresponding multiple execution unit identifiers, such as... Figure 8 As shown, the device 70 further includes: a first storage module 707, used to store multiple execution unit information into a blocking queue of a distributed cache according to multiple execution unit identifiers;

[0112] The first acquisition module 701 is specifically used to: read the information of the unit to be executed from the blocking queue in a loop.

[0113] Alternatively, in some embodiments, such as Figure 8 As shown, the device 70 further includes: a third acquisition module 708, used to acquire multiple target tasks, each of which has corresponding target task information; and a configuration module 709, used to construct corresponding multiple execution units according to the multiple target tasks, and configure the corresponding multiple execution unit information according to the multiple target task information.

[0114] Optionally, in some embodiments, the third acquisition module 708 is specifically used for: determining a reference time; determining multiple linked list positions based on the reference time; reading multiple candidate tasks from the circular linked list of the distributed cache according to the multiple linked list positions; and if the execution time of a candidate task meets a set condition, then using the candidate task as the target task.

[0115] Optionally, in some embodiments, the device 70 further includes:

[0116] The second storage module 710 is used to store multiple target tasks into a delayed queue of a distributed cache based on multiple execution times;

[0117] The configuration module 709 is specifically used for: if the current time reaches the target execution time, reading the target task to which the target execution time belongs from the delay queue; constructing the corresponding execution unit according to the target task, until all target tasks in the delay queue have been traversed.

[0118] Optionally, in some embodiments, the third acquisition module 708 is specifically used to: acquire multiple candidate tasks, each candidate task having a corresponding multiple execution time; generate multiple linked list positions according to the multiple execution times; and write the multiple candidate tasks into a circular linked list with reference to the multiple linked list positions.

[0119] In this embodiment, by acquiring information about the unit to be executed, including an identifier for the unit to be executed, and determining a target execution device corresponding to the identifier from multiple execution devices (where the multiple execution devices are configured in a distributed architecture), and providing the information about the unit to be executed to the target execution device, the target execution device is used to schedule the unit to be executed. This method supports the scheduling of a large number of distributed timed or real-time tasks, has a simple structure that is easy to deploy, and can also achieve horizontal scaling, thereby improving the task scheduling effect. This solves the technical problem in related technologies where scheduling systems do not support millions (a large number) of timed tasks and are difficult to deploy.

[0120] To implement the above embodiments, this disclosure also proposes a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the distributed scheduling method proposed in the foregoing embodiments of this disclosure.

[0121] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the distributed scheduling method proposed in the foregoing embodiments of this disclosure.

[0122] To implement the above embodiments, this disclosure also proposes a computer program product that, when the instruction processor in the computer program product is executed, performs the distributed scheduling method proposed in the foregoing embodiments of this disclosure.

[0123] Figure 9 A block diagram of an exemplary computer device suitable for implementing embodiments of the present disclosure is shown. Figure 9 The computer device 12 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0124] like Figure 9 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0125] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0126] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0127] Memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 9 Not shown; usually referred to as a "hard drive".

[0128] although Figure 9 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0129] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0130] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with computer device 12, and / or with any device that enables computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0131] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the distributed scheduling method mentioned in the foregoing embodiments.

[0132] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the applications disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0133] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0134] It should be noted that in the description of this disclosure, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0135] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0136] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0137] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0138] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0139] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0140] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0141] Although embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A distributed scheduling method, characterized in that, include: Obtain information about the unit to be executed, including: the identifier of the unit to be executed; From a plurality of execution devices, a target execution device corresponding to the identifier of the unit to be executed is determined, wherein the plurality of execution devices are configured in a distributed architecture; and The information of the unit to be executed is provided to the target execution device, which is used to schedule the unit to be executed to which the information of the unit to be executed belongs; Before obtaining the information of the unit to be executed, the method further includes: Determine a reference time; based on the reference time, determine multiple linked list positions, wherein the linked list includes a circular linked list of the distributed cache; based on the multiple linked list positions, read the corresponding multiple candidate tasks from the circular linked list of the distributed cache, wherein the circular linked list is divided into 24 sub-lists according to the number of seconds in a day. 60 There are 60 data blocks, each storing a set of candidate tasks. In the operation of determining the positions of multiple linked lists based on the reference time, the position of the data block is determined as the position of the linked list. If the execution time of the candidate task meets the set conditions, the candidate task is taken as the target task, and the multiple target tasks each have corresponding multiple target task information; Multiple execution units are constructed according to the multiple target tasks, and the information of the multiple execution units is configured according to the information of the multiple target tasks. Before obtaining the information of the unit to be executed, the method further includes: obtaining multiple initial unit information corresponding to the multiple execution units respectively; performing serialization processing on the multiple initial unit information to obtain multiple execution unit information in serialized form; the multiple execution unit information each includes a corresponding multiple execution unit identifier, and the method further includes: storing the multiple execution unit information into a blocking queue of a distributed cache according to the multiple execution unit identifiers; wherein, obtaining the information of the unit to be executed includes: cyclically reading the information of the unit to be executed from the blocking queue; Before constructing the corresponding multiple execution units according to the multiple target tasks, the method further includes: storing the multiple target tasks in the delay queue of the distributed cache according to the multiple execution times; wherein, constructing the corresponding multiple execution units according to the multiple target tasks includes: if the current time reaches the target execution time, reading the target task to which the target execution time belongs from the delay queue; constructing the corresponding execution unit according to the target task, until all target tasks in the delay queue have been traversed.

2. The method as described in claim 1, characterized in that, The information of the unit to be executed is in serialized form. After obtaining the information of the unit to be executed, the following steps are also included: The serialized information of the unit to be executed is deserialized to obtain the target information of the unit to be executed. The step of providing the information of the unit to be executed to the target execution device includes: The target execution unit information is provided to the target execution device.

3. The method as described in claim 1, characterized in that, The process of obtaining the information of the unit to be executed includes: The number of target execution units is determined, wherein the target execution unit is an execution unit in the execution state among multiple execution units; Determine the total number of the plurality of actuators; When the number of units is less than the total number of devices, the information of the unit to be executed is obtained.

4. The method as described in claim 1, characterized in that, Before determining the reference time, the following is also included: Multiple candidate tasks are obtained, and each of the multiple candidate tasks has a corresponding multiple execution time. Based on the multiple execution times, generate corresponding linked list positions; Referring to the positions of the multiple linked lists, the multiple candidate tasks are written into the circular linked list.

5. A distributed scheduling device, characterized in that, The device includes: The first acquisition module acquires information about the unit to be executed, the information including: the identifier of the unit to be executed; A determining module is configured to determine, from a plurality of execution devices, a target execution device corresponding to the identifier of the unit to be executed, wherein the plurality of execution devices are configured in a distributed architecture; and An execution module is used to provide the information of the unit to be executed to the target execution device, and the target execution device is used to schedule the unit to be executed to which the information of the unit to be executed belongs; The device further includes: The third acquisition module is used to acquire multiple target tasks, each of which has corresponding target task information. The configuration module is used to construct corresponding execution units according to the multiple target tasks, and to configure the corresponding execution unit information according to the multiple target task information. The third acquisition module is specifically used for: Determine the reference time; Based on the reference time, multiple linked list positions are determined, wherein the linked lists include circular linked lists of distributed caches; Based on the positions of the multiple linked lists, multiple candidate tasks are read from the circular linked list of the distributed cache, wherein the circular linked list is divided into 24 sub-lists according to the number of seconds in a day. 60 There are 60 data blocks, each storing a set of candidate tasks. In the operation of determining the positions of multiple linked lists based on the reference time, the position of the data block is determined as the position of the linked list. If the execution time of the candidate task meets the set conditions, then the candidate task will be used as the target task. The device further includes: The second acquisition module is used to acquire multiple initial unit information corresponding to the multiple execution units respectively; The serialization module is used to serialize the multiple initial unit information respectively to obtain multiple execution unit information in serialized form; The plurality of execution unit information includes corresponding plurality of execution unit identifiers, and the device further includes: a first storage module, used to store the plurality of execution unit information into a blocking queue of a distributed cache according to the plurality of execution unit identifiers; wherein, the first acquisition module is specifically used to: cyclically read the information of the unit to be executed from the blocking queue; The device further includes: a second storage module, configured to store the plurality of target tasks into the delay queue of the distributed cache according to the plurality of execution times; wherein, the configuration module is specifically configured to: if the current time reaches the target execution time, read the target task to which the target execution time belongs from the delay queue; construct a corresponding execution unit according to the target task to which it belongs, until all target tasks in the delay queue have been traversed.

6. The apparatus as claimed in claim 5, characterized in that, The information of the unit to be executed is in serialized form, and the device further includes: The deserialization module is used to deserialize the serialized information of the unit to be executed to obtain the target unit to be executed information; Specifically, the execution module is used to provide the target execution unit information to the target execution device.

7. The apparatus as claimed in claim 5, characterized in that, The first acquisition module includes: The first determining submodule is used to determine the number of target execution units, wherein the target execution unit is an execution unit in the execution state among multiple execution units; The second determining submodule is used to determine the total number of devices among the plurality of execution devices; The first acquisition submodule is used to acquire the information of the unit to be executed when the number of units is less than the total number of devices.

8. The apparatus as claimed in claim 5, characterized in that, The third acquisition module is specifically used for: Multiple candidate tasks are obtained, and each of the multiple candidate tasks has a corresponding multiple execution time. Based on the multiple execution times, generate corresponding linked list positions; Referring to the positions of the multiple linked lists, the multiple candidate tasks are written into the circular linked list.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method as described in any one of claims 1-4.

10. A storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method as described in any one of claims 1-4.

Images