Automatic task scheduling methods, devices, equipment, media and products
By dynamically adjusting scheduling strategies using server registration tables and scheduling policy tables in the absence of a centralized scheduling mechanism and a distributed coordinator, the problem of inflexible scheduling strategies is solved, achieving efficient and simplified automatic task scheduling and improving the system's processing efficiency and availability.
Patent Information
- Application Number
- CN202211175553.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-26
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-09-26
AI Technical Summary
In existing automatic task scheduling mechanisms, scheduling strategies are not easy to define flexibly or change dynamically, centralized scheduling mechanisms cannot guarantee high availability, and the introduction of a distributed coordinator will increase the complexity of system deployment.
Without the intervention of a centralized scheduling mechanism and a distributed coordinator, adaptive scheduling of tasks is achieved by adaptively adjusting the scheduling strategy according to the size of the server cluster, dynamically setting the server scheduling strategy using the server registration table and scheduling strategy table, and using an optimistic locking mechanism for task contention.
It simplifies the deployment structure of the scheduling system, improves the processing efficiency of automatic tasks, and can adapt to the needs of different scenarios, evenly distributing the task processing load.
Smart Images

Figure CN115495217B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of communication technology, and in particular relates to an automatic task scheduling method, apparatus, device, medium and product. Background Technology
[0002] In various distributed systems and components, centralized or distributed coordinators are widely used for scheduling or allocating global tasks, jobs, messages, and other data. Existing automatic task scheduling mechanisms lack flexibility in defining or dynamically changing scheduling strategies, and centralized scheduling mechanisms cannot guarantee high availability; if the centralized scheduling server crashes, the entire scheduling system becomes unavailable. Using a coordinator increases the complexity of system deployment. For example, introducing the distributed coordinator ZooKeeper requires at least three additional machines to ensure high availability, and the number of machines may increase depending on the scheduling workload. Summary of the Invention
[0003] This application provides an automatic task scheduling method, apparatus, device, medium, and product that can adaptively adapt the automatic task scheduling strategy according to the current server cluster size without the intervention of a centralized scheduling mechanism and a distributed coordinator.
[0004] In a first aspect, embodiments of this application provide an automatic task scheduling method, which includes: determining the number of currently active servers based on a server registration table recorded in a database; obtaining target configuration parameters for formulating scheduling strategies from a pre-set scheduling strategy table in the database; determining the scheduling strategy for each currently active server based on the number of currently active servers and the target configuration parameters, wherein the scheduling strategy is used to indicate the automatic tasks that the server needs to scan within a target scheduling period; scanning the automatic tasks to be processed according to the scheduling strategies of each server; and executing the scanned automatic tasks.
[0005] According to an embodiment of the first aspect of this application, the automatic task includes a first automatic task or a second automatic task. The first automatic task includes generating an automatic task instance based on a task definition, and the second automatic task includes processing the automatic task instance. Before determining the number of currently active servers based on a server registration table recorded in the database, the automatic task scheduling method further includes: generating task definitions for multiple automatic tasks to be processed and persisting the multiple task definitions to the database; when the automatic task is the first automatic task, scanning the automatic tasks to be processed according to the scheduling policies of each server, specifically including: scanning multiple task definitions according to the scheduling policies of each server; executing the scanned automatic task, specifically including: generating multiple automatic task instances based on the multiple task definitions; when the automatic task is the second automatic task, scanning the automatic tasks to be processed according to the scheduling policies of each server, specifically including: scanning multiple automatic task instances according to the scheduling policies of each server; executing the scanned automatic task, specifically including: processing the multiple automatic task instances.
[0006] According to the implementation of the first aspect of this application, the execution of the scanned automatic task specifically includes: for any i-th automatic task, if the i-th automatic task is scanned by one server, the server that scanned the i-th automatic task executes the i-th automatic task, where i is a positive integer; if the i-th automatic task is scanned by at least two servers, according to a pre-set contention mechanism, at least two servers contend for the i-th automatic task, and the server that wins the contention for the i-th automatic task executes the i-th automatic task, where i is a positive integer.
[0007] According to the implementation of the first aspect of this application, the scheduling strategy for each currently active server is determined based on the number of currently active servers and the target configuration parameters. Specifically, this includes: obtaining the number of pending automatic tasks; determining the number of slots based on the number of currently active servers, the number of pending automatic tasks, the number of automatic tasks in a preset slot, and a preset maximum offset ratio; for each server, selecting a value from an integer less than the number of slots and greater than or equal to 0 as a random number; determining the starting sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the random number and the number of automatic tasks in a slot; and determining the ending sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the starting sequence number and a preset task processing upper limit.
[0008] According to any of the foregoing embodiments of the first aspect of this application, the range between the start number and the end number is the range of sequence numbers of the automatic tasks that the server needs to scan within the target scheduling period, and there is an offset between the sequence number ranges corresponding to different servers.
[0009] According to the first aspect of this application, after scanning the automatic tasks to be processed according to the scheduling strategy of each server, the method further includes: obtaining the hostname of the target server and the time of scanning the automatic tasks, wherein the target server is the server that has completed scanning the automatic tasks within the target scheduling period; comparing the hostname of the target server with the hostnames of existing servers in the server registry; when the hostname of the target server exists in the server registry, updating the update time of the target server in the server registry based on the time of scanning the automatic tasks of the target server; when the hostname of the target server does not exist in the server registry, adding the hostname of the target server to the server registry, and adding the update time of the target server to the server registry based on the time of scanning the automatic tasks of the target server.
[0010] According to any of the foregoing embodiments of the first aspect of this application, determining the number of currently active servers based on the server registration table recorded in the database specifically includes: obtaining the update time of each server in the server registration table; for any i-th server, determining whether the update time of the i-th server is greater than the difference between the current time and a preset duration; when the update time of the i-th server is greater than the difference between the current time and the preset duration, determining the i-th server as a currently active server and including it in the number of currently active servers.
[0011] Secondly, embodiments of this application provide an automatic task scheduling device, which includes: a first determining module, configured to determine the number of currently active servers based on a server registration table recorded in a database; an obtaining module, configured to obtain target configuration parameters for formulating scheduling strategies from a pre-set scheduling strategy table in the database; a second determining module, configured to determine the scheduling strategy for each currently active server based on the number of currently active servers and the target configuration parameters, wherein the scheduling strategy is used to indicate the automatic tasks that the server needs to scan within a target scheduling period; a scanning module, configured to scan the automatic tasks to be processed according to the scheduling strategies of each server; and an execution module, configured to execute the scanned automatic tasks.
[0012] Thirdly, embodiments of this application provide an electronic device, which includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the automatic task scheduling method provided in the first aspect.
[0013] Fourthly, embodiments of this application provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the automatic task scheduling method provided in the first aspect.
[0014] Fifthly, embodiments of this application provide a computer program product stored in a non-volatile storage medium, which, when executed by at least one processor, implements the automatic task scheduling method provided in the first aspect.
[0015] The automatic task scheduling method, apparatus, device, medium, and product of this application embodiment directly determine the number of currently active servers based on the server registration table, and dynamically set the scheduling strategy for each currently active server according to the number of currently active servers and the target configuration parameters in the scheduling strategy table. This eliminates the need for a centralized scheduling mechanism and a distributed coordinator, simplifying the deployment structure of the scheduling system. Furthermore, because the scheduling strategy for each server is dynamically adjusted, it can flexibly adapt to different scenarios, whether it's a scenario with a large number of servers, fast execution, and low overhead, or a scenario with a small number of servers, slow execution, and high overhead, and distribute the servers as evenly as possible, thereby improving the processing efficiency of automatic tasks. Attached Figure Description
[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart illustrating an automatic task scheduling method.
[0018] Figure 2 This is a flowchart illustrating an automatic task scheduling method provided in an embodiment of this application;
[0019] Figure 3 This is a flowchart illustrating an automatic task scheduling method provided in another embodiment of this application;
[0020] Figure 4 This is a schematic diagram of the structure of an automatic task scheduling device provided in an embodiment of this application;
[0021] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0022] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.
[0023] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
[0024] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0025] Various modifications and variations can be made to this application without departing from its spirit or scope, which will be apparent to those skilled in the art. Therefore, this application is intended to cover modifications and variations falling within the scope of the corresponding claims (the claimed technical solutions) and their equivalents. It should be noted that the embodiments provided in this application can be combined with each other without contradiction.
[0026] Before describing the technical solutions provided in the embodiments of this application, the research results of the inventors of this application will be specifically explained to facilitate understanding of the embodiments of this application:
[0027] Figure 1 This is a flowchart illustrating an automatic task scheduling method. For example... Figure 1As shown, the Quartz framework can be used to trigger the scheduling of automatic tasks. Within a scheduling cycle, the Quartz framework triggers the generation and execution of automatic task instances every few seconds. The server's task generator dispatcher scans for task definitions according to the scheduling policy, and the task generator competes for task definitions using optimistic locking. The server that successfully competes for a task definition generates an automatic task instance. The server's task dispatcher scans for automatic task instances according to the scheduling policy, and the task handler competes for automatic task instances using optimistic locking. The server that successfully competes for a task executes the automatic task instance.
[0028] The inventors of this application have discovered through research that Figure 1 The automatic task scheduling method shown lacks a central scheduling node for centralized control when multiple servers are simultaneously scanning task definitions and automatic task instances. Furthermore, the individual servers are unaware of each other's existence and cannot coordinate task allocation. Each server mechanically follows its own scanning logic, scanning a certain number of pending task definitions and automatic task instances. This results in a scheduling strategy that cannot adapt to different application scenarios, and the overall processing efficiency of the scheduling system is also low.
[0029] In view of the inventors’ above-mentioned research findings, the embodiments of this application provide an automatic task scheduling method, apparatus, device, medium and product that can adaptively adapt the scheduling strategy of automatic tasks according to the current server cluster size without the intervention of a centralized scheduling mechanism and a distributed coordinator.
[0030] The automatic task scheduling method provided in the embodiments of this application will be introduced first below.
[0031] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.
[0032] Figure 2 This is a flowchart illustrating an automatic task scheduling method provided in an embodiment of this application. Figure 2 As shown, the automatic task scheduling method includes the following steps S201 to S205.
[0033] S201. Determine the number of currently active servers based on the server registration table recorded in the database.
[0034] A server registration table is pre-set in the database. This table records the hostname, type, and time of the last automatic scan task for each server; this embodiment does not impose specific limitations on these details. Each server records its current time in the server registration table during each automatic scan task, updating the record to the time of the last automatic scan task. Since each server's hostname is unique, the currently active servers can be determined based on the time of the last automatic scan task recorded in the server registration table, thus obtaining the number of currently active servers.
[0035] S202. Obtain the target configuration parameters for formulating the scheduling strategy from the pre-set scheduling strategy table in the database.
[0036] A scheduling strategy table is pre-set in the database. The scheduling strategy table may include, but is not limited to, target configuration parameters such as the number of automatic tasks in a slot, the upper limit of task processing, and the maximum offset ratio. The preset target configuration parameter values can be used as the basis for formulating scheduling strategies.
[0037] S203. Based on the number of currently active servers and the target configuration parameters, determine the scheduling policy for each currently active server. The scheduling policy is used to indicate the automatic tasks that the server needs to scan within the target scheduling period.
[0038] After obtaining the number of currently active servers and the target configuration parameters, the scheduling system can formulate a scheduling strategy for each currently active server according to the pre-set scanning rules, so as to instruct it to scan the corresponding automatic tasks within the target scheduling period.
[0039] S204. Based on the scheduling policies of each server, scan for pending automatic tasks.
[0040] After receiving the corresponding scheduling policy, each server scans for the corresponding automatic tasks to be processed within the target scheduling period according to the instructions of the scheduling policy.
[0041] S205. Perform the automatic tasks after scanning.
[0042] After each server has scanned all the pending automated tasks within the target scheduling period, it continues to execute those scanned automated tasks.
[0043] The above describes the specific implementation of the automatic task scheduling method provided in this application. This application directly determines the number of currently active servers based on the server registration table, and dynamically sets the scheduling strategy for each currently active server according to the number of currently active servers and the target configuration parameters in the scheduling strategy table. This eliminates the need for a centralized scheduling mechanism and a distributed coordinator, simplifying the deployment structure of the scheduling system. Furthermore, since the scheduling strategy for each server is dynamically adjusted, it can flexibly adapt to different scenarios, whether the number of servers is large, execution is fast, and overhead is low, or the number of servers is small, execution is slow, and overhead is high, ensuring a more even distribution across the servers and improving the processing efficiency of automatic tasks.
[0044] Figure 3 This is a flowchart illustrating an automatic task scheduling method provided in another embodiment of this application. Figure 3 As shown, within a scheduling cycle, the automatic task scheduling method includes the following steps S301 to S307.
[0045] Step S301: The Quartz framework triggers automatic task instance generation and automatic task instance execution every few seconds.
[0046] Step S302: Before scanning the automatic task, each server needs to update its own information in the server registration table to indicate that it is alive.
[0047] Step S303: Determine the number of currently active servers by querying the server registration table.
[0048] Step S304: Obtain the target configuration parameters by querying the scheduling policy table, and formulate a scheduling policy based on the number of currently active servers and the target configuration parameters.
[0049] Step S305: The server's task generator dispatcher scans task definitions according to the scheduling policy. The task generator competes for task definitions using optimistic locking. The server that successfully competes for a task definition generates an automatic task instance.
[0050] Step S306: The server's task dispatcher scans for automatic task instances according to the scheduling policy. The task executor competes for automatic task instances using optimistic locking. The server that successfully competes for the automatic task instance executes the automatic task instance.
[0051] Step S307: One scheduling cycle ends.
[0052] In one example, the automated task includes a first automated task or a second automated task. The first automated task includes generating automated task instances based on task definitions, and the second automated task includes processing automated task instances. Before determining the number of currently active servers based on the server registration table recorded in the database, the automated task scheduling method further includes: generating task definitions for multiple automated tasks to be processed and persisting the multiple task definitions to the database; when the automated task is the first automated task, scanning the automated tasks to be processed according to the scheduling policies of each server, specifically including: scanning multiple task definitions according to the scheduling policies of each server; executing the scanned automated task, specifically including: generating multiple automated task instances based on the multiple task definitions; when the automated task is the second automated task, scanning the automated tasks to be processed according to the scheduling policies of each server, specifically including: scanning multiple automated task instances according to the scheduling policies of each server; executing the scanned automated task, specifically including: processing the multiple automated task instances.
[0053] Specifically, the scheduling of automated tasks is divided into three stages. In the automated task definition stage, the automated task application, based on its own business scenario, calls the automated task scheduling service through the task definition interface provided by the Quartz framework to define different types of automated tasks. The automated task component then persists multiple task definitions to the database. An automated task must be instantiated as an automated task instance before it can be scheduled by the Quartz framework. In the automated task instance generation stage, the Quartz framework provides interfaces for generating and querying automated task instances. By calling the automated task instance generation service through the Quartz framework, after the server scans and completes multiple task definitions, an automated task instance is generated based on the definitions and persisted to the database; this is the first automated task. In the automated task instance execution stage, the Quartz framework calls the automated task instance execution service. After the server scans and completes multiple automated task instances, the automated task instance or batch processing task is scheduled for execution; this is the second automated task.
[0054] Quartz is an open-source job scheduling framework that provides a rich set of job schedulers. The scheduling system uses the Quartz framework to define automatic tasks, generate automatic task instances, and schedule and execute automatic task instances, providing a simple yet powerful mechanism for job scheduling in Java applications.
[0055] In one example, executing the scanned automatic task specifically includes: for any i-th automatic task, if the i-th automatic task is scanned by one server, the server that scanned the i-th automatic task executes the i-th automatic task, where i is a positive integer; if the i-th automatic task is scanned by at least two servers, according to a pre-set contention mechanism, at least two servers contend for the i-th automatic task, and the server that wins the contention for the i-th automatic task executes the i-th automatic task, where i is a positive integer.
[0056] Specifically, as long as one server is connected to the database, it can handle automatic task instances. If multiple servers are connected to the database, because each server has an equal interface for generating and scheduling automatic task instances, they will compete for the task definitions and automatic task instances available for processing according to a pre-set contention mechanism. Only the server that wins the data processing right can execute the subsequent automatic task that generates or processes an automatic task instance; otherwise, it will abandon processing the task definition or automatic task instance.
[0057] Before servers compete for task definitions, they need to scan multiple task definitions and filter those that meet a first preset condition. This first preset condition may include: the task status is normal, the current time is between the preset task start and end times, and the current time is before the next automatic task instance execution time. The task status can be categorized as pending, executing, successful, failed, and canceled, depending on the actual situation. After filtering out task definitions that meet the first preset condition, these task definitions are submitted to the task generators of each server for contention using an optimistic locking mechanism. The server that successfully acquires the task can generate an automatic task instance.
[0058] Before servers compete for automatic task instances, they need to scan multiple automatic task instances to filter out those that meet a second preset condition. This second preset condition may include the automatic task instance being in a pending execution state and the current time being after a preset start time for the automatic task instance. The automatic task instance state can be categorized as pending execution, executing, successful execution, failed execution, and canceled, depending on the actual situation. After filtering out automatic task instances that meet the second preset condition, these instances are handed over to the task executors of each server, using an optimistic locking mechanism for contention. The server that successfully acquires the instance can execute it and is responsible for maintaining the instance's status (successful or failed execution) and recording auxiliary information about the execution result, such as exception descriptions.
[0059] In one example, the scheduling strategy for each active server is determined based on the number of currently active servers and the target configuration parameters. Specifically, this includes: obtaining the number of pending automatic tasks; determining the number of slots based on the number of currently active servers, the number of pending automatic tasks, the number of automatic tasks in a preset slot, and the preset maximum offset ratio; for each server, selecting a value from an integer less than the number of slots and greater than or equal to 0 as a random number; determining the starting sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the random number and the number of automatic tasks in a slot; and determining the ending sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the starting sequence number and the preset task processing upper limit.
[0060] Specifically, first, the number of currently active servers is obtained by querying the server registration table. Then, the number of automated tasks in a pre-defined slot, the pre-defined maximum offset ratio, and the pre-defined task processing limit are obtained by querying the scheduling policy table. Here, the number of automated tasks in a pre-defined slot represents how many task definitions or automated task instances are grouped together; the pre-defined maximum offset ratio represents the maximum offset ratio calculated based on the number of currently active servers; and the pre-defined task processing limit represents the maximum number of task definitions or automated task instances that can be processed each time. Second, based on the number of pending automated tasks, the number of currently active servers, the number of automated tasks in a pre-defined slot, and the pre-defined maximum offset ratio, the number of slots is calculated using the following expression: slotNum = min(ceiling(totalNum / slotSize), (activeServers-1)*maxSlotRatio+1), where slotNum is the number of slots, ceiling is the rounding up value, and totalNum... The first part of the algorithm defines the number of automated tasks to be processed, where `slotSize` is the number of automated tasks in a preset slot, `activeServers` is the number of currently active servers, and `maxSlotRatio` is the preset maximum offset ratio. Next, a random number is calculated based on the number of slots, resulting in any integer within a range less than the number of slots and greater than or equal to 0, i.e., `rand = randInt(slotNum)`, where `rand` is the random number and `randInt` is the function that selects a random number based on the number of slots. Finally, based on the random number and the preset number of automated tasks in a slot, the starting sequence number of the automated tasks the server needs to scan within the target scheduling period is determined. Based on the starting sequence number and the preset task processing upper limit, the ending sequence number of the automated tasks the server needs to scan within the target scheduling period is determined. The range between the starting and ending sequence numbers is determined by the following expression: `[rand*slotSize, rand*slotSize + maxRows]`, where `maxRows` is the preset task processing upper limit.
[0061] For example, if the number of automatic task instances to be processed is 5000, the number of automatic tasks in a preset slot is 50, the preset maximum offset ratio is 3, and the preset task processing limit is 200, when there is 1 server, slotNum = min(100,0) = 0, rand = randInt(0) = 0, and the 0th to 200th records are scanned within a target scheduling period, processing a maximum of 200 data entries; when there are 3 servers, slotNum = min(100,7) = 7, server 1 rand = 0, and the 0th to 200th records are scanned within a target scheduling period; server 2 rand = 3, and the 150th to 350th records are scanned within a target scheduling period; server 3 rand = 6, and the 300th to 500th records are scanned within a target scheduling period. The overlapping parts of the scans are contested by each machine through optimistic locking to ensure that they are not executed repeatedly. Approximately 300 to 500 data entries can be processed within a target scheduling period.
[0062] When multiple automated task servers scan automated tasks simultaneously, they compete for the tasks using optimistic locking. The target configuration parameters in the scheduling strategy table are preset according to the number of deployed servers, so that the scanning of each server is staggered to a certain extent. Without the intervention of a centralized scheduling mechanism and a distributed coordinator, the deployment structure is simplified and the overall processing efficiency of the scheduling system is improved.
[0063] In one example, the range between the start and end sequence numbers represents the sequence number range of the automated tasks that the server needs to scan within the target scheduling period, and there is an offset between the sequence number ranges corresponding to different servers.
[0064] Specifically, the range of sequence numbers of the automated tasks that the server needs to scan within the target scheduling period is defined by the expression [rand*slotSize, rand*slotSize+maxRows]. Different servers determine their corresponding scan sequence number range based on the random numbers they select. Therefore, each server selects different random numbers, and their corresponding sequence number ranges also have a certain offset. This, to some extent, avoids the phenomenon of multiple servers scanning automated tasks within the same sequence number range simultaneously, reduces contention between servers, and improves overall processing efficiency.
[0065] In one example, after scanning the pending automatic tasks according to the scheduling policies of each server, the process further includes: obtaining the hostname of the target server and the time of scanning the automatic tasks, where the target server is the server that has completed scanning the automatic tasks within the target scheduling period; comparing the hostname of the target server with the hostnames of existing servers in the server registry; if the hostname of the target server exists in the server registry, updating the update time of the target server in the server registry based on the time of scanning the automatic tasks of the target server; if the hostname of the target server does not exist in the server registry, adding the hostname of the target server and the update time of the target server in the server registry based on the time of scanning the automatic tasks of the target server.
[0066] Specifically, the server registry records the hostname of each target server and the time of its automated scanning task. After each target server completes scanning of the automated tasks within a target scheduling cycle, it compares the hostname of each server with the hostnames already recorded in the server registry. If the comparison matches, it means that the target server's information already exists in the server registry; in this case, only the time when the target server just executed its scanning task needs to be updated in the server registry. If the comparison does not match, it means that the server is a newly added server to the scheduling system; in this case, the server's hostname needs to be recorded in the server registry, and the time when its subsequent scanning tasks are executed needs to be updated in the server registry.
[0067] By setting up a server registration table in the database, before the automatic task server triggers the automatic task instance generation service and the automatic task instance execution service each time using the Quartz framework, the information of each server is updated in the server registration table. This allows each server to clearly know how many servers are currently active in the scheduling system, so that the corresponding scheduling strategy can be flexibly adjusted according to the number of currently active servers, rather than mechanically determining the scheduling strategy based on the number of servers in the scheduling system.
[0068] In one example, the number of currently active servers is determined based on the server registration table recorded in the database. Specifically, this includes: obtaining the update time of each server in the server registration table; for any i-th server, determining whether the update time of the i-th server is greater than the difference between the current time and the preset duration; when the update time of the i-th server is greater than the difference between the current time and the preset duration, the i-th server is determined as a currently active server and included in the number of currently active servers.
[0069] Specifically, a preset duration is set as the basis for determining whether a server is currently active. If the last update time of the server is before the current time and after a certain moment when the current time minus the preset duration, that is, when the update time of the server is greater than the difference between the current time and the preset duration, the server can be regarded as a currently active server and counted in the number of currently active servers.
[0070] By setting a preset duration, the currently active servers can be accurately selected and the number of currently active servers can be counted, so that the corresponding scheduling strategy can be flexibly adjusted according to the number of currently active servers.
[0071] Based on the automatic task scheduling method provided in the above embodiments, this application also provides specific implementations of an automatic task scheduling device. Please refer to the following embodiments.
[0072] First see Figure 4 The automatic task scheduling device 40 provided in this application embodiment includes the following modules:
[0073] The first determining module 401 is used to determine the number of currently active servers based on the server registration table recorded in the database;
[0074] The acquisition module 402 is used to obtain the target configuration parameters for formulating the scheduling strategy from the pre-set scheduling strategy table in the database;
[0075] The second determining module 403 is used to determine the scheduling strategy of each currently active server based on the number of currently active servers and the target configuration parameters. The scheduling strategy is used to indicate the automatic tasks that the server needs to scan within the target scheduling period.
[0076] The scanning module 404 is used to scan for automated tasks to be processed according to the scheduling policies of each server.
[0077] Execution module 405 is used to perform automated tasks after scanning.
[0078] The automatic task scheduling device provided in this application directly determines the number of currently active servers based on the server registration table, and dynamically sets the scheduling strategy for each currently active server according to the number of currently active servers and the target configuration parameters in the scheduling strategy table. This eliminates the need for a centralized scheduling mechanism and a distributed coordinator, simplifying the deployment structure of the scheduling system. Furthermore, because the scheduling strategy for each server is dynamically adjusted, it can flexibly adapt to different scenarios, whether it's a scenario with a large number of servers, fast execution, and low overhead, or a scenario with a small number of servers, slow execution, and high overhead, and distribute the servers as evenly as possible, thereby improving the processing efficiency of automatic tasks.
[0079] In some embodiments, the automatic task includes a first automatic task or a second automatic task. The first automatic task includes generating an automatic task instance based on a task definition, and the second automatic task includes processing the automatic task instance. The automatic task scheduling device 40 may further include a generation module for generating task definitions for multiple automatic tasks to be processed and persisting the multiple task definitions to a database. The scanning module 404 is specifically used for: when the automatic task is the first automatic task, scanning multiple task definitions according to the scheduling policies of each server; executing the scanned automatic task, specifically including: generating multiple automatic task instances based on the multiple task definitions; when the automatic task is the second automatic task, scanning multiple automatic task instances according to the scheduling policies of each server; and executing the scanned automatic task, specifically including: processing the multiple automatic task instances.
[0080] In some embodiments, the execution module 405 is specifically used for: for any i-th automatic task, if the i-th automatic task is scanned by one server, the server that scanned the i-th automatic task executes the i-th automatic task, where i is a positive integer; if the i-th automatic task is scanned by at least two servers, according to a pre-set contention mechanism, at least two servers contend for the i-th automatic task, and the server that wins the contention for the i-th automatic task executes the i-th automatic task, where i is a positive integer.
[0081] In some embodiments, the second determining module 403 is specifically configured to: obtain the number of pending automatic tasks; determine the number of slots based on the number of currently active servers, the number of pending automatic tasks, the number of automatic tasks in a preset slot, and a preset maximum offset ratio; for each server, select a value from integers less than the number of slots and greater than or equal to 0 as a random number; determine the starting sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the random number and the number of automatic tasks in a slot; and determine the ending sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the starting sequence number and a preset task processing upper limit.
[0082] In some embodiments, the range between the start number and the end number is the range of sequence numbers of the automatic tasks that the server needs to scan within the target scheduling period, and there is an offset between the sequence number ranges corresponding to different servers.
[0083] In some embodiments, the automatic task scheduling device 40 may further include an update module, configured to obtain the hostname of the target server and the time of scanning the automatic task, wherein the target server is a server that has completed scanning the automatic task within the target scheduling period; compare the hostname of the target server with the hostnames of existing servers in the server registry; when the hostname of the target server exists in the server registry, update the update time of the target server in the server registry based on the time of scanning the automatic task of the target server; when the hostname of the target server does not exist in the server registry, add the hostname of the target server and the update time of the target server in the server registry based on the time of scanning the automatic task of the target server.
[0084] In some embodiments, the first determining module 401 is specifically used to: obtain the update time of each server in the server registration table; for any i-th server, determine whether the update time of the i-th server is greater than the difference between the current time and the preset duration; when the update time of the i-th server is greater than the difference between the current time and the preset duration, determine the i-th server as a currently active server and include it in the number of currently active servers.
[0085] Figure 4 Each module in the illustrated device has the ability to implement Figure 2 The functions of each step in the process and their corresponding technical effects are described in detail here for the sake of brevity.
[0086] Based on the automatic task scheduling method provided in the above embodiments, this application also provides specific implementation methods for electronic devices. Please refer to the following embodiments.
[0087] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.
[0088] The electronic device may include a processor 501 and a memory 502 storing computer program instructions.
[0089] Specifically, the processor 501 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0090] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one example, memory 502 may include removable or non-removable (or fixed) media, or memory 502 may be non-volatile solid-state memory. Memory 502 may be internal or external to the integrated gateway disaster recovery device.
[0091] In one example, memory 502 may be read-only memory (ROM). In one example, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.
[0092] Memory 502 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.
[0093] The processor 501 reads and executes computer program instructions stored in the memory 502 to achieve... Figure 2 The methods / steps S201 to S205 in the illustrated embodiments achieve the following: Figure 2 The technical effects achieved by executing the methods / steps shown in the examples are not elaborated here for the sake of brevity.
[0094] In one example, the electronic device may also include a communication interface 503 and a bus 510. Wherein, as... Figure 5As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 510 and complete communication with each other.
[0095] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.
[0096] Bus 510 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.
[0097] Furthermore, in conjunction with the automatic task scheduling method in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the automatic task scheduling methods in the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as electronic circuits, semiconductor memory devices, ROM, random access memory, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, and hard disks.
[0098] This application also provides a computer program product, which is stored in a non-volatile storage medium and, when executed by at least one processor, implements the automatic task scheduling method provided in the above embodiments.
[0099] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.
[0100] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0101] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0102] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.
[0103] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.
Claims
1. An automatic task scheduling method, characterized in that, include: Determine the number of currently active servers based on the server registration table recorded in the database; Obtain the target configuration parameters for formulating the scheduling strategy from the pre-set scheduling strategy table in the database; Based on the number of currently active servers and the target configuration parameters, a scheduling strategy for each currently active server is determined. The scheduling strategy is used to indicate the automatic tasks that the server needs to scan within the target scheduling period. Scan for pending automated tasks according to the scheduling policies of each server; Execute the automated task after scanning; The step of determining the scheduling strategy for each currently active server based on the number of currently active servers and the target configuration parameters specifically includes: Obtain the number of the automated tasks to be processed; The number of slots is determined based on the number of currently active servers, the number of automated tasks to be processed, the number of automated tasks in a preset slot, and the preset maximum offset ratio; For each of the servers, a value is selected as a random number from an integer that is less than the number of slots and greater than or equal to 0. Based on the random number and the number of automatic tasks in a slot, determine the starting sequence number of the automatic tasks that the server needs to scan within the target scheduling period; Based on the starting sequence number and the preset task processing upper limit, the end sequence number of the automatic tasks that the server needs to scan within the target scheduling period is determined.
2. The method according to claim 1, characterized in that, The automatic task includes a first automatic task or a second automatic task. The first automatic task includes generating an automatic task instance according to a task definition, and the second automatic task includes processing the automatic task instance. Before determining the number of currently active servers based on the server registration table recorded in the database, the method further includes: Generate task definitions for multiple automated tasks to be processed, and persist the multiple task definitions to the database; When the automatic task is the first automatic task, scanning the automatic tasks to be processed according to the scheduling policy of each server specifically includes: scanning multiple task definitions according to the scheduling policy of each server; executing the scanned automatic task specifically includes: generating multiple automatic task instances according to the multiple task definitions. When the automatic task is the second automatic task, scanning the automatic tasks to be processed according to the scheduling policy of each server specifically includes: scanning multiple automatic task instances according to the scheduling policy of each server; executing the scanned automatic task specifically includes: processing multiple automatic task instances.
3. The method according to claim 1, characterized in that, The automated task performed after the scan specifically includes: For any i-th automatic task, if the i-th automatic task is scanned by a server, the i-th automatic task is executed by the server that scanned the i-th automatic task, where i is a positive integer; If the i-th automatic task is scanned by at least two servers, according to a pre-set contention mechanism, at least two servers will compete for the i-th automatic task, and the server that wins the competition for the i-th automatic task will execute the i-th automatic task, where i is a positive integer.
4. The method according to claim 1, characterized in that, The range between the start number and the end number is the range of sequence numbers of the automatic tasks that the server needs to scan within the target scheduling period, and there is an offset between the sequence number ranges corresponding to different servers.
5. The method according to claim 1, characterized in that, After scanning for pending automated tasks according to the scheduling policies of each server, the process further includes: Obtain the hostname of the target server and the time of scanning the automatic task, wherein the target server is the server that has completed scanning the automatic task within the target scheduling period; Compare the hostname of the target server with the hostname of the server already existing in the server registration table; When the hostname of the target server exists in the server registry, the update time of the target server in the server registry is updated based on the time when the automatic task of scanning the target server is performed. When the hostname of the target server does not exist in the server registration table, the hostname of the target server is added to the server registration table, and the update time of the target server is added to the server registration table based on the time of the automatic task of scanning the target server.
6. The method according to claim 5, characterized in that, The step of determining the number of currently active servers based on the server registration table recorded in the database specifically includes: Obtain the update time of each server in the server registration table; For any i-th server, determine whether the update time of the i-th server is greater than the difference between the current time and the preset duration; When the update time of the i-th server is greater than the difference between the current time and the preset duration, the i-th server is determined as a currently active server and included in the number of currently active servers.
7. An automatic task scheduling device, characterized in that, The device includes: The first determination module is used to determine the number of currently active servers based on the server registration table recorded in the database. The acquisition module is used to acquire target configuration parameters for formulating scheduling strategies from a pre-set scheduling strategy table in the database; The second determining module is configured to: determine the scheduling strategy for each currently active server based on the number of currently active servers and the target configuration parameters; the scheduling strategy is used to indicate the number of automatic tasks that the server needs to scan within the target scheduling period, and is also used to obtain the number of automatic tasks to be processed; determine the number of slots based on the number of currently active servers, the number of automatic tasks to be processed, the number of automatic tasks in a preset slot, and a preset maximum offset ratio; for each server, select a value from integers less than the number of slots and greater than or equal to 0 as a random number; determine the starting sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the random number and the number of automatic tasks in a slot; and determine the ending sequence number of the automatic tasks that the server needs to scan within the target scheduling period based on the starting sequence number and a preset task processing upper limit. The scanning module is used to scan for automated tasks to be processed according to the scheduling policies of each server. An execution module is used to perform the automated tasks after scanning.
8. An electronic device, characterized in that, The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the automatic task scheduling method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the automatic task scheduling method as described in any one of claims 1 to 6.
10. A computer program product, characterized in that, The computer program product is stored in a non-volatile storage medium, and when executed by at least one processor, the computer program product implements the automatic task scheduling method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Data prediction method, server and computer readable storage medium
CN108038563A
Intelligent task scheduling method and device, storage medium and electronic equipment
CN110716796A
