Task scheduling method and task scheduling system

Abstract

The invention relates to the technical field of task scheduling, in particular to a task scheduling method and a task scheduling system.The method comprises the steps that a working mode is determined based on sensing information of a collaborative sensing module and each associated scheduling center; if it is determined that the working mode of the current dispatching center is in the multi-active mode, obtaining a target to-be-processed task and a correspondingly configured execution object from a to-be-processed task table; wherein the dispatching centers corresponding to all the machine rooms survive in the multi-activity mode; the execution object is a scheduling center for executing the corresponding to-be-processed task; if it is determined that the execution object is the current scheduling center, the current scheduling center is controlled to execute the target to-be-processed task, and otherwise, a target execution object is determined between the current scheduling center and the execution object according to the condition of the executor under the execution object and the target to-be-processed task is executed. Therefore, task scheduling can be realized simply, efficiently and safely.

Description

Technical Field

[0001] This application relates to the field of task scheduling technology, and in particular to a task scheduling method and a task scheduling system. Background Technology

[0002] Currently, the open-source task scheduling framework XXL-JOB is widely used in the industry due to its advantages such as lightweight, easy integration, and visual management.

[0003] In typical multi-datacenter deployment practices, enterprises often adopt the following two methods to achieve high availability: Primary / standby mode: Only one data center is used to deploy the active dispatch center, while the other data center serves as a cold standby. Switching depends on manual intervention or an external monitoring system. Active-active mode: Multiple data centers each deploy independent scheduling center instances, and synchronize metadata through a shared database to form a logical cluster.

[0004] However, when the data center fails, it can easily lead to task loss or require more complex remedial measures. Summary of the Invention

[0005] In view of this, embodiments of this application provide a task scheduling method and a task scheduling system that can achieve task scheduling simply, efficiently, and securely.

[0006] In a first aspect, embodiments of this application provide a task scheduling method applicable to a task scheduling system, the task scheduling system comprising: multiple scheduling centers, an executor cluster corresponding to each scheduling center, and a collaborative sensing module; Based on the sensing information of the collaborative sensing module and each associated scheduling center, the working mode is determined; If it is determined that the current scheduling center is in a multi-active mode, then the target task to be processed and the corresponding configured execution object are obtained from the task to be processed table; wherein, in the multi-active mode, the scheduling centers corresponding to all data centers are active; the execution object is the scheduling center that executes the corresponding task to be processed; If the execution object is determined to be the current scheduling center, then the current scheduling center is controlled to execute the target task to be processed; otherwise, based on the executor status of the execution object, a target execution object is determined between the current scheduling center and the execution object, and the target task to be processed is executed.

[0007] In some embodiments, the sensing information includes node change information; The process of determining the working mode based on the sensing information of the collaborative sensing module and each associated scheduling center includes: The current scheduling center is controlled to listen for node change information. When the node change information is received, the temporary sequential node data is traversed to determine whether there is a corresponding temporary sequential node in all scheduling centers. If it exists, then the working mode is a multi-active mode; otherwise, it is a non-multi-active mode.

[0008] In some embodiments, the method includes: When each of the scheduling centers starts up, it registers its own metadata with the collaborative perception module. The collaborative perception module generates a corresponding temporary sequence node based on each metadata and obtains the temporary sequence node data. If the collaborative sensing module detects a disconnection with any scheduling center, it controls the collaborative sensing module to delete the corresponding temporary sequential node from the temporary sequential node data and send node change information to each scheduling center.

[0009] In some embodiments, the method further includes: If it is determined that the current scheduling center is in a non-active mode, then the current scheduling center is controlled to execute each of the pending tasks in the order configured in the pending task table.

[0010] In some embodiments, the method further includes: Configure the task list to be processed using the following method: The first target scheduling center is designated as the execution target for the first type of pending tasks; The second category of tasks to be processed is divided according to the resource allocation and preset ratio strategy of each dispatch center.

[0011] In some embodiments, determining the target execution object from between the current scheduling center and the execution object and executing the target task to be processed based on the executor status of the execution object includes: If it is determined that the execution object configured in the pending task table is not the current scheduling center, determine whether there is an available executor in the target scheduling center configured for the target pending task; If it exists, the current scheduling center is controlled to skip the target task to be processed; If it does not exist, then control the current scheduling center to execute the target task to be processed.

[0012] In some embodiments, the following method is used to detect whether there are available executors in each scheduling center: Traverse the set of executor addresses registered in the target scheduling center and check the tag attribute of each executor address; if the tag attribute does not belong to the target scheduling center, it is determined that there is no available executor in the target scheduling center.

[0013] In some embodiments, the method further includes: Before each actuator registers its own information with the corresponding scheduling center, it determines the target scheduling center to which it belongs based on the IP segment information of the computer room where the actuator is located, and initiates registration information with the target scheduling center. The registration information includes the label information of the computer room.

[0014] In some embodiments, the method further includes: If the scheduling center corresponding to the computer room successfully acquires the database lock, then the scheduling center that successfully acquires the database lock within the preset scheduling time period is the current scheduling center.

[0015] Secondly, embodiments of this application provide a task scheduling system, which includes: a control module, a distributed task scheduling platform, and a collaborative sensing module; the distributed task scheduling platform includes multiple scheduling centers and an executor cluster. The control module is used to implement a task scheduling method provided in the first aspect of this application.

[0016] The embodiments of this application have the following beneficial effects: This application determines the working mode based on the perception information of the collaborative perception module and each associated scheduling center. If the current scheduling center is determined to be in a multi-active mode, the target task to be processed and its corresponding configured execution object are obtained from the task list. In this multi-active mode, all scheduling centers corresponding to all data centers are active. The execution object is the scheduling center that executes the corresponding task to be processed. If the execution object is determined to be the current scheduling center, the current scheduling center is controlled to execute the target task to be processed; otherwise, based on the executor status of the execution object, the target execution object is determined between the current scheduling center and the execution object, and the target task to be processed is executed. This application determines the working mode based on the information of each scheduling center perceived by the collaborative perception module, and executes different task scheduling for different modes. This allows for simple, efficient, and secure task scheduling without missing any tasks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A structural block diagram of a task scheduling system according to an embodiment of this application is shown; Figure 2 A flowchart of a task scheduling method according to an embodiment of this application is shown; Figure 3 This paper illustrates the interaction flowchart between the collaborative perception module and the scheduling center in the task scheduling method of this application embodiment; Figure 4 A flowchart illustrating the determination of the working mode in the task scheduling method of this application is shown; Figure 5 A flowchart illustrating the configuration of the task table in the task scheduling method of this application is shown. Figure 6 A flowchart illustrating the task processing in the task scheduling method of this application is shown.

[0019] Explanation of key component symbols: 100 - Collaborative Sensing Module; 200 - Scheduling Center; 300 - Actuator; 400 - Database. Detailed Implementation

[0020] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0021] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] In the following text, the terms "comprising," "having," and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more combinations thereof. Furthermore, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0023] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.

[0024] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0025] Figure 1 A structural block diagram of a task scheduling system according to an embodiment of this application is shown. Exemplarily, the task scheduling system includes: a control module, a distributed task scheduling platform, and a collaborative sensing module 100; the distributed task scheduling platform includes multiple scheduling centers 200 and an executor cluster. The executor cluster includes multiple executors 300.

[0026] Multiple dispatch centers 200 are deployed in different data centers. Each dispatch center 200 shares the same backend database 400 and senses each other's online status through the collaborative sensing module 100 corresponding to the distributed coordination service.

[0027] In this embodiment, the scheduling center 200 and the executor cluster are implemented based on the XXL-JOB architecture. XXL-JOB is an open-source distributed task scheduling platform whose architecture includes a scheduling center, executors, and a metadata storage module (usually a relational database). The scheduling center provides core functions such as task management, triggering scheduling, and failure retry. The executor 300 is embedded in the business application as a lightweight client and responds to scheduling commands. In this embodiment, ZooKeeper is used to implement executor service discovery.

[0028] The dispatch center (Admin), also known as "XXL-JOB Admin," primarily includes the following functions: Provides a Web UI for task management (CRUD operations); is responsible for task scheduling logic (such as selecting executors and triggering execution); records task execution logs, failure retries, scheduling strategy configurations, etc.; and manages the registration information and status of executors.

[0029] An executor is the actual executor of a task, typically deployed on a business node, and can be multiple instances. Its main functions include: registering its own information with the scheduling center; receiving and executing tasks issued by the scheduling center; and sending the task execution results back to the scheduling center.

[0030] XXL-Job task scheduling process: 1) The executor starts up and registers its information with the scheduling center. The registration information includes the executor name (AppName), IP address, port, and pending tasks list.

[0031] 2) Users configure a list of tasks to be processed in the scheduling center; the task information in the list of tasks to be processed includes task name, executor, scheduling time, scheduling policy, failure policy and execution priority, etc.

[0032] 3) The scheduling center selects the executor according to the scheduling strategy, such as round-robin, failover, consistent hashing, etc.

[0033] 4) The scheduling center sends an HTTP / RPC request to the selected executor to trigger task execution.

[0034] 5) The executor executes the task logic and returns the result.

[0035] 6) The scheduling center records execution logs and status. The following examples illustrate this task scheduling method.

[0036] Figure 2 A flowchart of a task scheduling method according to an embodiment of this application is shown. Exemplarily, the task scheduling method includes the following steps: S100 determines the working mode based on the sensing information of the collaborative sensing module 100 and each associated scheduling center 200.

[0037] Multiple dispatch centers 200 are deployed in different computer rooms. Each dispatch center 200 shares the same backend database 400 and senses each other's online status through the distributed coordination service collaborative sensing module 100.

[0038] In one implementation, such as Figure 3 As shown, the method in this application embodiment includes: S101, each dispatch center 200 registers its own metadata with the collaborative sensing module 100 upon startup. The collaborative sensing module 100 generates a corresponding temporary sequence node based on each metadata, thus obtaining temporary sequence node data. The metadata of the dispatch center 200 includes the IP address of the dispatch center 200 (which is also the IP address of the computer room corresponding to the dispatch center 200).

[0039] S102, when the collaborative sensing module 100 detects that it has lost connection with any scheduling center 200, it controls the collaborative sensing module 100 to delete the corresponding temporary sequential node from the temporary sequential node data and send node change information to each scheduling center 200.

[0040] The collaborative perception module 100 is implemented based on a distributed coordination service, which meets the following conditions: it uses an atomic broadcast protocol to ensure strong data consistency among multiple nodes, supports session lifecycle management based on ephemeral sequential nodes, and provides an asynchronous event listening interface to respond to node data changes. The distributed coordination service is Apache ZooKeeper (referred to as ZooKeeper middleware) or a coordination service with the same protocol semantics. ZooKeeper middleware provides a hierarchical namespace (ZNode)-based data model, strongly consistent read and write operations, and an event notification (Watcher) mechanism. In this application, ZooKeeper is configured to only perform ephemeral node creation, sequential node generation, and Watcher event triggering.

[0041] When each scheduling center 200 starts up, it writes its own IP address into a temporary sequential node in the ZooKeeper middleware.

[0042] When a scheduling center 200 loses its connection with the ZooKeeper middleware or crashes, the temporary sequential node will be automatically deleted by the ZooKeeper middleware, its registered metadata will also be deleted, and node change information will be sent to other scheduling centers 200. Node change information includes, but is not limited to, a delete node event (deleting a temporary sequential node) or an add node event (adding a new temporary sequential node).

[0043] Other dispatch centers 200 can promptly detect the online or offline status of other dispatch centers 200 by listening to node change information.

[0044] In one implementation, the sensing information includes node change information; like Figure 4 As shown, in step S100, based on the sensing information of the collaborative sensing module 100 and each associated scheduling center 200, the working mode is determined, including: S110 controls the current scheduling center to listen for node change information. When node change information is received, it iterates through the temporary sequential node data to determine whether there is a corresponding temporary sequential node in all scheduling centers 200.

[0045] If S120 exists, the working mode is multi-active; otherwise, it is non-multi-active.

[0046] If all scheduling centers 200 have corresponding temporary sequence nodes, it indicates that all scheduling centers 200 are online, and the task scheduling system in this embodiment operates in a multi-active mode. If at least one scheduling center 200 is detected to lack a corresponding temporary sequence node, it indicates that a scheduling center 200 is offline, and the task scheduling system (or the current scheduling center's operating mode) operates in a non-multi-active mode. In this embodiment, the operating mode of the task scheduling system is consistent with the operating mode of the scheduling centers 200.

[0047] As an example, the ZooKeeper middleware is used to monitor the liveness status of 200 tasks at each scheduling center in real time, dynamically determine the working mode, and select task execution paths accordingly. Specifically, this includes: When the two scheduling centers IDC1 and IDC2 start up, they register their respective metadata with the ZooKeeper middleware; the ZooKeeper middleware then creates temporary sequential nodes corresponding to IDC1 and IDC2 respectively and stores them in the database 400. When determining the working mode, the current scheduling center periodically sends a `getChildren(" / scheduler / active")` request to the ZooKeeper middleware to obtain a list of temporary sequential nodes registered by all surviving scheduling centers in database 400; If the returned node list contains two nodes, ` / scheduler / active / 0000000001` (IDC1) and ` / scheduler / active / 0000000002` (IDC2), then the task scheduling system is determined to be in multi-active mode (i.e., both scheduling centers 200 are online).

[0048] Each scheduling center continuously listens for the `NodeChildrenChanged` event of the ` / scheduler / active` node to monitor node change information.

[0049] When the ZooKeeper middleware detects that the scheduling center IDC2 has lost connection with ZooKeeper due to a crash, the ZooKeeper middleware automatically deletes its temporary node `sc_sh_0000000002` and immediately pushes the `NodeChildrenChanged` event to all listeners.

[0050] After receiving the `NodeChildrenChanged` event, the scheduling center IDC1 immediately executes `getChildren(" / scheduler / active",false)`, traversing the returned list of nodes; if only `sc_bj_0000000001` remains, it determines that the task scheduling system has entered non-active mode.

[0051] This embodiment uses a dual confirmation approach of event-driven and active traversal to compress the mode switching response time to ≤200ms (actual measurement P99=187ms), meeting the financial-grade second-level fault self-healing requirements.

[0052] S200: If it is determined that the current scheduling center is in multi-active mode, then the target task to be processed and its corresponding configured execution object are obtained from the task to be processed table. In multi-active mode, all scheduling centers corresponding to all data centers are active; the execution object is the scheduling center that executes the corresponding task to be processed.

[0053] Exemplary, the physical source of the pending task table in this application embodiment is the XXL-JOB database (or other compatible relational database); all task configurations (such as executors, Cron expressions, routing strategies, failure retries, etc.) are persisted in this database 400; the pending task table includes: multiple pending tasks and their corresponding priority configurations (configured execution objects). For example, the pending task table is the xxl_job_info table initialized during XXL-JOB deployment. If the scheduling center 200 includes scheduling center 1 (IDC1) and scheduling center 2 (IDC2), the pending task table is shown in Table 1: Table 1. Tasks to be processed

[0054] In one implementation, such as Figure 5 As shown, the method in this application embodiment further includes: Configure the pending task table using the following method: S201, designates the first target scheduling center as the execution object of the first type of pending tasks.

[0055] The first category of pending tasks includes, but is not limited to, strongly consistency-sensitive tasks. For example, the first category of pending tasks includes, but is not limited to, tasks that must be executed by a specific scheduling center 200.

[0056] In the task configuration center (ConfigCenter), explicitly specify `execution_target=IDC1`. For example, payment reconciliation tasks must be executed by the data center scheduling center IDC1 (because a direct connection to the core database is required).

[0057] S202, the second category of tasks to be processed is divided according to the resource allocation and preset ratio strategy of each dispatch center 200.

[0058] The second category of tasks to be processed includes, but is not limited to, tasks that utilize elastic computing. For example, configuring `execution_target=dynamic` and associating it with a preset proportional strategy (such as `{"IDC1":40%,"IDC2":60%}`).

[0059] Understandably, in this application embodiment, tasks to be processed can be assigned to each scheduling center 200 through both manual and automatic methods.

[0060] Among them, manual assignment: You can manually assign the task to be processed to any scheduling center 200 (named after the data center name) in the current multi-active mode. For example, task 1 is assigned to scheduling center 200 IDC1 and task 2 is assigned to scheduling center IDC2.

[0061] Automatic allocation: Based on the resource configuration of the data center, all tasks can be allocated to different data centers for execution in a proportional manner, thereby improving resource utilization and avoiding the problem of uneven resource allocation due to free contention.

[0062] It should be noted that: Resource configuration includes the number of instances, CPU size, memory, I / O, and other resources.

[0063] For example, according to a preset ratio, the program pulls out all pending tasks in the startup state. If a data center has already been assigned, it will not reassign the data center. For example, if a special task requires a specific data center to be executed, the program will then calculate the number of tasks to be handled by each dispatch center data center according to the set ratio (IDC1:60%, IDC2:40%), and then take the tasks that meet the required number from the pulled task table.

[0064] Furthermore, to improve processing efficiency, embodiments of this application also include; Each dispatch center periodically collects the CPU utilization, memory remaining rate, task queue depth, and average execution time of the actuator cluster in its local data center, and generates dynamic weight coefficients through an exponentially weighted moving average algorithm to correct the preset ratio strategy.

[0065] In multi-active mode, the scheduling center 200 dynamically adjusts the weights based on the real-time CPU utilization, memory level, and network RTT of each data center executor cluster (e.g., if the CPU of the data center corresponding to IDC1 is greater than 90%, its weight is temporarily reduced to 20%). Finally, the target execution object is selected from the available scheduling centers 200 according to the weighted round-robin algorithm.

[0066] Furthermore, the method in this application embodiment also includes: If the scheduling center 200 corresponding to the computer room successfully acquires the database lock, then the scheduling center 200 that successfully acquires the database lock within the preset scheduling time period will be the current scheduling center.

[0067] As an example, the scheduling center is determined based on the exclusive lock native to XXL-JOB and the next scheduling time to determine which scheduling center is allowed to execute the task.

[0068] This application embodiment determines the current scheduling center based on a database lock preemption and scheduling window binding mechanism. For example, the successful preemptor obtains a preset scheduling time period of 30 seconds, during which it has the right to distribute tasks globally; other scheduling centers 200 only execute local tasks during this window period and do not participate in cross-center task routing; 10 seconds before the window expires, the successful preemptor automatically renews the lock (to avoid accidental release due to network jitter); if the lock renewal fails, it actively exits the scheduling role.

[0069] S300: If the execution target is determined to be the current scheduling center, then control the current scheduling center to execute the target task to be processed; otherwise, based on the status of the executor 300 under the execution target, determine the target execution target from between the current scheduling center and the execution target and execute the target task to be processed.

[0070] In one implementation, if it is determined that the current scheduling center is in a non-active-active mode, the current scheduling center is controlled to execute each pending task in the order of the pending tasks configured in the pending task table.

[0071] Each scheduling center 200 monitors the nodes registered with the collaborative sensing module 100. When a registered node's child nodes change, the scheduling center 200 iterates through the node data to determine if each scheduling center 200 in each data center has a registered node. If so, the current working mode is multi-active, and the priority configured in the pending task table determines whether to execute the pending task. If not every scheduling center 200 in each data center has a registered node, the current scheduling center directly executes the currently acquired pending task, ensuring that tasks are executed on time and not missed.

[0072] In one implementation, such as Figure 6 As shown, based on the executor 300 under the execution object, the target execution object is determined from the current scheduling center and the execution object, and the target pending task is executed, including: S410, if it is determined that the execution object configured in the task to be processed table is not the current scheduling center, determine whether there is an available executor in the target scheduling center configured for the target task to be processed.

[0073] S420, if present, controls the current scheduling center to skip the target pending task.

[0074] If the execution target configured for the target task is not the current data center, the modification of the scheduling time of the target task will be abandoned. When the database row lock is released, the row lock will be acquired by the other data center that was waiting for the lock, and then the target task will be acquired, thereby obtaining the execution permission.

[0075] For example, when the scheduling center 200 skips a task, it does not modify the `trigger_next_time` field of that task in the `xxl_job_info` table; it only recalculates and updates the `trigger_next_time` field according to the task scheduling cycle after taking over the task.

[0076] If S430 does not exist, it controls the current scheduling center to execute the target pending task.

[0077] The current scheduling center executes the target pending task and modifies the next scheduling time of the pending task.

[0078] If the multi-active mode is not met or there is no executor 300 under the configured execution object, then the current scheduling center will directly ignore the priority configuration of the pending task and execute the pending task directly, thereby ensuring that the task will not be missed.

[0079] As an example, all scheduling centers 200 will compete for scheduling opportunities (implemented through database row locks). When scheduling center IDC1 acquires the lock, IDC2 can only be in a blocked state. At this time, scheduling center IDC1 retrieves all pending tasks that need to be scheduled from database 400 and analyzes each pending task: The system determines whether all conditions for multi-active scheduling are met, and then checks the configured execution object to see if IDC1 or IDC2 is prioritized. If IDC1 is prioritized, it executes the task directly, modifying its status and next scheduling time. If IDC2 is prioritized, IDC1 skips the task without changing any data. After IDC1 finishes executing all tasks, it releases the lock, allowing IDC2 to acquire it and enter scheduling. The testing logic is the same as for IDC1, retrieving the previously skipped tasks, thus achieving priority scheduling. The entire process is stable, efficient, and reliable. Furthermore, the following method is used to detect whether each scheduling center 200 has an available executor: Iterate through the set of executor addresses registered in the target scheduling center and check the tag attribute of each executor address; if the tag attribute does not belong to the target scheduling center, it is determined that there is no available executor in the target scheduling center.

[0080] If the execution target of the pending task captured by the current scheduling center is another data center, and if the multi-active check passes (all scheduling centers 200 are in active service state), then the set of registered executor addresses is traversed, and the data center tag attribute of each executor address is checked. If none of the data center tag attributes are in the data center corresponding to the configured execution target, it means that there is no available executor 300 in that data center. In this case, the current scheduling center directly takes over the execution of the pending task and changes it to the executed state to prevent the pending task from being missed.

[0081] In one embodiment, the method of this application further includes: Before each actuator 300 registers its own information with the corresponding scheduling center, it determines the target scheduling center to which it belongs based on the IP segment information of the computer room where the actuator 300 is located, and initiates registration information to the target scheduling center. The registration information includes the label information of the computer room.

[0082] This application embodiment improves the xxl-job platform. Before the executor 300 (corresponding to the application system) registers with the scheduling center 200, it determines whether it belongs to IDC1 or IDC2 based on the IP segment of the data center it is in, and then initiates the registration. Based on the original information, the data center tag information (e.g., zone=IDC1) is transmitted to the corresponding scheduling center.

[0083] In this embodiment of the application, during data center scheduling, the system determines whether the current scheduling center needs to schedule based on the priority data centers (corresponding execution objects) configured in the task-to-process table, while considering the following conditions: Is the other party's data center dispatch center still operational? Does the other party's computer room have a usable actuator?

[0084] If the other data center does not meet the execution conditions, the current data center will directly take over the execution of the pending task; if the other data center meets the scheduling conditions, the current scheduler will ignore the execution of the current task, so that the other data center can quickly obtain the task execution permission when it gets a scheduling opportunity.

[0085] In one embodiment, this application also includes a predictive task takeover method based on timing anomaly detection. The method further includes: Historical operational metric sequences of the scheduling centers corresponding to each data center are collected and input into a trained lightweight time-series anomaly detection model, which outputs the probability of downtime within a specified future time window. When the probability exceeds a preset threshold, a preloading instruction is sent to the scheduling center corresponding to the target data center to be taken over, enabling it to complete task context initialization and connection pool warm-up in advance. Specifically, a lightweight time-series anomaly detection model (LSTM-AutoEncoder) is constructed, using historical operational metric sequences composed of metrics from the scheduling centers of each data center over the past 2 hours (ZooKeeper session timeouts, DB connection pool wait times, and task failure rates) as input. For example, the model outputs "probability of IDC1 downtime in the next 15 minutes" every 5 minutes. When the probability > 85%, a "preparatory takeover instruction" is sent to IDC2 in advance, and IDC2 preloads the task context and warms up the executor connection pool. When a real downtime occurs, the takeover latency is reduced from seconds to milliseconds.

[0086] In this embodiment, ZooKeeper is configured as a collaborative status awareness middleware for a multi-datacenter scheduling center cluster. Each scheduling center 200 declares its online status by creating temporary nodes on it and listens to changes in other nodes to determine in real time whether the entire active-active cluster is in a fully available state, thereby providing a decision basis for task priority scheduling and automatic fault takeover.

[0087] This application also provides a terminal device, exemplary of which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to enable the terminal device to perform the functions of the various modules in the above-described task scheduling method or task scheduling system.

[0088] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0089] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.

[0090] This application also provides a computer-readable storage medium for storing the computer program used in the aforementioned terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0092] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0093] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0094] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A task scheduling method, characterized in that, Applicable to a task scheduling system, the task scheduling system includes: multiple scheduling centers, an executor cluster corresponding to each scheduling center, and a collaborative sensing module; Based on the sensing information of the collaborative sensing module and each associated scheduling center, the working mode is determined; If it is determined that the current scheduling center is in a multi-active mode, then the target task to be processed and the corresponding configured execution object are obtained from the task to be processed table; wherein, in the multi-active mode, the scheduling centers corresponding to all data centers are active; the execution object is the scheduling center that executes the corresponding task to be processed; If the execution object is determined to be the current scheduling center, then the current scheduling center is controlled to execute the target task to be processed; otherwise, based on the executor status of the execution object, a target execution object is determined between the current scheduling center and the execution object, and the target task to be processed is executed.

2. The task scheduling method according to claim 1, characterized in that, The sensing information includes node change information; The process of determining the working mode based on the sensing information of the collaborative sensing module and each associated scheduling center includes: The current scheduling center is controlled to listen for node change information. When the node change information is received, the temporary sequential node data is traversed to determine whether there is a corresponding temporary sequential node in all scheduling centers. If it exists, then the working mode is a multi-active mode; otherwise, it is a non-multi-active mode.

3. The task scheduling method according to claim 2, characterized in that, The method includes: When each of the scheduling centers starts up, it registers its own metadata with the collaborative perception module. The collaborative perception module generates a corresponding temporary sequence node based on each metadata and obtains the temporary sequence node data. If the collaborative sensing module detects a disconnection with any scheduling center, it controls the collaborative sensing module to delete the corresponding temporary sequential node from the temporary sequential node data and send node change information to each scheduling center.

4. The task scheduling method according to claim 1, characterized in that, The method further includes: If it is determined that the current scheduling center is in a non-active mode, then the current scheduling center is controlled to execute each of the pending tasks in the order configured in the pending task table.

5. The task scheduling method according to claim 1, characterized in that, The method further includes: Configure the task list to be processed using the following method: The first target scheduling center is designated as the execution target for the first type of pending tasks; The second category of tasks to be processed is divided according to the resource allocation and preset ratio strategy of each dispatch center.

6. The task scheduling method according to claim 1, characterized in that, The step of determining the target execution object from between the current scheduling center and the execution object and executing the target task to be processed based on the executor status of the execution object includes: If it is determined that the execution object configured in the pending task table is not the current scheduling center, determine whether there is an available executor in the target scheduling center configured for the target pending task; If it exists, the current scheduling center is controlled to skip the target task to be processed; If it does not exist, then control the current scheduling center to execute the target task to be processed.

7. The task scheduling method according to claim 6, characterized in that, The following method is used to detect whether there are available executors in each scheduling center: Traverse the set of executor addresses registered in the target scheduling center and check the tag attribute of each executor address; if the tag attribute does not belong to the target scheduling center, it is determined that there is no available executor in the target scheduling center.

8. The task scheduling method according to claim 1, characterized in that, The method further includes: Before each actuator registers its own information with the corresponding scheduling center, it determines the target scheduling center to which it belongs based on the IP segment information of the computer room where the actuator is located, and initiates registration information with the target scheduling center. The registration information includes the label information of the computer room.

9. The task scheduling method according to any one of claims 1-8, characterized in that, The method further includes: If the scheduling center corresponding to the computer room successfully acquires the database lock, then the scheduling center that successfully acquires the database lock within the preset scheduling time period is the current scheduling center.

10. A task scheduling system, characterized in that, The task scheduling system includes: a control module, a distributed task scheduling platform, and a collaborative sensing module; the distributed task scheduling platform includes multiple scheduling centers and an executor cluster. The control module is used to implement the task scheduling method as described in any one of claims 1-9.

Images