Multi-platform oriented task scheduling method, device and equipment

Abstract

The disclosure provides a multi-platform-oriented task scheduling method, device and equipment, relates to the technical field of computers, and particularly relates to the technical field of distributed clusters, resource management, task scheduling, cloud computing and the like. The specific implementation scheme is as follows: in any scheduling period, a plurality of platform corresponding schedulable task quantities are acquired, and a platform capacity snapshot is acquired; wherein the schedulable task quantity is used to indicate the number of target tasks that can be scheduled by the corresponding platform in the current scheduling period; for any platform, the total scheduling quota of the platform is determined according to the schedulable task quantity and the schedulable device quantity of the platform in the capacity snapshot; wherein the total scheduling quota is used to indicate the upper limit of the number of target tasks that can be pulled from the platform in the current scheduling period; the task execution unit is called to pull a first target task from the platform based on the validity period of the platform capacity snapshot, and the first target task does not exceed the total scheduling quota; the schedulable device corresponding to the platform is called to execute the first target task.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of distributed clusters, resource management, task scheduling, and cloud computing. In particular, it relates to a task scheduling method, apparatus, and device for multiple platforms. Background Technology

[0002] In the field of distributed task scheduling, related technologies typically rely on Worker units to complete the entire task scheduling and execution process. The Worker is the core working unit responsible for actively polling and retrieving tasks from the queue, and requesting corresponding device resources to complete task execution. It can be an independent process or thread instance and is the fundamental carrier in the distributed scheduling system responsible for computing power execution and resource adaptation. In scenarios with shared devices across multiple platforms, the industry generally adopts a resource pool sharing mechanism, registering the same device in resource pools on multiple platforms. During task execution, the Worker preempts device resources by locking devices and requesting resources, thereby achieving device reuse, reducing hardware deployment costs, and improving resource utilization.

[0003] Currently, distributed scheduling systems typically employ various static and semi-static scheduling strategies, such as a fixed number of workers, static resource pool partitioning, node resource label matching, task queue length limits, and global concurrency control, to constrain task scheduling and execution logic and ensure the basic stability of system operation. However, this type of scheduling and control scheme suffers from the limitation that the total number of global devices cannot accurately represent the actual available resources and execution capabilities of a single platform. In real-world business scenarios, it is common for a single platform to accumulate a large number of tasks awaiting execution, even though the platform currently has no idle devices. Because the system cannot accurately perceive the real-time resource status of the platform, it will still assign workers to repeatedly pick up tasks from that platform. Ultimately, due to insufficient device resources, task execution will fail and frequently rollback, greatly wasting system computing resources and reducing the overall execution efficiency and stability of distributed task scheduling. Summary of the Invention

[0004] This disclosure provides a task scheduling method, apparatus, and device for multiple platforms.

[0005] According to one aspect of this disclosure, a task scheduling method for multiple platforms is provided. The method includes: in any scheduling cycle, obtaining the schedulable task quantity corresponding to multiple platforms and obtaining a platform capacity snapshot; wherein the schedulable task quantity is used to indicate the number of target tasks that can be scheduled on the corresponding platform in the current scheduling cycle, and the platform capacity snapshot is generated based on the number of schedulable devices and schedulable devices corresponding to the multiple platforms in the current scheduling cycle, wherein the schedulable devices are devices among multiple execution devices that serve the corresponding platform and can be scheduled; for any platform, determining the total scheduling quota of the platform according to the schedulable task quantity and the number of schedulable devices corresponding to the platform; wherein the total scheduling quota is used to indicate the upper limit of the number of target tasks that can be pulled from the platform in the current scheduling cycle; invoking a task execution unit to pull a first target task from the platform, not exceeding the total scheduling quota, based on the validity period of the platform capacity snapshot; and invoking the schedulable devices corresponding to the platform to execute the first target task.

[0006] According to another aspect of this disclosure, a task scheduling apparatus for multiple platforms is provided. The apparatus includes: an acquisition module, configured to acquire the schedulable task quantity corresponding to multiple platforms and acquire a platform capacity snapshot in any scheduling cycle; wherein the schedulable task quantity is used to indicate the number of target tasks that can be scheduled on the corresponding platform in the current scheduling cycle, and the platform capacity snapshot is generated based on the number of schedulable devices and schedulable devices corresponding to the multiple platforms in the current scheduling cycle, and the schedulable devices are devices among multiple execution devices that serve the corresponding platform and can be scheduled; a first determination module, configured to determine the total scheduling quota of the platform for any given platform based on the schedulable task quantity and the number of schedulable devices corresponding to the platform; wherein the total scheduling quota is used to indicate the upper limit of the number of target tasks that can be pulled from the platform in the current scheduling cycle; a pulling module, configured to call a task execution unit to pull a first target task from the platform based on the validity period of the platform capacity snapshot, not exceeding the scheduling quota; and an execution module, configured to call the schedulable devices corresponding to the platform to execute the first target task.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the multi-platform task scheduling method proposed in the foregoing aspect of this disclosure.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to execute the multi-platform task scheduling method proposed in the foregoing aspect of this disclosure.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the multi-platform task scheduling method proposed in the foregoing aspect of this disclosure.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure; Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure; Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure; Figure 4 This is an architecture diagram of the multi-platform data acquisition task dynamic scheduling system provided in this disclosure; Figure 5 This is a schematic diagram according to the fourth embodiment of the present disclosure; Figure 6 This is a block diagram of an electronic device used to implement the multi-platform task scheduling method of the embodiments of this disclosure. Detailed Implementation

[0012] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] Currently, distributed scheduling systems have at least the following limitations: First, relying solely on the total number of devices cannot reflect the actual available resource capabilities of each platform. When a platform has a large backlog of tasks awaiting execution but no available devices, workers will continue to repeatedly accept tasks from that platform, ultimately causing task execution to be rolled back.

[0014] Second, in scenarios where devices are shared across multiple platforms, the same device may be repeatedly included in the resource pools of multiple platforms for statistical purposes, resulting in an inflated statistical count and causing the total number of configured Workers to exceed the actual concurrent capacity that the device can handle.

[0015] Third, task assignment and equipment application are two independent processes. The scheduling module cannot predict the platform resource status during the task assignment stage, making it difficult to avoid assigning tasks to platforms with no available capacity in advance.

[0016] Fourth, operations such as quota deduction, task dequeueing, and running status marking are separated. When the system malfunctions, inconsistencies may arise, such as quota being deducted but the task not starting, or a task being dequeued but not marked as running.

[0017] To address at least one of the aforementioned problems, this disclosure proposes a task scheduling method, apparatus, and device for multiple platforms.

[0018] Figure 1 The diagram is based on the first embodiment of this disclosure. It should be noted that the multi-platform task scheduling method of this disclosure can be applied to a multi-platform task scheduling device to realize the multi-platform task scheduling function.

[0019] The multi-platform task scheduling device can also be software within an electronic device, such as multi-platform task scheduling software. The following embodiments use a multi-platform task scheduling device as an example for illustration.

[0020] like Figure 1 As shown, this multi-platform task scheduling method may include the following steps: Step 101: In any scheduling cycle, obtain the schedulable task volume corresponding to multiple platforms and obtain a platform capacity snapshot.

[0021] The schedulable task quantity can be used to indicate the number of target tasks that the corresponding platform can schedule in the current scheduling cycle. It should be noted that this disclosure does not limit the target tasks; for example, they can be data acquisition tasks, etc. It should also be noted that the target tasks that the schedulable task quantity indicates the corresponding platform can schedule in the current scheduling cycle can include tasks that were originally in a pending execution state, as well as delayed (delayed_pending) tasks that have reached their execution time.

[0022] Among them, the platform capacity snapshot is generated based on the number of schedulable devices and the number of schedulable devices corresponding to multiple platforms in this scheduling cycle.

[0023] Among them, the schedulable device can be a device that serves the corresponding platform and is schedulable among multiple execution devices. For example, for platform a, the devices that serve the platform and are schedulable in this scheduling cycle include device 1, device 2, and device 3. It should be noted that the execution device can be a physical machine (such as a mobile terminal device, a real device, etc.), or a virtualized instance (such as a virtual machine, container, cloud phone instance, etc.), emulator, etc., and this disclosure does not impose any restrictions on it.

[0024] The number of schedulable devices can be used to indicate the number of schedulable devices corresponding to a given platform. As an example, suppose device 1 serves platform a and is schedulable, device 2 serves platform b and is schedulable, device 3 serves platform a and is not schedulable, and device 4 serves platform a and is schedulable. Then, the schedulable devices for platform a include device 1 and device 4, and the number of schedulable devices is 2. The schedulable devices for platform b include device 2, and the number of schedulable devices is 1.

[0025] It should also be noted that the execution device can be a device serving a single platform, i.e., a platform-dedicated device, or a device serving multiple platforms, i.e., a shared device.

[0026] In any of the embodiments disclosed herein, during any scheduling period, the target tasks to be processed in the task queue of any platform among multiple platforms can be statistically analyzed to obtain the number of schedulable tasks corresponding to any platform. Furthermore, a platform capacity snapshot can be generated based on the number of schedulable devices and the number of schedulable devices corresponding to each platform in this scheduling period.

[0027] Optionally, in some embodiments, the platform capacity snapshot may have corresponding attribute information, which may include validity period, generation time, expiration time, snapshot identifier, etc. The snapshot identifier can be used to uniquely identify the corresponding platform capacity snapshot.

[0028] It should also be noted that this disclosure does not limit the number of schedulable devices (i.e., the number of schedulable devices) for any platform; it can be one or more.

[0029] Step 102: For any platform, determine the total scheduling quota of the platform based on the number of schedulable tasks and the number of schedulable devices corresponding to the platform.

[0030] The total scheduling quota (also known as the total claim quota) can be used to indicate the upper limit of the number of target tasks that can be pulled from the platform in this scheduling cycle.

[0031] As one possible implementation, for any platform, the minimum of the schedulable task quantity and the schedulable device quantity corresponding to the platform can be determined as the platform's total scheduling quota. For example, for platform a, assuming the schedulable task quantity is m and the schedulable device quantity is n, the total scheduling quota 'quota' for platform a can be determined according to the following formula: quota=min(m,n);(1) Here, min() is the function that takes the minimum value.

[0032] Step 103: Invoke the task execution unit to pull the first target task from the platform, which does not exceed the total scheduling quota, based on the validity period of the platform capacity snapshot.

[0033] The task execution unit can be used to pull tasks from the platform, and can be a process or thread instance, such as a Worker unit in a distributed system.

[0034] As an example, in response to determining that a platform capacity snapshot is valid, a task execution unit can be invoked to pull a first target task from the platform, not exceeding the total scheduling quota. It should be noted that this disclosure does not limit the number of task execution units invoked; there may be one or more.

[0035] Step 104: Invoke the schedulable device corresponding to the platform to execute the first target task.

[0036] As an example, the task execution unit can be controlled to invoke a schedulable device serving the platform and execute a first target task through that schedulable device. For example, for any platform, the schedulable devices serving the platform include device 1 and device 3, then the task execution unit can invoke device 1 or device 3 to execute the first target task retrieved by the task execution unit.

[0037] As an application scenario, the multi-platform task scheduling method disclosed herein can be applied to multi-platform mobile applications, cloud phones, real devices or emulator automated data collection systems. This system can match and schedule batch problems, multiple target platforms and multiple execution devices. The execution devices can include dedicated devices that serve only a single platform or shared devices that can serve multiple platforms.

[0038] The multi-platform task scheduling method of this disclosure involves obtaining the schedulable task quantity for multiple platforms and acquiring a platform capacity snapshot during any scheduling cycle. The schedulable task quantity indicates the number of target tasks that can be scheduled on the corresponding platform during the current scheduling cycle. The platform capacity snapshot is generated based on the schedulable device quantity and schedulable devices for multiple platforms during the current scheduling cycle. A schedulable device is a device among multiple execution devices that serves the corresponding platform and can be scheduled. For any platform, a total scheduling quota is determined based on the schedulable task quantity and schedulable device quantity. The total scheduling quota indicates the upper limit of the number of target tasks that can be retrieved from the platform during the current scheduling cycle. A task execution unit is invoked to retrieve a first target task from the platform, not exceeding the total scheduling quota, based on the validity period of the platform capacity snapshot. The schedulable device corresponding to the platform is then invoked to execute the first target task. Therefore, by constructing a platform capacity snapshot based on the real-time schedulable device quantity of each platform in each scheduling cycle, and combining it with the platform's schedulable task quantity to determine the total scheduling quota of each platform in a single cycle, dynamic matching between task scheduling quantity and the platform's actual device concurrency capability can be achieved. At the same time, by strongly binding the task retrieval process with the validity period of the platform capacity snapshot, expired or outdated platform capacity status data can be effectively avoided for task scheduling operations. This effectively avoids invalid scheduling and repeated task rollback from the root, improves the accuracy and stability of task scheduling in multi-platform and multi-device scenarios, increases the resource utilization of devices, and reduces invalid scheduling overhead and computing power waste.

[0039] like Figure 2 As shown, this disclosure also proposes a task scheduling method for multiple platforms. Figure 2 As illustrated in the schematic diagram of the second embodiment of this disclosure, the multi-platform task scheduling method may include the following steps: Step 201: In any scheduling cycle, obtain the number of schedulable tasks corresponding to multiple platforms.

[0040] The number of schedulable tasks can be used to indicate the number of target tasks that the platform can schedule in the current scheduling cycle.

[0041] It should be noted that the execution process of step 201 can refer to the execution process of any embodiment of this disclosure, and will not be repeated here.

[0042] Step 202: Obtain device profiles for multiple execution devices.

[0043] Optionally, in some embodiments, the device profile may include device-level status information, device capability information, and platform-level status information of any executing device in the current scheduling cycle.

[0044] Among them, device-level status information can be used to indicate the device-level status of the corresponding execution device. Device-level status can include, but is not limited to: idle status, busy status (or occupied status), unhealthy status, draining status, etc.

[0045] Among them, equipment capability information can be used to indicate the platforms that the corresponding execution equipment can support or serve.

[0046] Platform-level status information can be used to indicate the operating status of execution devices under the corresponding platform. For example, it can include blocked status, normal status (or non-blocking status), etc. For instance, execution device 1 serves platform a and platform b. However, the operating status of execution device 1 under platform a is blocked, while the operating status of execution device 1 under platform b is non-blocking.

[0047] Step 203: Based on the device profile, assign multiple execution devices to the corresponding platforms.

[0048] As one possible implementation, for any execution device, the target platform served by the execution device is determined from multiple platforms based on the device capability information of the execution device; based on the device-level status information and platform-level status information of the execution device in the current scheduling cycle, it is determined whether the execution device meets the allocation conditions with respect to the target platform; in response to the execution device meeting the allocation conditions with respect to the target platform, the execution device is allocated to the target platform as a schedulable device corresponding to the target platform.

[0049] In the embodiments of this disclosure, for any execution device, the platform served by the execution device can be determined from multiple platforms based on the device capability information of the execution device, i.e., the platforms supported or served by the device, and that platform can be used as the target platform. It should be noted that this disclosure does not limit the number of target platforms.

[0050] The allocation conditions can be preset.

[0051] Optionally, in some embodiments, the allocation conditions may include any of the following: the device-level status information of the execution device in the current scheduling cycle is a first preset state (e.g., idle state); the platform-level status information of the execution device regarding the target platform in the current scheduling cycle is a second preset state (e.g., non-blocking state).

[0052] For example, for any execution device, assuming that the target platform corresponding to the execution device includes platform a and platform b, the device-level status information of the execution device in the current scheduling cycle indicates that the execution device is in an idle state, and the platform-level status information of the execution device with respect to platform a indicates that the execution device's running state on platform a is a blocked state, then it is determined that the execution device does not meet the allocation conditions with respect to platform a; assuming that the platform-level status information of the execution device with respect to platform b indicates that the execution device's running state on platform b is a non-blocking state, then it is determined that the execution device meets the allocation conditions with respect to platform b.

[0053] Therefore, in this embodiment of the disclosure, when the execution device meets the allocation conditions with respect to the target platform, the execution device can be allocated to the target platform as a schedulable device corresponding to the target platform.

[0054] Therefore, by first determining the target platforms that the execution devices can serve based on their device capability information, and then combining the device-level status information and platform-level status information of the execution devices, the dual status verification and allocation conditions of the devices can be completed. This enables accurate allocation of execution devices, effectively avoids problems such as resource mismatch, device idleness or platform overload, maintains the compliance and availability of schedulable devices on each platform, and further improves the orderliness and operational reliability of multi-platform task scheduling.

[0055] As one possible implementation, a first target platform is determined from multiple target platforms based on at least one of the following: the current remaining task volume, platform priority, service level agreement priority, and historical service volume; and the device is assigned to the first target platform as a schedulable device corresponding to the first target platform.

[0056] The current remaining task quantity can be used to indicate the number of tasks that remain to be executed and not assigned on the corresponding target platform within the current scheduling period.

[0057] Platform priority can be used to indicate the importance level of a corresponding platform.

[0058] The Service Level Agreement (SLA) priority can be a dynamic priority based on the commitments or agreements made in the SLA.

[0059] Historical service volume can be used to record quantitative data on the number of times the execution device provided task execution services to the corresponding target platform within a historical time period. For example, it can indicate the number of times the execution device provided task execution services to the target platform in the past month, or the duration of task execution services provided by the execution device to the target platform in the past year, etc. It should be noted that this disclosure does not limit the length or range of the historical time period, nor does it limit the form of the quantitative data, which can be the number of task executions, total task time, data processing volume, etc.

[0060] As an example, we can prioritize the platform with the largest number of remaining tasks, that is, determine the platform with the largest number of remaining tasks as the first target platform.

[0061] As another example, the priority can be allocated according to the pre-configured platform priority weight ratio. For example, if the priority weight of platform A and platform B is 2:1, then the execution device can be allocated with a probability of approximately 2:1.

[0062] As an example, a fair strategy could be adopted, which involves polling each target platform based on the number of times it has historically acquired the execution device, or prioritizing the platform with the least historical service volume, to obtain the first target platform, in order to prevent a particular target platform from occupying the execution device for an extended period.

[0063] As another example, based on SLA priority, the target platform with higher SLA commitments (such as expedited tasks) is designated as the first target platform.

[0064] It should be noted that the above method for determining the first target platform is merely exemplary. In practical applications, other methods may also be used, and this disclosure does not impose any restrictions on them.

[0065] Therefore, it is possible to comprehensively consider multiple target platforms based on various indicators such as the remaining task volume, platform priority, service level agreement priority, and historical service volume. The execution equipment is then assigned to the selected first target platform, improving the rationality and effectiveness of the allocation of execution equipment. This approach balances business needs with resource balance, rationally allocates resources, avoids platform task backlog and load imbalance, maintains the orderly execution of various businesses according to priority, and enhances the rationality, fairness, and overall operational efficiency of multi-platform task scheduling.

[0066] Step 204: In response to the completion of the allocation of multiple execution devices, the schedulable devices corresponding to multiple platforms are counted to obtain the number of schedulable devices and the number of schedulable devices for any platform in this scheduling cycle.

[0067] It should be noted that the explanation of schedulable devices and the quantity of schedulable devices in step 101 also applies to this embodiment, and will not be repeated here.

[0068] Step 205: Generate a platform capacity snapshot based on the number of schedulable devices and the number of schedulable devices corresponding to multiple platforms in this scheduling cycle.

[0069] Step 206: For any platform, determine the total scheduling quota of the platform based on the number of schedulable tasks and the number of schedulable devices corresponding to the platform.

[0070] The total scheduling quota indicates the upper limit of the number of target tasks that can be pulled from the platform during the current scheduling cycle.

[0071] It should be noted that the execution process of steps 205 to 206 can refer to the execution process of any embodiment of this disclosure, and is not limited here.

[0072] As one possible implementation, for any platform, the scheduling quota can be obtained by taking the minimum value between the number of schedulable tasks and the number of schedulable devices for that platform. Therefore, by determining the platform's scheduling quota by taking the minimum value between the number of schedulable tasks and the number of schedulable devices, the upper limit of single-time scheduling for the platform can be accurately determined, avoiding task overflow or waste of equipment resources. This enables dynamic matching of tasks and equipment resources, helps maintain the orderly operation of scheduling work on various platforms, and improves overall resource utilization efficiency and scheduling control accuracy.

[0073] Step 207: Invoke the task execution unit to pull the first target task from the platform, which does not exceed the total scheduling quota, based on the validity period of the platform capacity snapshot.

[0074] It should be noted that the execution process of step 207 can refer to the execution process of any embodiment of this disclosure, and is not limited here.

[0075] In one possible implementation of this disclosure, the total expected scheduling volume is determined based on the total scheduling quota corresponding to multiple platforms; and the task execution units used to retrieve the target tasks in the current scheduling cycle are scaled up or down based on the total expected scheduling volume.

[0076] The expected total demand can be used to indicate the total number of task execution units that need to be scheduled in the current scheduling cycle.

[0077] As an example, assuming there are n platforms and the total scheduling quota corresponding to the i-th platform is quota[i], i∈[1,n], the expected total scheduling demand w can be determined according to the following formula: (2) Furthermore, based on the expected total scheduling demand and the actual number of currently active task execution units, the task execution units used to retrieve the target tasks in this scheduling cycle can be scaled up or down.

[0078] In one possible implementation of this disclosure, the target scheduling quantity is obtained by taking the minimum value between the expected total scheduling demand and the preset maximum scheduling quantity; the actual number of task execution units actually running in the current scheduling cycle is obtained; in response to the target scheduling quantity being greater than the actual quantity, a number of new task execution units matching the difference between the target scheduling quantity and the actual quantity are created; in response to the target scheduling quantity being less than the actual quantity, a number of task execution units matching the difference between the target scheduling quantity and the actual quantity are released.

[0079] The preset maximum scheduling quantity can be a pre-defined upper limit for the number of task execution units that can be scheduled.

[0080] In this disclosure, there are no restrictions on the form of the difference between the target scheduling quantity and the actual quantity; it can be the original value or the absolute value of the difference.

[0081] As an example, the target scheduling quantity is obtained by taking the minimum value between the expected total scheduling demand and the preset maximum scheduling quantity. Then, the actual number of task execution units actually running in the current scheduling cycle is obtained. When the target scheduling quantity is greater than the actual quantity, the current number of task execution units cannot meet the scheduling demand, and a corresponding number of task execution units need to be added. When the target scheduling quantity is less than the actual quantity, it means that the current number of actually running task execution units is sufficient to cover the scheduling demand, and the excess task execution units are idle. These redundant idle execution units can be released, achieving dynamic adaptation of the number of task execution units. Therefore, when the target scheduling quantity is greater than the actual quantity, a number of new task execution units matching the difference between the target and actual quantities are created. When the target scheduling quantity is less than the actual quantity, a number of task execution units matching the difference between the target and actual quantities are released. When the target scheduling quantity equals the actual quantity, no scaling is required.

[0082] In summary, by leveraging the scheduling quotas of each platform to calculate the total expected scheduling demand, and dynamically scaling up or down the task execution units accordingly, the size of the execution units can be precisely matched with the actual task requirements. This avoids resource redundancy and idleness, prevents task backlog due to insufficient computing power, effectively reduces scheduling costs, and improves overall task execution efficiency and system elasticity. Therefore, by comparing the expected total scheduling demand with the preset maximum scheduling quantity to effectively determine the target scheduling quantity, and then adjusting resources based on the actual number of currently running task execution units, the scheduling scale can be strictly controlled. Task execution units can be created and released on demand, effectively avoiding the risk of resource overruns and continuously maintaining a balance between the number of task execution units and the business load.

[0083] In summary, by comprehensively considering the total scheduling quota of each platform to effectively determine the number of execution units to be scheduled, and then dynamically scaling up or down the task execution units accordingly, the scale of the task execution units can be precisely matched with the actual task requirements. This can effectively avoid resource redundancy and idleness, prevent task backlog caused by insufficient computing power, effectively reduce scheduling costs, and improve the overall execution efficiency and operational stability of tasks.

[0084] Step 208: Invoke the schedulable device corresponding to the platform to execute the first target task.

[0085] In one possible implementation of this disclosure, the task execution unit requests a device lease from the schedulable devices of the platform; in response to the successful application for the device lease, the schedulable device with the successful lease is bound to the first target task, and the schedulable device with the successful lease executes the bound first target task.

[0086] As an example, the task execution unit can apply for a device lease from the schedulable devices recorded by the platform. If the device lease application is successful, the task execution unit can establish a binding relationship between the successfully leased schedulable device and the first target task, and then call the successfully leased schedulable device to execute the first target task with which it has established a binding relationship.

[0087] Therefore, by completing the equipment application through the equipment lease mechanism and binding the successfully applied equipment with the task for execution, the exclusive use of equipment resources can be achieved, avoiding conflicts caused by multiple tasks competing for resources, and helping to maintain the orderliness and reliability of task execution.

[0088] In one possible implementation of this disclosure, in response to a failed device lease application, the first target task can be returned to the platform's task queue. Therefore, if a device lease application fails, the first target task is returned to the platform's task queue, enabling reasonable task rollback and retries, preventing abnormal task termination, and allowing tasks to wait for idle device resources before being rescheduled for execution, thus improving the integrity and fault tolerance of the overall scheduling process.

[0089] This disclosure discloses a multi-platform task scheduling method that acquires device profiles for multiple execution devices; allocates these devices to corresponding platforms based on the device profiles; in response to the allocation of these devices, it statistically analyzes the schedulable devices for each platform to obtain the number and number of schedulable devices for that platform in the current scheduling cycle; and generates a platform capacity snapshot based on the number and number of schedulable devices for each platform in the current scheduling cycle. Therefore, by leveraging the device profiles of the execution devices, the method can accurately allocate them to corresponding platforms, thereby achieving the acquisition of platform capacity snapshots. This improves the accuracy, adaptability, and reliability of platform capacity snapshots, enabling a true reflection of the actual resource carrying capacity of each platform. This facilitates subsequent task scheduling and resource scheduling, enhancing the efficiency, stability, and resource utilization of multi-platform task scheduling.

[0090] like Figure 3 As shown, this disclosure also proposes a task scheduling method for multiple platforms. Figure 3 The diagram is based on the third embodiment of this disclosure. Based on any of the above embodiments of this disclosure, the multi-platform task scheduling method may further include the following steps: Step 301: Perform at least one round of iterative processing on the platform. In any round of iterative processing, determine whether the current moment is within the validity period corresponding to the platform capacity snapshot, and determine whether the remaining scheduling quota of the platform in this round of iterative processing is greater than the set threshold.

[0091] The threshold can be preset, such as 0, etc., and this disclosure does not restrict its value.

[0092] The remaining scheduling quota can be used to indicate the upper limit of the number of target tasks that the corresponding platform can pull from the platform during the corresponding round of processing.

[0093] It should be noted that the number of rounds of processing performed on different platforms may be the same or different, and this disclosure does not impose any restrictions on this.

[0094] Step 302: If the current time is within the validity period of the platform capacity snapshot and the remaining scheduling quota is greater than the set threshold, the task execution unit in this round of iteration is called to perform a dequeue operation on the target task in the platform's task queue to obtain the first target task in this round of iteration.

[0095] It should be noted that, for any platform, during at least one round of processing on that platform, the task execution units called in different rounds of processing may be the same or different, and this disclosure does not impose any restrictions on this.

[0096] As one possible implementation, if the current time is not within the validity period corresponding to the platform capacity snapshot, or if the remaining scheduling quota of the platform in this round of iteration is not greater than the set threshold, then the current round and subsequent iterations of processing on the platform will be stopped.

[0097] Step 303: Based on the number of the first target tasks in this round of iterative processing, the remaining scheduling quota of the platform in this round of iterative processing is deducted to obtain the remaining scheduling quota of the platform in the next round of processing.

[0098] It should be noted that during this scheduling cycle, the total number of target tasks dispatched during at least one round of processing on the platform shall not exceed the total scheduling quota corresponding to the platform.

[0099] As an example, assuming the number of first target tasks dequeued in this iteration is k, the remaining scheduling quota of the platform in this iteration is deducted, and the remaining scheduling quota of the platform in the next iteration can be obtained according to the following formula: w'=wk; (3) Where w is the remaining scheduling quota of the platform in this round of iteration; w' is the remaining scheduling quota of the platform in the next round of processing.

[0100] It is understandable that after obtaining the first target task in this round of iterative processing, the first target task can be verified. Therefore, in one possible implementation of this disclosure embodiment, the first target task can be verified to obtain a verification result; in response to the verification result indicating that the first target task does not exist, or that the current state of the first target task is canceled or completed, the first target task is discarded, and no quota deduction is performed on the remaining scheduling quota of the platform in this round of iterative processing.

[0101] As an example, after obtaining the first objective task of this round of iterative processing, if the first objective task is any of the following: 1. The primary objective does not exist; 2. The primary objective task is neither in a pending execution state nor in a delayed execution state; 3. The primary objective task is either cancelled or completed. This indicates that the first target task is dirty member data, and the first target task can be discarded. The remaining scheduling quota of the platform in this round of iteration will not be deducted.

[0102] Therefore, after a task is dequeued, it can be verified. Invalid, canceled, or completed tasks can be discarded directly while retaining the original scheduling quota. This can prevent invalid tasks from occupying resources and quotas, maintain the statistical accuracy of scheduling quotas, reduce useless scheduling behavior, and improve the overall resource utilization efficiency and the rigor of the scheduling process.

[0103] In one possible implementation of the embodiments of this disclosure, at least one of the following processes can also be performed in an atomic operation manner: performing state update processing on the first target task of the current iteration process; recording the retrieval timestamp of the first target task of the current iteration process and the snapshot identifier of the platform capacity snapshot.

[0104] For example, while deducting the remaining scheduling quota from the platform during this iteration, the status of the first target task in this iteration can be updated to running using atomic operations. The claimed_at, heartbeat_at, and snapshot_id fields corresponding to the first target task are recorded. The claimed_at field records the timestamp when the first target task was claimed (entering running), used for calculating waiting time and timeout determination. The heartbeat_at field records the time of the last reported live heartbeat during the Worker's (referred to as the task execution unit in this disclosure) execution, allowing the scheduler to determine if the first target task is still executing normally (a long period without updates can be considered abnormal and trigger a rollback). The snapshot_id field records which capacity snapshot the first target task was claimed under, used for subsequent reconciliation, auditing, and abnormal rollback to determine which processing round the first target task should belong to. Therefore, by performing tasks such as updating task status, retrieving timestamps, and recording snapshot identifiers through atomic operations, the integrity and consistency of related data operations can be maintained, avoiding data corruption and information loss in concurrent scenarios, and enabling accurate traceability of task flow and scheduling information.

[0105] In one possible implementation of this disclosure, if the remaining scheduling quota is successfully deducted during the current iteration process but the dequeue operation on the platform's task queue fails, a rollback operation can be performed on the platform's remaining scheduling quota. Similarly, if the remaining scheduling quota is successfully deducted during the current iteration process, the dequeue operation on the platform's task queue succeeds, but the verification of the status of the first target task obtained based on the dequeue operation fails, a rollback operation can be performed on the platform's remaining scheduling quota. This allows for timely correction of scheduling data, avoids unnecessary loss of scheduling quota, maintains the accuracy of the remaining scheduling quota value, effectively prevents data deviations and resource statistics errors in the scheduling process, and improves the rigor and fault tolerance of the overall scheduling mechanism.

[0106] The task scheduling method for multiple platforms disclosed in this embodiment involves executing at least one round of processing on the platform. In any round of processing, it is determined whether the current moment is within the validity period corresponding to the platform capacity snapshot, and whether the remaining scheduling quota of the platform in this round of iteration is greater than a set threshold. If the current moment is within the validity period corresponding to the platform capacity snapshot and the remaining scheduling quota is greater than the set threshold, the task execution unit in this round of iteration is invoked to perform a dequeue operation on the target task in the platform's task queue to obtain the first target task of this round of iteration. Based on the number of the first target tasks in this round of iteration, the remaining scheduling quota of the platform in this round of iteration is deducted to obtain the remaining scheduling quota of the platform in the next round of processing. The total number of tasks dequeued cumulatively in at least one round of processing does not exceed the total scheduling quota. Therefore, the timing of target task dequeueing can be controlled by relying on the validity period of the platform capacity snapshot and the remaining scheduling quota. The quota can be deducted in real time and the total number of tasks dequeued can be limited. This can strictly control the scale of platform task scheduling, prevent over-quota scheduling, maintain the controllability and stability of task scheduling, and realize the dynamic updating and cyclic reuse of scheduling quota, thereby improving the orderliness of multi-platform task scheduling and the accuracy of resource management.

[0107] To clearly illustrate the multi-platform task scheduling method disclosed herein, the following explanation is provided with examples.

[0108] As an example, the multi-platform task scheduling method disclosed herein is applied to a multi-platform data acquisition task dynamic scheduling system. Figure 4 An architecture diagram of a multi-platform data acquisition task dynamic scheduling system, such as... Figure 4 As shown, the multi-platform data acquisition task dynamic scheduling system 400 includes a task queue module 410, a device pool module 420, a capacity snapshot generation module 430, a platform quota management module 440, a worker (referred to as a task execution unit in this disclosure) dynamic management module 450, several workers 460, an atomic task acquisition module 470, and a device leasing module 480, wherein: The task queue module 410 is used to count the number of tasks to be collected on each platform and obtain the amount of tasks to be processed on each platform (referred to as the schedulable task amount in this disclosure). It should be noted that the statistical objects can include pending tasks (referred to as tasks in the pending execution state in this disclosure) and delayed_pending tasks that have reached their execution time (referred to as delayed execution tasks in this disclosure).

[0109] The device pool module 420 is used to read the current device-level status (referred to as device-level status information in this disclosure), the platform capabilities supported by the device (referred to as device capability information in this disclosure), and the platform-level status at the device-platform dimension (referred to as platform-level status information in this disclosure); wherein, the device-level status includes idle status, busy status, abnormal status, and empty status.

[0110] The capacity snapshot generation module 430 is used to determine, for any given device, whether it can access a platform for its services based on the device's current device-level status, the platform capabilities it supports, and the platform-level status at the device-platform dimension. For dedicated devices, if it is determined that the device can access a platform for its services, it can be included in the platform capacity of the corresponding platform. For shared devices, only one target platform is allowed to be included in the capacity snapshot. Optionally, the target platform can be selected based on the remaining number of tasks on the supported platform (referred to as the current remaining task quantity in this disclosure), the platform weight (referred to as the platform priority in this disclosure), the fairness policy, or the SLA priority, to serve as the schedulable device corresponding to the target platform. After multiple devices are allocated, a capacity snapshot (referred to as the platform capacity snapshot in this disclosure) containing a snapshot identifier, validity period, and platform capacity is generated. The platform capacity can be used to record the number of schedulable devices and the number of schedulable devices for each platform.

[0111] The platform quota management module 440 is used to obtain the quota (referred to as the total scheduling quota in this disclosure) for any platform by taking the minimum value from the schedulable task quantity and schedulable equipment quantity corresponding to that platform.

[0112] The Worker dynamic management module 450 is used to sum the claim amounts corresponding to multiple platforms to obtain the expected number of Workers (referred to as the expected total demand scheduling in this disclosure), and to expand or shrink the system based on the maximum number of Workers (referred to as the preset maximum scheduling number in this disclosure) and the actual number of Workers (referred to as the actual number in this disclosure).

[0113] Worker460 is used to read the current snapshot identifier and call the atomic task acquisition module 470 during the process of acquiring acquisition tasks on the corresponding platform.

[0114] The atomic task claiming module 470 is used to complete the following steps in the same atomic process: verify whether the capacity snapshot indicated by the current snapshot identifier is within the validity period, check the current remaining claiming quota of the platform (referred to as the remaining scheduling quota in this disclosure), deduct the quota from the current remaining claiming quota, perform a dequeue operation on the task queue, mark the status of the dequeued acquisition task as running, and write the claimed_at field, heartbeat_at field and snapshot_id field.

[0115] Worker460 can also be used to invoke the device lease module 480. The device lease module 480 is used to apply for a device lease from the schedulable devices of the platform to which the dequeued collection task belongs, based on the platform's availability as recorded in the capacity snapshot. If the lease is successful, the collection task is executed using the successfully leased schedulable device. If the lease fails, the dequeued collection task is returned to the corresponding queue (e.g., to the pending execution queue or the delayed execution queue). It should be noted that the consumed quota in this round is not immediately refunded but is recalculated in the next snapshot.

[0116] It should also be noted that after a new capacity snapshot is generated, the remaining claimable amount under the old snapshot will automatically become invalid or be overwritten, and the Worker shall not continue to consume the remaining claimable amount corresponding to the old snapshot.

[0117] Optionally, the atomic task acquisition module 470 can also be used to: not perform quota deduction processing or dequeue operation on the task queue when the capacity snapshot indicated by the snapshot identifier is not within its validity period; not perform dequeue operation on the task queue when the remaining domain quota of the platform is 0; not modify the status of the dequeued collection task before its corresponding execution time, i.e., not modify it to running; clean up dirty members of the queue when the dequeued collection task does not exist or its status is not pending / delayed_pending, but do not perform quota deduction processing; roll back the remaining domain quota when quota deduction processing has been successfully performed but dequeueing has failed; and roll back the remaining domain quota and avoid marking it as running when the status verification of the dequeued collection task fails after dequeueing.

[0118] As one possible implementation, the multi-platform data acquisition task dynamic scheduling system disclosed herein can be applied to the operation and maintenance backend, and at least one of the following can be displayed in the operation and maintenance backend or the scheduling and diagnostic interface: The current snapshot identifier, snapshot generation time and validity period; the number of tasks to be processed on each platform, platform capacity, claim quota, and actual number of Workers; the platform affiliation of shared devices in this snapshot; the reason for platform capacity being 0; log information on Worker consumption quotas, task claiming and lease application, etc.

[0119] In summary, the multi-platform data acquisition task dynamic scheduling system disclosed herein can move the platform capacity calculation from the Worker execution phase to the periodic scheduling loop. The scheduling loop generates a platform capacity snapshot with a snapshot identifier and writes the task claim quota under the current capacity snapshot to each platform. The Worker is only allowed to claim tasks after consuming the platform quota under the current capacity snapshot.

[0120] The multi-platform data acquisition task dynamic scheduling system disclosed herein has at least the following advantages: 1. Workers only accept tasks with actual platform capacity, effectively reducing invalid task acceptance and repeated task re-queuing.

[0121] 2. Shared devices are only counted by a single platform in the same capacity snapshot, eliminating the problem of artificially inflated concurrency capacity.

[0122] 3. The number of Workers is determined by the number of tasks to be collected on the platform and the available capacity of the platform, which improves the accuracy of service scaling up and down.

[0123] 4. Use snapshot identifiers to distinguish different scheduling rounds to prevent quota data from interfering with each other between different rounds.

[0124] 5. An atomic retrieval mechanism is adopted to ensure the consistency of data among platform quotas, task queues, and running status.

[0125] To implement the above embodiments, this disclosure also provides a task scheduling device for multiple platforms. For example... Figure 5 As shown, Figure 5 This is a schematic diagram according to the fifth embodiment of the present disclosure. The multi-platform task scheduling device 500 may include: an acquisition module 510, a first determination module 520, a pull module 530, and an execution module 540.

[0126] The acquisition module 510 is used to acquire the number of schedulable tasks corresponding to multiple platforms and acquire a platform capacity snapshot in any scheduling cycle. The number of schedulable tasks is used to indicate the number of target tasks that can be scheduled on the corresponding platform in the current scheduling cycle. The platform capacity snapshot is generated based on the number of schedulable devices and schedulable devices corresponding to multiple platforms in the current scheduling cycle. The schedulable devices are the devices among multiple execution devices that serve the corresponding platform and can be scheduled.

[0127] The first determining module 520 is used to determine the total scheduling quota of any platform based on the schedulable task volume and schedulable device volume of the platform; wherein, the total scheduling quota is used to indicate the upper limit of the number of target tasks that can be pulled from the platform in the current scheduling cycle.

[0128] The pull module 530 is used to call the task execution unit to pull the first target task from the platform based on the validity period of the platform capacity snapshot. This task does not exceed the scheduling quota.

[0129] The execution module 540 is used to call the schedulable device corresponding to the platform to execute the first target task.

[0130] In one possible implementation of this disclosure, the acquisition module 510 is configured to: acquire device profiles of multiple execution devices; allocate the multiple execution devices to corresponding platforms according to the device profiles; in response to the completion of the allocation of multiple execution devices, count the schedulable devices corresponding to the multiple platforms to obtain the number of schedulable devices and the number of schedulable devices for any platform in the current scheduling cycle; and generate a platform capacity snapshot based on the number of schedulable devices and the number of schedulable devices corresponding to the multiple platforms in the current scheduling cycle.

[0131] In one possible implementation of this disclosure, the device profile includes device-level status information, device capability information, and platform-level status information of any device in the current scheduling cycle; the acquisition module 510 is configured to: for any execution device, determine the target platform served by the execution device from multiple platforms based on the device capability information of the execution device; determine whether the execution device meets the allocation conditions with respect to the target platform based on the device-level status information and platform-level status information of the execution device in the current scheduling cycle; and in response to the execution device meeting the allocation conditions with respect to the target platform, allocate the execution device to the target platform as a schedulable device corresponding to the target platform.

[0132] In one possible implementation of this disclosure, there are multiple target platforms. The acquisition module 510 is used to: determine a first target platform from the multiple target platforms based on at least one of the current remaining task volume, platform priority, service level agreement priority, and historical service volume corresponding to the multiple target platforms; wherein, the historical service volume is used to record quantitative data of the execution device providing task execution services to the corresponding target platform within a historical time period; and allocate the device to the first target platform as a schedulable device corresponding to the first target platform.

[0133] In one possible implementation of this disclosure, the multi-platform task scheduling device 500 may further include: The determination module is used to determine the expected total scheduling demand based on the total scheduling quota corresponding to multiple platforms; wherein, the expected total scheduling demand is used to indicate the total number of task execution units that need to be scheduled in this round of scheduling cycle; The scaling module is used to scale up or down the task execution units used to pull target tasks in the current scheduling cycle according to the expected total scheduling volume.

[0134] In one possible implementation of this disclosure, the scaling module is configured to: obtain a target scheduling quantity by taking the minimum value between the expected total scheduling demand and the preset maximum scheduling quantity; wherein the preset maximum scheduling quantity is used to indicate the maximum number of schedulable task execution units; obtain the actual number of task execution units actually running in the current scheduling cycle; in response to the target scheduling quantity being greater than the actual quantity, create a number of new task execution units matching the difference between the target scheduling quantity and the actual quantity; and in response to the target scheduling quantity being less than the actual quantity, release a number of task execution units matching the difference between the target scheduling quantity and the actual quantity.

[0135] In one possible implementation of this disclosure, the pull module 530 is configured to: perform at least one round of iterative processing on the platform, wherein, during any round of iterative processing, it is determined whether the current moment is within the validity period corresponding to the platform capacity snapshot, and whether the remaining scheduling quota of the platform in this round of iterative processing is greater than a set threshold; if the current moment is within the validity period corresponding to the platform capacity snapshot and the remaining scheduling quota is greater than the set threshold, the task execution unit in this round of iterative processing is invoked to perform a dequeue operation on the target task in the platform's task queue to obtain the first target task in this round of iterative processing; based on the number of the first target task in this round of iterative processing, the remaining scheduling quota of the platform in this round of iterative processing is deducted to obtain the remaining scheduling quota of the platform in the next round of processing; wherein, the total number of tasks dequeued in at least one round of processing does not exceed the total scheduling quota.

[0136] In one possible implementation of this disclosure, the multi-platform task scheduling device 500 may further include: The rollback module is used to: perform a rollback operation on the platform's remaining scheduling quota in response to a successful deduction of the remaining scheduling quota during the current iteration process and a failure to perform a dequeue operation on the platform's task queue; and perform a rollback operation on the platform's remaining scheduling quota in response to a successful deduction of the remaining scheduling quota during the current iteration process, a successful dequeue operation on the platform's task queue, and a failure to verify the status of the first target task obtained based on the dequeue operation.

[0137] In one possible implementation of this disclosure, the multi-platform task scheduling device 500 may further include: The verification module is used to verify the first target task in order to obtain the verification result.

[0138] The processing module is used to discard the first target task in response to the verification result indicating that the first target task does not exist, or the current state of the first target task is canceled or completed, and not to perform quota deduction processing on the remaining scheduling quota of the platform in this round of iteration processing.

[0139] In one possible implementation of this disclosure, the execution module 540 is configured to: call the task execution unit to apply for a device lease from the schedulable devices of the platform; in response to the successful application for the device lease, bind the successfully leased schedulable device to the first target task, and have the successfully leased schedulable device execute the bound first target task.

[0140] In one possible implementation of this disclosure, the multi-platform task scheduling device 500 may further include: a return module, used to return the first target task to the platform's task queue in response to a failed device lease application.

[0141] In one possible implementation of this disclosure, the first determining module 520 is configured to: for any platform, take the minimum value from the number of schedulable tasks and the number of schedulable devices corresponding to the platform to obtain the platform's scheduling quota.

[0142] The multi-platform task scheduling device of this disclosure acquires the schedulable task quantity corresponding to multiple platforms and obtains a platform capacity snapshot in any scheduling cycle. The schedulable task quantity indicates the number of target tasks that can be scheduled on the corresponding platform in the current scheduling cycle. The platform capacity snapshot is generated based on the schedulable device quantity and schedulable devices corresponding to multiple platforms in the current scheduling cycle. A schedulable device is a device among multiple execution devices that serves the corresponding platform and can be scheduled. For any platform, a total scheduling quota is determined based on the schedulable task quantity and schedulable device quantity corresponding to the platform. The total scheduling quota indicates the upper limit of the number of target tasks that can be retrieved from the platform in the current scheduling cycle. A task execution unit is invoked to retrieve a first target task from the platform, not exceeding the total scheduling quota, based on the validity period of the platform capacity snapshot. The schedulable device corresponding to the platform is invoked to execute the first target task. Therefore, by constructing a platform capacity snapshot based on the real-time schedulable device quantity of each platform in each scheduling cycle, and combining it with the platform's schedulable task quantity to determine the total scheduling quota of each platform in a single cycle, dynamic matching between task scheduling quantity and the platform's actual device concurrency capability can be achieved. At the same time, by strongly binding the task retrieval process with the validity period of the platform capacity snapshot, expired or outdated platform capacity status data can be effectively avoided for task scheduling operations. This effectively avoids invalid scheduling and repeated task rollback from the root, improves the accuracy and stability of task scheduling in multi-platform and multi-device scenarios, increases the resource utilization of devices, and reduces invalid scheduling overhead and computing power waste.

[0143] The collection, storage, use, processing, transmission, provision, and disclosure of users' personal information are all conducted with the consent of the users and comply with relevant laws and regulations, and do not violate public order and good morals.

[0144] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0145] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0146] like Figure 6As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded into random access memory (RAM) 603 from storage unit 608. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0147] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0148] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a multi-platform task scheduling method. For example, in some embodiments, the multi-platform task scheduling method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the multi-platform task scheduling method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the multi-platform task scheduling method by any other suitable means (e.g., by means of firmware).

[0149] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0151] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0153] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0154] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0155] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0156] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A task scheduling method for multiple platforms, the method comprising: In any scheduling cycle, the number of schedulable tasks corresponding to multiple platforms is obtained, and a platform capacity snapshot is obtained; wherein, the number of schedulable tasks is used to indicate the number of target tasks that can be scheduled on the corresponding platform in this scheduling cycle, and the platform capacity snapshot is generated based on the number of schedulable devices and schedulable devices corresponding to the multiple platforms in this scheduling cycle, wherein the schedulable devices are devices that serve the corresponding platform and can be scheduled among multiple execution devices; For any of the aforementioned platforms, the total scheduling quota for the platform is determined based on the schedulable task volume and the schedulable device volume corresponding to the platform; wherein, the total scheduling quota is used to indicate the upper limit of the number of target tasks that can be pulled from the platform in the current scheduling cycle; The task execution unit is invoked to pull a first target task from the platform, not exceeding the total scheduling quota, based on the validity period of the platform capacity snapshot. The platform's corresponding schedulable device is invoked to execute the first target task.

2. The method according to claim 1, wherein, The process of obtaining a platform capacity snapshot includes: Obtain device profiles of the plurality of execution devices; Based on the device profile, the plurality of execution devices are assigned to the corresponding platforms; In response to the completion of the allocation of the plurality of execution devices, the schedulable devices corresponding to the plurality of platforms are statistically analyzed to obtain the number of schedulable devices and the number of schedulable devices for any platform in the current scheduling cycle; Based on the number of schedulable devices and the number of schedulable devices corresponding to the multiple platforms in the current scheduling cycle, a platform capacity snapshot is generated.

3. The method according to claim 2, wherein, The device profile includes device-level status information, device capability information, and platform-level status information of any of the devices in the current scheduling cycle; the step of allocating the multiple execution devices to corresponding platforms according to the device profile includes: For any of the aforementioned execution devices, based on the device capability information of the execution device, the target platform served by the execution device is determined from the plurality of platforms; Based on the device-level status information and platform-level status information of the execution device in the current scheduling cycle, it is determined whether the execution device meets the allocation conditions for the target platform. In response to the execution device meeting the allocation conditions with respect to the target platform, the execution device is allocated to the target platform as a schedulable device corresponding to the target platform.

4. The method according to claim 3, wherein, The target platforms are multiple, and the process of assigning the execution device to the target platform as a schedulable device corresponding to the target platform includes: A first target platform is determined from the multiple target platforms based on at least one of the current remaining task volume, platform priority, service level agreement priority, and historical service volume; wherein, the historical service volume is used to record quantitative data of the execution device providing task execution services to the corresponding target platform within a historical time period; The execution device is assigned to the first target platform as a schedulable device corresponding to the first target platform.

5. The method according to claim 1, wherein, The method further includes: Based on the total scheduling quota corresponding to the multiple platforms, the expected total scheduling demand is determined; wherein, the expected total scheduling demand is used to indicate the total number of task execution units that need to be scheduled in the current scheduling cycle; Based on the expected total scheduling volume, the task execution units used to retrieve the target tasks in the current scheduling cycle are scaled up or down.

6. The method according to claim 5, wherein, The scaling up or down process for the task execution units used to retrieve the target tasks in the current scheduling cycle, based on the expected total scheduling demand, includes: The target scheduling quantity is obtained by taking the minimum value between the expected total scheduling quantity and the preset maximum scheduling quantity; wherein, the preset maximum scheduling quantity is used to indicate the maximum number of schedulable task execution units; Obtain the actual number of task execution units that actually run in the current scheduling cycle; In response to the target scheduling quantity being greater than the actual quantity, a number of new task execution units matching the difference between the target scheduling quantity and the actual quantity are created. In response to the target scheduling quantity being less than the actual quantity, a number of task execution units matching the difference between the target scheduling quantity and the actual quantity are released.

7. The method according to claim 1, wherein, The task execution unit, based on the validity period of the platform capacity snapshot, pulls a first target task from the platform, not exceeding the total scheduling quota, including: At least one round of iterative processing is performed on the platform, wherein, during any round of the iterative processing, it is determined whether the current moment is within the validity period corresponding to the platform capacity snapshot, and whether the remaining scheduling quota of the platform in this round of iterative processing is greater than a set threshold. If the current time is within the validity period corresponding to the platform capacity snapshot and the remaining scheduling quota is greater than the set threshold, the task execution unit in the current iteration process is invoked to perform a dequeue operation on the target task in the platform's task queue to obtain the first target task in the current iteration process; Based on the number of the first target tasks in the current iteration process, the remaining scheduling quota of the platform in the current iteration process is deducted to obtain the remaining scheduling quota of the platform in the next iteration process. In this process, the total number of tasks dequeued during at least one round of iteration does not exceed the total scheduling quota.

8. The method according to claim 7, wherein, The method further includes: Perform at least one of the following processes atomically: The first objective task in this round of iterative processing is updated with a new state. Record the timestamp of the first target task to be received in this round of iterative processing and the snapshot identifier of the platform capacity snapshot.

9. The method according to claim 8, wherein, The method further includes: In response to the successful deduction of the remaining scheduling quota during the current iteration process and the failure of the dequeue operation on the platform's task queue, a rollback operation is performed on the remaining scheduling quota of the platform. In response to the successful deduction of remaining scheduling quota during the current iteration process, the successful dequeue operation on the platform's task queue, and the failure of the status verification of the first target task obtained based on the dequeue operation, a rollback operation is performed on the platform's remaining scheduling quota.

10. The method according to claim 7, wherein, After the task execution unit in the current iteration process performs a dequeue operation on the target task in the platform's task queue to obtain the first target task of the current iteration process, the method further includes: The first target task is verified to obtain the verification result; In response to the verification result indicating that the first target task does not exist, or that the current state of the first target task is canceled or completed, the first target task is discarded, and the platform does not perform quota deduction processing on the remaining scheduling quota in the current iteration process.

11. The method according to claim 1, wherein, The step of invoking the schedulable device corresponding to the platform to execute the first target task includes: The task execution unit is invoked to request a device lease from the schedulable devices of the platform; In response to the successful application for the equipment lease, the successfully leased schedulable equipment is bound to the first target task, and the successfully leased schedulable equipment executes the bound first target task.

12. The method according to claim 11, wherein, The method further includes: In response to the failure of the device lease application, the first target task is returned to the task queue of the platform.

13. The method according to any one of claims 1-12, wherein, For any of the aforementioned platforms, determining the total scheduling quota for the platform based on the schedulable task volume and the schedulable device volume corresponding to the platform includes: For any of the aforementioned platforms, the minimum value is taken from the schedulable task quantity and schedulable device quantity corresponding to the platform to obtain the scheduling quota of the platform.

14. A task scheduling device for multiple platforms, the device comprising: The acquisition module is used to acquire the schedulable task quantity corresponding to multiple platforms and acquire a platform capacity snapshot in any scheduling cycle; wherein, the schedulable task quantity is used to indicate the number of target tasks that can be scheduled on the corresponding platform in the current scheduling cycle, and the platform capacity snapshot is generated based on the schedulable device quantity and schedulable devices corresponding to the multiple platforms in the current scheduling cycle, wherein the schedulable devices are devices that serve the corresponding platform and can be scheduled among multiple execution devices; The first determining module is used to determine the total scheduling quota of any platform based on the schedulable task quantity and the schedulable device quantity corresponding to the platform; wherein the total scheduling quota is used to indicate the upper limit of the number of target tasks that can be pulled from the platform in the current scheduling cycle. The pull module is used to call the task execution unit to pull a first target task from the platform, not exceeding the scheduling quota, based on the validity period of the platform capacity snapshot. The execution module is used to invoke the schedulable device corresponding to the platform to execute the first target task.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 13.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 13.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images