Remote sensing image calculation task scheduling method and system in resource limited scene

Abstract

The invention relates to the technical field of distributed computing and resource scheduling, and provides a remote sensing image computing task scheduling method and system in a resource limited scene, and the method comprises the steps: injecting a submitted remote sensing image computing task into a task queue of a uniform resource pool; monitoring the state of each node in the resource pool and the computing cluster, obtaining a to-be-scheduled task from the task queue, screening out all candidate nodes meeting a hard condition from the cluster nodes through a filtering strategy according to the resource demand of the task, and scoring each candidate node based on a multi-dimensional scoring strategy; calculating a multi-dimensional weighted comprehensive score of each candidate node, and selecting the candidate node with the highest comprehensive score as a target node; and binding the task to the target node, and receiving and executing the task by the Agent on the node. According to the method, dynamic allocation and load balancing of resources are realized while the task execution efficiency is ensured, so that the overall utilization rate of a computing cluster and the intelligent level of scheduling are improved.

Description

Technical Field

[0001] This invention relates to the field of distributed computing and resource scheduling technology, and more specifically, to a method and system for scheduling remote sensing image computing tasks in resource-limited scenarios. Specifically, it is applicable to efficient and intelligent multi-strategy collaborative scheduling of batch computing tasks for image data in resource-constrained cloud computing or edge computing environments. Background Technology

[0002] With the rapid development of remote sensing technology, the amount of satellite and aerial remote sensing data has exploded, leading to increasingly frequent raster computation and vector analysis tasks based on this data. These tasks are typically computationally intensive, memory-intensive, and require high I / O throughput.

[0003] Currently, traditional task scheduling strategies (such as first-come, first-served and round-robin scheduling) perform poorly in resource-constrained environments, often leading to uneven resource utilization, high task queuing latency, and poor user experience. Especially in multi-user shared cloud computing platforms, tasks from different users with different priorities compete for limited computing resources (such as GPUs and memory), and the lack of fine-grained scheduling mechanisms easily triggers resource contention and performance interference. Although existing technologies have introduced priority scheduling and load balancing mechanisms, they still lack deep adaptation to the characteristics of remote sensing computing tasks and cannot perform collaborative optimization based on multi-dimensional information such as task type, data locality, and user level, making it difficult to further improve the overall efficiency and resource utilization of the cluster. Therefore, there is an urgent need for an intelligent scheduling method for remote sensing computing scenarios to achieve efficient resource utilization and precise guarantee of task performance. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing a method and system for scheduling remote sensing image computing tasks in resource-limited scenarios, so as to achieve efficient resource utilization and accurate guarantee of task performance.

[0005] According to a first aspect of the present invention, a method for scheduling remote sensing image computing tasks in resource-constrained scenarios is provided, comprising:

[0006] Step S1: Receive the remote sensing image calculation task submitted by the user and attached with an identity authentication token;

[0007] Step S2: Inject the remote sensing image calculation task into the task queue of the unified resource pool;

[0008] Step S3: Monitor the status of the task queue in the unified resource pool and the status of each node in the computing cluster through the scheduler. When the task queue is not empty, obtain the next task to be scheduled and its metadata. The metadata includes at least user identity, resource requirement type and task type.

[0009] Step S4: Based on the resource requirements of the task to be scheduled, a filtering strategy is used to select all candidate nodes from all nodes of the computing cluster that meet the hard conditions of the task to be scheduled, and a candidate node list is generated.

[0010] Step S5: Score each candidate node in the candidate node list based on the multi-dimensional scoring strategy to obtain the multi-dimensional score of each candidate node.

[0011] Step S6: Based on the multidimensional score of each candidate node, calculate the weighted comprehensive score of each candidate node, and determine the candidate node with the highest comprehensive score as the target node.

[0012] Step S7: Bind the task to be scheduled to the target node, and retrieve and execute the task to be scheduled based on the Agent on the target node;

[0013] Step S8: During the execution of the task to be scheduled, the execution status of the task to be scheduled is reported to the unified resource pool in real time; after the task to be scheduled is completed, the occupied resources are released and the resource pool status is updated.

[0014] According to a second aspect of the present invention, a remote sensing image computing task scheduling system for resource-constrained scenarios is provided, comprising:

[0015] The receiving module is used to receive remote sensing image calculation tasks submitted by users and accompanied by authentication tokens;

[0016] The monitoring module is used to inject the remote sensing image computing task into the task queue of the unified resource pool; and to monitor the status of the task queue in the unified resource pool and the status of each node in the computing cluster through the scheduler. When the task queue is not empty, the next task to be scheduled and its metadata are obtained. The metadata includes at least user identity, resource requirement type and task type.

[0017] The filtering module is used to filter out all candidate nodes that meet the hard conditions of the task to be scheduled from all nodes of the computing cluster according to the resource requirements of the task to be scheduled, and generate a candidate node list.

[0018] The calculation module is used to score each candidate node in the candidate node list based on a multi-dimensional scoring strategy, calculate the multi-dimensional score of each candidate node, and calculate the weighted comprehensive score of each candidate node based on the multi-dimensional score of each candidate node, and determine the candidate node with the highest comprehensive score as the target node.

[0019] The execution module is used to bind the task to be scheduled to the target node, and to retrieve and execute the task based on the Agent on the target node;

[0020] The reporting module is used to report the execution status of the scheduled task to the unified resource pool in real time during the execution of the scheduled task; after the scheduled task is completed, it releases the occupied resources and updates the resource pool status.

[0021] This invention provides a method and system for scheduling remote sensing image computing tasks in resource-constrained scenarios. The method involves injecting submitted remote sensing image computing tasks into a task queue within a unified resource pool; monitoring the status of each node in the resource pool and computing cluster; retrieving tasks to be scheduled from the task queue; filtering candidate nodes that meet the hard requirements based on the task's resource needs using a filtering strategy; scoring each candidate node based on a multi-dimensional scoring strategy; calculating the multi-dimensional weighted comprehensive score of each candidate node; and selecting the candidate node with the highest comprehensive score as the target node; binding the task to the target node, where an agent on that node receives and executes the task. Through unified resource pooling and a multi-dimensional intelligent scoring strategy, this method significantly improves cluster resource utilization efficiency and multi-tenant service quality, achieving efficient and fair scheduling of remote sensing computing tasks. Attached Figure Description

[0022] Figure 1 A flowchart of a remote sensing image computing task scheduling method in a resource-constrained scenario is provided as an embodiment of the present invention;

[0023] Figure 2 This is a flowchart of a remote sensing image computing task scheduling strategy in a resource-constrained scenario according to an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of a remote sensing image computing task scheduling system in a resource-constrained scenario, provided by an embodiment of the present invention.

[0025] Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0026] Figure 5 This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0028] This invention provides a method for scheduling remote sensing image computing tasks. It decouples tasks and resources by constructing a unified resource pool and adopts a two-stage scheduling mechanism that combines filtering and multi-dimensional scoring. It comprehensively considers multiple factors such as user level, resource demand matching degree, node load balancing and data localization, and finally selects the optimal computing node through a weighted decision model. This significantly improves the cluster resource utilization, ensures the quality of multi-tenant services and optimizes task execution efficiency in resource-constrained environments.

[0029] Figure 1 This invention illustrates a method for scheduling remote sensing image computing tasks in resource-constrained scenarios, as provided in one embodiment of the present invention. See [link to relevant documentation]. Figure 1 and Figure 2 The computational task scheduling method mainly includes the following steps:

[0030] Step S1: Receive the remote sensing image calculation task submitted by the user and attached with an identity authentication token.

[0031] Understandably, in step S1, the user submits a remote sensing image calculation task request through the client interface. This request is encapsulated in JSON format, containing the parameters necessary for task execution, the path to the required input remote sensing data files, the desired output format, and a JWT token for authentication. Upon receiving the request, the system gateway first verifies the validity and permissions of the token. Only after successful verification does it forward the task request to the scheduling system.

[0032] Step S2: Inject the remote sensing image calculation task into the task queue of the unified resource pool.

[0033] Understandably, user-submitted remote sensing image computation tasks are placed into a unified resource pool's task queue. This unified resource pool is a global task queue data structure maintained in memory, used to store all submitted but not yet scheduled task objects. This resource pool provides atomic task enqueueing and dequeueing interfaces and supports concurrent access from multiple scheduler instances, thus ensuring high availability and scalability of the system.

[0034] Step S3: Monitor the status of the task queue in the unified resource pool and the status of each node in the computing cluster through the scheduler. When the task queue is not empty, obtain the next task to be scheduled and its metadata. The metadata includes at least user identity, resource requirement type and task type.

[0035] Understandably, remote sensing image computing tasks are placed in a task queue within a unified resource pool and executed by the nodes in the computing cluster. Therefore, during the execution of remote sensing image computing tasks, it is necessary to retrieve the tasks to be scheduled from the task queue of the unified resource pool and allocate the corresponding computing nodes.

[0036] In one embodiment of the present invention, a scheduler monitors the status of the task queue in the unified resource pool and the status of each node in the computing cluster. The scheduler polls and checks the status of the task queue in the resource pool at fixed time intervals. When a non-empty queue is detected, the scheduler acquires the next task to be scheduled through a mutex lock mechanism and synchronously obtains detailed metadata information of the task and real-time resource status snapshots of each node in the cluster from the cluster metadata service, providing data support for subsequent scheduling decisions.

[0037] Understandably, this embodiment of the invention implements a scheduling loop mechanism, where the scheduling loop is the core driving mechanism of the scheduler. It runs continuously as an independent background service, responsible for coordinating the entire process from task acquisition to resource binding. This service is implemented in a multi-threaded manner to achieve high concurrency processing capabilities. The specific implementation steps are as follows:

[0038] The first step is for the scheduler to actively monitor the task queue status in the unified resource pool and obtain real-time resource metrics for each compute node by interacting with the cluster monitoring system. When it detects that the task queue is not empty and there are available resources in the cluster that meet the basic conditions, a scheduling cycle is immediately triggered.

[0039] The second step is for the scheduler to acquire a distributed lock to ensure data consistency in a multi-instance contention environment and atomically retrieve the next task to be scheduled from the head of the resource pool task queue.

[0040] Third, the scheduler queries the metadata service or parses the task object itself to obtain key metadata, including user identity, resource requirement type, and task type. After data acquisition, the scheduler hands over the task object and metadata to the filtering strategy stage for processing, releases the distributed lock, and then re-enters the listening state to begin the next round of the loop.

[0041] Step S4: Based on the resource requirements of the task to be scheduled, a filtering strategy is used to select all candidate nodes from all nodes of the computing cluster that meet the hard conditions of the task to be scheduled, and a candidate node list is generated.

[0042] Understandably, after a task to be scheduled is extracted from the task queue of the unified resource pool, a candidate node list is generated by filtering all nodes in the computing cluster based on the resource requirements of the task to be scheduled and selecting all candidate nodes that meet the hard conditions of the task to be scheduled.

[0043] This stage aims to quickly and efficiently eliminate all nodes that do not meet the hardware requirements, forming a smaller list of candidate nodes, thereby reducing the computational overhead of subsequent scoring strategies. The implementation of this stage relies on a set of predefined hard filtering rules. The specific implementation process is as follows:

[0044] The first step is to initialize filtering conditions based on the resource request declaration of the task to be scheduled. These filtering conditions mainly include whether the number of available GPUs on the node is greater than or equal to the task request amount, whether the available memory capacity of the node meets the minimum requirements of the task, and whether the node label matches the runtime environment or hardware constraints required by the task.

[0045] The second step involves the scheduler traversing all active nodes in the current cluster in parallel, matching each node against the above filtering conditions, and discarding any node that fails to fully meet all the conditions.

[0046] The third step is to aggregate all the nodes that have passed the verification into a candidate node list, and then pass this list as input to the scoring strategy stage for fine-tuning.

[0047] Specifically, the filtering formula for the filtering strategy is:

[0048] ;

[0049] in, For the filtered results, a value of 1 indicates a node. A value of 0 indicates that the filter did not pass. and They are nodes Available GPU and memory resources; and Tasks to be scheduled Requested GPU and memory resources; Represents a node The tags contain tasks to be scheduled. All required labels.

[0050] Step S5: Score each candidate node in the candidate node list based on the multi-dimensional scoring strategy to obtain the multi-dimensional score of each candidate node.

[0051] Understandably, if step S4 filters and selects candidate nodes from all nodes in the computing cluster, then this step assigns the most suitable computing node to the task to be scheduled from all candidate nodes.

[0052] In this embodiment of the invention, the scoring strategy is the core manifestation of scheduling intelligence and plays a crucial role in the optimization phase. This strategy calculates a comprehensive score for each candidate node through multi-dimensional quantitative evaluation, thereby identifying the globally optimal target node. The implementation process consists of three steps:

[0053] The first step involves independently scoring each node in the candidate node list using four metrics in parallel. These include a user-level strategy that maps user identity to a priority multiplier, a resource demand prediction strategy that predicts resource demand based on the XGBoost model and calculates the matching degree with node availability, a load balancing strategy that calculates load rate based on real-time CPU and memory utilization of nodes, and a data localization strategy that evaluates the network distance between task data and node storage location.

[0054] Specifically, the scheduler will quantify the score of each node in the candidate node list based on a set of configurable, pluggable scoring strategies. Each strategy evaluates the node's suitability from a specific optimization dimension. The specific implementation is as follows:

[0055] The user tier strategy, based on business rules, aims to ensure service quality for high-value users. A user priority mapping table is maintained, mapping different user identities to different score multipliers. High-priority users will receive a higher base score on all candidate nodes, thus gaining an advantage in resource competition. The scoring formula for the user tier strategy in step S5 is:

[0056] ;

[0057] in, Score the user level strategy; The preset base score; The user priority multiplier is determined based on the user U's identity level when submitting the task. The rules for determining the user priority multiplier are as follows: 1.5 for administrators, 1.2 for VIP users, 1.0 for regular users, and 0.8 for trial users.

[0058] The resource demand forecasting strategy is based on a machine learning model for refined resource model matching. The system uses a pre-trained XGBoost regression model, taking the task's metadata features (input data size, computational operation type, output format, processing precision) as input, to predict the precise amount of resources required for its actual runtime. Then, it calculates the matching degree between this predicted value and the node's currently available resources, and converts the matching degree into a score. The formula for this matching degree conversion is:

[0059] ;

[0060] in, Score the resource demand forecasting strategy; This is the full score for the strategy; For tasks estimated using resource prediction models The required amount of critical resources; For nodes The amount of this critical resource currently available.

[0061] Load balancing strategies aim to improve overall system efficiency, avoid computational hotspots, and balance cluster load. They calculate real-time load rates based on metrics such as CPU and memory utilization. Nodes with lower load rates receive higher scores, guiding new tasks to be scheduled more frequently on less busy nodes, thus improving overall cluster resource utilization and reducing node overload risk. The formula for calculating node load rate is:

[0062] ;

[0063] in, Score the load balancing strategy; This is the full score for the strategy; and They are nodes The current CPU utilization and memory utilization, with values ​​ranging from [value range missing]. .

[0064] Data localization strategies aim to minimize data transfer overhead, which is crucial for data-intensive remote sensing computing tasks. They score data based on the physical storage location of the required data and its distance from the network topology of the computing nodes. Data stored on the node's local disk receives the highest score, followed by data stored on a storage node within the same rack, then data stored on a remote node within the same cluster, and data retrieved from remote storage (such as public cloud object storage) receives the lowest score. Prioritizing nodes with high scores can significantly reduce data preparation time before task execution. The formula for calculating the data localization strategy is as follows:

[0065] ;

[0066] in, Score the data localization strategy; This is the full score for the strategy; , , The discount factor is decreasing, and 1> > > >0; Data is stored on the node's local disk. Data is stored on storage nodes within the same rack. Data is stored on a remote node within the same cluster. This allows data to be pulled from remote storage, such as public cloud object storage.

[0067] Step S6: Based on the multidimensional score of each candidate node, calculate the weighted comprehensive score of each candidate node, and determine the candidate node with the highest comprehensive score as the target node.

[0068] Understandably, step S5 above calculates the user level strategy score, resource demand prediction strategy score, load balancing strategy score, and data localization strategy score for each candidate node. This step calculates the weighted comprehensive score for each candidate node based on its multidimensional score.

[0069] Specifically, the weighted decision-making and node binding in step S6 is the final step in the scheduling strategy. Its purpose is to integrate the quantitative results of the multi-dimensional scoring strategy and select a unique optimal node target from the candidate nodes through a configurable, comprehensive decision-making model. The scheduler calculates the weighted comprehensive score for each candidate node. The formula for calculating this score is as follows:

[0070] ;

[0071] in, This is the final weighted score for node N; , , , These are the weight coefficients corresponding to each strategy. , , , These represent the scores for each strategy, and satisfy the following conditions: .

[0072] The candidate node with the highest weighted composite score is selected as the target node and assigned to the task to be scheduled.

[0073] Step S7: Bind the task to be scheduled to the target node, and retrieve and execute the task based on the Agent on the target node.

[0074] Step S8: During the execution of the task to be scheduled, the execution status of the task to be scheduled is reported to the unified resource pool in real time; after the task to be scheduled is completed, the occupied resources are released and the resource pool status is updated. During the execution of the task to be scheduled, the execution status of the task to be scheduled is reported to the unified resource pool in real time, and after the task to be scheduled is completed, the occupied resources in the unified resource pool are released and the unified resource pool is updated.

[0075] Understandably, after selecting the target node from all candidate nodes, the task to be scheduled is bound to the target node, and the agent on the target node receives and executes the task to be scheduled.

[0076] The task execution phase is responsible for implementing scheduling decisions and managing the task's lifecycle. This process is primarily automated by the cluster infrastructure and includes the following steps: The scheduler calls the cluster APIServer interface to perform a binding operation, setting the node selection field of the task object to the selected target node name, thus completing task scheduling at the declarative level. The Agent program running on the target node continuously listens for API Server change events. When it detects a new task being bound to the node, it immediately triggers the task acquisition process. Following the task specifications, the Agent sequentially completes preparatory work such as pulling the specified container image, allocating computing resources (CPU, memory) locally, and mounting the specified data storage volume, and finally starts the task command in the isolated container environment. During task execution, the Agent continuously monitors the container status and periodically reports heartbeat information such as resource utilization and progress metrics to the monitoring system. This information is ultimately aggregated into a unified resource pool for real-time status display. After task execution, the Agent automatically performs resource cleanup and reports the task completion status. The resource pool then reclaims all resource quotas used by the task and updates the global resource view, thus forming a complete closed loop.

[0077] See Figure 3 This paper illustrates a remote sensing image computing task scheduling system in a resource-constrained scenario according to an embodiment of the present invention. The system includes:

[0078] The receiving module 301 is used to receive remote sensing image calculation tasks submitted by users and attached with identity authentication tokens;

[0079] The monitoring module 302 is used to inject the remote sensing image computing task into the task queue of the unified resource pool; and to monitor the status of the task queue in the unified resource pool and the status of each node in the computing cluster through the scheduler. When the task queue is not empty, the next task to be scheduled and its metadata are obtained. The metadata includes at least user identity, resource requirement type and task type.

[0080] The filtering module 303 is used to filter out all candidate nodes that meet the hard conditions of the task to be scheduled from all nodes of the computing cluster according to the resource requirements of the task to be scheduled, and generate a candidate node list.

[0081] The calculation module 304 is used to score each candidate node in the candidate node list based on a multi-dimensional scoring strategy, calculate the multi-dimensional score of each candidate node; and calculate the weighted comprehensive score of each candidate node based on the multi-dimensional score of each candidate node, and determine the candidate node with the highest comprehensive score as the target node.

[0082] Execution module 305 is used to bind the task to be scheduled to the target node, and to retrieve and execute the task to be scheduled based on the Agent on the target node;

[0083] The reporting module 306 is used to report the execution status of the scheduled task to the unified resource pool in real time during the execution of the scheduled task; after the scheduled task is completed, it releases the occupied resources and updates the resource pool status.

[0084] It is understood that the remote sensing image computing task scheduling system provided by the present invention corresponds to the remote sensing image computing task scheduling method provided by the foregoing embodiments. The relevant technical features of the remote sensing image computing task scheduling system in resource-limited scenarios can be referred to the relevant technical features of the remote sensing image computing task scheduling method in resource-limited scenarios, and will not be repeated here.

[0085] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 4As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, it implements the steps of a remote sensing image computing task scheduling method in resource-limited scenarios.

[0086] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, it implements the following steps of a remote sensing image computing task scheduling method in a resource-constrained scenario.

[0087] This invention provides a method for scheduling remote sensing image computing tasks in resource-constrained scenarios. By unifying resource pooling and using a multi-dimensional intelligent scoring strategy, and comprehensively considering factors such as user level, resource demand matching degree, node load balancing, and data localization, it achieves efficient and fair scheduling of remote sensing computing tasks, significantly improving cluster resource utilization efficiency and multi-tenant service quality.

[0088] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0089] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0091] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0092] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0093] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0094] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for scheduling remote sensing image computing tasks in resource-constrained scenarios, characterized in that, include: Step S1: Receive the remote sensing image calculation task submitted by the user and attached with an identity authentication token; Step S2: Inject the remote sensing image calculation task into the task queue of the unified resource pool; Step S3: Monitor the status of the task queue in the unified resource pool and the status of each node in the computing cluster through the scheduler. When the task queue is not empty, obtain the next task to be scheduled and its metadata. The metadata includes at least user identity, resource requirement type and task type. Step S4: Based on the resource requirements of the task to be scheduled, a filtering strategy is used to select all candidate nodes from all nodes of the computing cluster that meet the hard conditions of the task to be scheduled, and a candidate node list is generated. Step S5: Score each candidate node in the candidate node list based on the multi-dimensional scoring strategy to obtain the multi-dimensional score of each candidate node. Step S6: Based on the multidimensional score of each candidate node, calculate the weighted comprehensive score of each candidate node, and determine the candidate node with the highest comprehensive score as the target node. Step S7: Bind the task to be scheduled to the target node, and retrieve and execute the task to be scheduled based on the Agent on the target node; Step S8: During the execution of the task to be scheduled, the execution status of the task to be scheduled is reported to the unified resource pool in real time. Once the scheduled task is completed, release the occupied resources and update the resource pool status.

2. The remote sensing image computing task scheduling method according to claim 1, characterized in that, The filtering strategy in step S4 includes at least the following: filtering out nodes whose GPU quantity does not meet the requirements of the scheduled task, filtering out nodes whose memory capacity does not meet the requirements of the scheduled task, and filtering out nodes that do not have the specified label for the scheduled task. The filtering formula is as follows: ； in, For the filtered results, a value of 1 indicates a node. The filter indicates that a value of 0 represents a failed pass. Indicates tasks to be scheduled; and They are nodes Available GPU and memory resources; and Tasks to be scheduled Requested GPU and memory resources; Represents a node The tags contain tasks to be scheduled. All required labels, The label representing node N, Indicates tasks to be scheduled All required labels.

3. The remote sensing image computing task scheduling method according to claim 1, characterized in that, The multi-dimensional scoring strategy in step S5 includes at least a user level strategy, a resource demand prediction strategy, a load balancing strategy, and a data localization strategy.

4. The remote sensing image computing task scheduling method according to claim 3, characterized in that, The scoring formula for the user level strategy is as follows: ； in, Score the user level strategy; The preset base score; This is the user's priority multiplier, determined based on the identity level of the user U who submitted the task. The identity level of user U is determined based on the user's authentication token.

5. The remote sensing image computing task scheduling method according to claim 3, characterized in that, The scoring formula for the resource demand forecasting strategy is as follows: ； in, Score the resource demand forecasting strategy; The full score for resource demand forecasting strategies; Tasks to be scheduled are estimated using a resource prediction model. The required amount of critical resources; For nodes The amount of critical resources currently available.

6. The remote sensing image computing task scheduling method according to claim 5, characterized in that, The resource prediction model is a regression model trained based on historical task data. The input features of the resource prediction model include metadata features of the task T to be predicted. The metadata features include at least the data input size, computation operation type, output result format, and processing precision.

7. The remote sensing image computing task scheduling method according to claim 3, characterized in that, The scoring formula for the load balancing strategy is as follows: ； in, Score the load balancing strategy; This represents the full score for the load balancing strategy. and They are nodes The current CPU utilization and memory utilization, with values ​​ranging from [value range missing]. .

8. The remote sensing image computing task scheduling method according to claim 3, characterized in that, The scoring formula for the data localization strategy is as follows: ； in, Score the data localization strategy; This represents the maximum score for the data localization strategy. , , The discount factor is decreasing, and 1> > > >0; Data is stored on the node's local disk. Data is stored on storage nodes within the same rack. Data is stored on a remote node within the same cluster. To retrieve data from remote storage.

9. The remote sensing image computing task scheduling method according to claim 3, characterized in that, Step S6, based on the multidimensional score of each candidate node, calculates the weighted comprehensive score of each candidate node, including: ; in, The weighted composite score of candidate node N; , , , These are the weighting coefficients for user-level strategy, resource demand prediction strategy, load balancing strategy, and data localization strategy, respectively. , , , These represent the user-level strategy score, resource demand prediction strategy score, load balancing strategy score, and data localization strategy score, respectively, and satisfy the following conditions: .

10. A remote sensing image computing task scheduling system for resource-constrained scenarios, characterized in that, include: The receiving module is used to receive remote sensing image calculation tasks submitted by users and accompanied by authentication tokens; The monitoring module is used to inject the remote sensing image computing task into the task queue of the unified resource pool; The scheduler monitors the status of the task queue in the unified resource pool and the status of each node in the computing cluster. When the task queue is not empty, it obtains the next task to be scheduled and its metadata, which includes at least user identity, resource requirement type and task type. The filtering module is used to filter out all candidate nodes that meet the hard conditions of the task to be scheduled from all nodes of the computing cluster according to the resource requirements of the task to be scheduled, and generate a candidate node list. The calculation module is used to score each candidate node in the candidate node list based on a multi-dimensional scoring strategy, and calculate the multi-dimensional score of each candidate node. Based on the multidimensional score of each candidate node, a weighted comprehensive score is calculated for each candidate node, and the candidate node with the highest comprehensive score is determined as the target node. The execution module is used to bind the task to be scheduled to the target node, and to retrieve and execute the task based on the Agent on the target node; The reporting module is used to report the execution status of the task to be scheduled to the unified resource pool in real time during the execution of the task to be scheduled. Once the scheduled task is completed, release the occupied resources and update the resource pool status.

Images