A remote sensing image processing resource dynamic hierarchical scheduling method and device

Abstract

The application discloses a remote sensing image processing resource dynamic hierarchical scheduling method and device, and the method comprises the following steps: acquiring remote sensing image processing task information; processing the remote sensing image processing task information to obtain the number of image processing tasks to be completed by a ground system within a time period tau; mapping the remote sensing image processing task to obtain a remote sensing image processing algorithm set; processing the remote sensing image processing algorithm set to obtain the computing resource consumption information of each meta task; processing the number of image processing tasks to be completed by the ground system within the time period tau according to the computing resource consumption information of each meta task to obtain the resource requirement information of the remote sensing image processing task to be executed by the system; and performing resource dynamic hierarchical scheduling on the remote sensing image processing task to be processed according to the remote sensing image processing task priority and the resource requirement information of the remote sensing image processing task to be executed by the system.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and application technology, and in particular to a method and apparatus for dynamic hierarchical scheduling of remote sensing image processing resources. Background Technology

[0002] With the widespread application of cloud computing technology across various fields, and the increasing scale and complexity of cloud computing applications, resource scheduling has become a key issue in cloud computing applications. Resource scheduling in the cloud environment mainly includes the scheduling of computing resources, storage resources, and network resources. The goal is to achieve efficient resource utilization and performance optimization to meet user needs and service level agreement requirements. Its performance directly impacts the service of the entire cloud platform. In compute-intensive applications, CPUs have unique advantages in memory access, serial processing, and complex process control. Compared to CPUs, GPUs have finer thread granularity, lower cache requirements, a larger number of computing cores, and stronger floating-point computing capabilities, making them better suited for parallel computing of large-scale data. The computing power consumed by remote sensing image processing mainly includes general-purpose computing resources such as CPUs, GPUs, and memory, but the computing power requirements vary across different processing flows. Existing cloud computing resource scheduling algorithms mainly include priority-based scheduling algorithms, queue-based scheduling algorithms, load balancing-based scheduling algorithms, and heuristic-based scheduling algorithms. They typically design resource allocation algorithms with general virtualization resources such as CPU and memory as the main resources, and give different resource scheduling strategies with the optimization goal of maximizing resource utilization. However, they do not take into account the characteristics and attributes of remote sensing image processing tasks in a comprehensive manner, and are not compatible with the special requirements of computing resources and actual application scenarios of remote sensing image processing. Summary of the Invention

[0003] The technical problem this invention aims to solve is to provide a method and apparatus for dynamic hierarchical scheduling of remote sensing image processing resources. Based on meta-task requirement assessment, this method achieves dynamic hierarchical scheduling of remote sensing image processing resources. The core of this method is its focus on practical application scenarios in remote sensing image processing. It achieves refined assessment of computing resources through the classification and decomposition of the image processing workflow and meta-task requirement analysis. Furthermore, it introduces a scenario-based task hierarchical scheduling strategy to realize dynamic resource allocation for remote sensing image processing scenarios. This method overcomes the shortcomings of existing technologies, improves the utilization efficiency of remote sensing image processing computing resources, and has significant engineering practical application value.

[0004] To address the aforementioned technical problems, a first aspect of the present invention discloses a method for dynamic hierarchical scheduling of remote sensing image processing resources, the method comprising:

[0005] S1, Obtain remote sensing image processing task information; the remote sensing image processing task information includes sensor type, working mode, working duration, data reception start time, data reception end time, and remote sensing image processing task priority information.

[0006] S2, process the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within the time period τ;

[0007] S3 maps remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms.

[0008] The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer;

[0009] The remote sensing image processing algorithm set includes a preprocessing algorithm set Y, a data cataloging algorithm set S, an image correction algorithm set T, and a model training algorithm set Z; the processing software or plugin in each processing flow of the remote sensing image processing algorithm set is regarded as a meta-task.

[0010] S4, process the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task; the computational resource consumption information of each meta-task includes the number of CPUs, memory size, and number of GPUs;

[0011] S5. Based on the computational resource consumption information of each meta-task, the number of image processing tasks that the ground system needs to complete within the time period τ is processed to obtain the resource requirement information for the system to execute a single remote sensing image processing task and the resource requirement information for the remote sensing image processing tasks that the system needs to execute.

[0012] S6. Based on the priority of the remote sensing image processing task, the resource requirement information of the system to execute a single remote sensing image processing task, and the resource requirement information of the remote sensing image processing tasks to be executed by the system, the remote sensing image processing task to be processed is dynamically scheduled in a hierarchical manner.

[0013] As an optional implementation, in the first aspect of the present invention, processing the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within the time period τ includes:

[0014] S21, The remote sensing image processing task information is processed to obtain a single image processing task;

[0015] S22, process the single image processing task to obtain the number of image processing tasks that the ground system needs to complete at time t;

[0016] S23, process the number of image processing tasks that the ground system needs to complete at time t to obtain the number of image processing tasks that the ground system needs to complete within time period τ.

[0017] As an optional implementation, in the first aspect of the present invention, the expression for the single image processing task is:

[0018] rtask={Sensor,Mode,Period,St,Et}

[0019] Where rtask is a single image processing task, Sensor is the sensor type, Mode is the working mode, Period is the working duration, St is the data reception start time, and Et is the data reception end time.

[0020] As an optional implementation, in the first aspect of the present invention, the expression for the number of image processing tasks that the ground system needs to complete at time t is:

[0021]

[0022] Where f(t) is the number of image processing tasks that the ground system needs to complete at time t, and rtask i For the i-th image processing task, St i Let Et be the start time for data reception of the i-th image processing task. i Let be the end time of data reception for the i-th image processing task.

[0023] As an optional implementation, in the first aspect of the present invention, processing the number of image processing tasks that the ground system needs to complete at time t to obtain the number of image processing tasks that the ground system needs to complete within time period τ includes:

[0024] Using an image processing task calculation model, the number of image processing tasks that the ground system needs to complete at time t is processed to obtain the number of image processing tasks that the ground system needs to complete within time period τ.

[0025] The image processing task calculation model expression is:

[0026]

[0027] Where F(τ) is the number of image processing tasks that the ground system needs to complete within time period τ, f(t) is the number of image processing tasks that the ground system needs to complete at time t, h(t) is the processing response function, and t0 is the reference time for the start of the calculation. This represents convolution.

[0028] As an optional implementation, in the first aspect of the present invention, the mapping of remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms includes:

[0029] By using a remote sensing image processing task mapping model, remote sensing image processing tasks are mapped to obtain a set of remote sensing image processing algorithms.

[0030] The expression for the remote sensing image processing task mapping model is:

[0031] f:rtask i →E i (Y i ,S i ,T i Z i )

[0032] Where f represents the mapping relationship, rtask i For the i-th image processing task, Y i =(y1,y2,…,y m () represents the set of algorithms for the i-th preprocessing process. Let S be the i1-th element in the i-th preprocessing algorithm set, where i1 = 1, 2, ..., m, and m is the number of elements in the i-th preprocessing algorithm set. i =(s1,s2,…,s n () represents the set of algorithms for the i-th data cataloging process. Let T be the i2th element in the i-th data cataloging process algorithm set, where i2 = 1, 2, ..., n, and n is the number of elements in the i-th data cataloging process algorithm set. i =(t1,t2,…,t p () represents the set of algorithms for the i-th image correction process. Z represents the i-th element in the i-th image correction process algorithm set, where i3 = 1, 2, ..., p, and p is the number of elements in the i-th image correction process algorithm set. i =(z1,z2,…,z) q () represents the set of algorithms for the training process of the i-th model. Let be the i-th 4th element in the i-th model training process algorithm set, where i4 = 1, 2, ..., q, and q is the number of elements in the i-th model training process algorithm set. The values ​​of m, n, p, and q are related to the actual business system and can be directly obtained from the system configuration. The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer. One processing software or plugin from the preprocessing process algorithm set Y, the data cataloging process algorithm set S, the image correction process algorithm set T, and the model training process algorithm set Z is considered as one meta-task. i Let be the i-th set of remote sensing image processing algorithms.

[0033] As an optional implementation, in the first aspect of the present invention, processing the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task includes:

[0034] Statistical analysis is performed on the historical execution of each meta-task to obtain information on the computational resource consumption of each meta-task;

[0035] The expression for the computational resource consumption information of each meta-task is as follows:

[0036]

[0037] in, This section contains the computational resource consumption information for the i-th meta-task. zycpu represents CPU, Tci represents the number of CPUs in the i-th meta-task, zygpu represents GPU, Tgi represents the number of GPUs in the i-th meta-task, zymem represents memory, and Tmi represents the memory size of the i-th meta-task.

[0038] As an optional implementation, in the first aspect of the present invention, the resource requirement information expression for the remote sensing image processing task to be performed by the system is:

[0039]

[0040] Where ZY(τ) represents the resource requirements of the remote sensing image processing tasks that the system needs to perform within the time period τ, and t0 is the reference time for the start of the calculation. Let H(t) represent convolution, and H(t) be the mapping between task and resource consumption. This section provides information on the computational resource consumption of the i-th meta-task. `zycpu` represents CPU, `Tci` is the number of CPUs in the i-th meta-task, `zygpu` represents GPU, `Tgi` is the number of GPUs in the i-th meta-task, `zymem` represents memory, `Tmi` is the memory size of the i-th meta-task, and `rtask`... i Let i be the i-th image processing task.

[0041] A second aspect of this invention discloses a dynamic hierarchical scheduling device for remote sensing image processing resources, the device comprising:

[0042] The information acquisition module is used to acquire remote sensing image processing task information; the remote sensing image processing task information includes sensor type, working mode, working duration, data reception start time, data reception end time, and remote sensing image processing task priority information.

[0043] The information processing module is used to process the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within the time period τ;

[0044] The task mapping module is used to map remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms.

[0045] The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer;

[0046] The remote sensing image processing algorithm set includes a preprocessing algorithm set Y, a data cataloging algorithm set S, an image correction algorithm set T, and a model training algorithm set Z; the processing software or plugin in each processing flow of the remote sensing image processing algorithm set is regarded as a meta-task.

[0047] The meta-task processing module is used to process the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task; the computational resource consumption information of each meta-task includes the number of CPUs, memory size, and number of GPUs.

[0048] The resource requirement calculation module is used to process the number of image processing tasks that the ground system needs to complete within the time period τ based on the computational resource consumption information of each meta-task, so as to obtain the resource requirement information of the system executing a single remote sensing image processing task and the resource requirement information of the remote sensing image processing tasks that the system needs to execute.

[0049] The hierarchical scheduling module is used to dynamically schedule resources for the remote sensing image processing tasks to be processed based on the priority of the remote sensing image processing tasks, the resource requirements of the system to execute a single remote sensing image processing task, and the resource requirements of the remote sensing image processing tasks to be executed by the system.

[0050] As an optional implementation, in the second aspect of the present invention, processing the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within the time period τ includes:

[0051] S21, The remote sensing image processing task information is processed to obtain a single image processing task;

[0052] S22, process the single image processing task to obtain the number of image processing tasks that the ground system needs to complete at time t;

[0053] S23, process the number of image processing tasks that the ground system needs to complete at time t to obtain the number of image processing tasks that the ground system needs to complete within time period τ.

[0054] As an optional implementation, in the second aspect of the present invention, the expression for the single image processing task is:

[0055] rtask={Sensor,Mode,Period,St,Et}

[0056] Where rtask is a single image processing task, Sensor is the sensor type, Mode is the working mode, Period is the working duration, St is the data reception start time, and Et is the data reception end time.

[0057] As an optional implementation, in the second aspect of the present invention, the expression for the number of image processing tasks that the ground system needs to complete at time t is:

[0058]

[0059] Where f(t) is the number of image processing tasks that the ground system needs to complete at time t, and rtask i For the i-th image processing task, St i Let Et be the start time for data reception of the i-th image processing task. i Let be the end time of data reception for the i-th image processing task.

[0060] As an optional implementation, in the second aspect of the present invention, processing the number of image processing tasks that the ground system needs to complete at time t to obtain the number of image processing tasks that the ground system needs to complete within time period τ includes:

[0061] Using an image processing task calculation model, the number of image processing tasks that the ground system needs to complete at time t is processed to obtain the number of image processing tasks that the ground system needs to complete within time period τ.

[0062] The image processing task calculation model expression is:

[0063]

[0064] Where F(τ) is the number of image processing tasks that the ground system needs to complete within time period τ, f(t) is the number of image processing tasks that the ground system needs to complete at time t, h(t) is the processing response function, and t0 is the reference time for the start of the calculation. This represents convolution.

[0065] As an optional implementation, in the second aspect of the present invention, the mapping of the remote sensing image processing task to obtain a set of remote sensing image processing algorithms includes:

[0066] By using a remote sensing image processing task mapping model, remote sensing image processing tasks are mapped to obtain a set of remote sensing image processing algorithms.

[0067] The expression for the remote sensing image processing task mapping model is:

[0068] f:rtaski →E i (Y i ,S i ,T i Z i )

[0069] Where f represents the mapping relationship, rtask i For the i-th image processing task, Y i =(y1,y2,…,y m () represents the set of algorithms for the i-th preprocessing process. Let S be the i1-th element in the i-th preprocessing algorithm set, where i1 = 1, 2, ..., m, and m is the number of elements in the i-th preprocessing algorithm set. i =(s1,s2,...,s n () represents the set of algorithms for the i-th data cataloging process. Let T be the i2th element in the i-th data cataloging process algorithm set, where i2 = 1, 2, ..., n, and n is the number of elements in the i-th data cataloging process algorithm set. i =(t1,t2,…,t p () represents the set of algorithms for the i-th image correction process. Z represents the i-th element in the i-th image correction process algorithm set, where i3 = 1, 2, ..., p, and p is the number of elements in the i-th image correction process algorithm set. i =(z1,z2,...,z) q () represents the set of algorithms for the training process of the i-th model. Let m be the 4th element in the i-th model training process algorithm set, where i4 = 1, 2, ..., q, and q is the number of elements in the i-th model training process algorithm set. The values ​​of m, n, p, and q are related to the actual business system and can be directly obtained from the system configuration. The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer. One processing software or plugin from the preprocessing process algorithm set Y, the data cataloging process algorithm set S, the image correction process algorithm set T, and the model training process algorithm set Z is considered as one meta-task. i Let be the i-th set of remote sensing image processing algorithms.

[0070] As an optional implementation, in the second aspect of the present invention, processing the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task includes:

[0071] Statistical analysis is performed on the historical execution of each meta-task to obtain information on the computational resource consumption of each meta-task;

[0072] The expression for the computational resource consumption information of each meta-task is as follows:

[0073]

[0074] in, This section contains the computational resource consumption information for the i-th meta-task. zycpu represents CPU, Tci represents the number of CPUs in the i-th meta-task, zygpu represents GPU, Tgi represents the number of GPUs in the i-th meta-task, zymem represents memory, and Tmi represents the memory size of the i-th meta-task.

[0075] As an optional implementation, in the second aspect of the present invention, the resource requirement information expression for the remote sensing image processing task to be performed by the system is:

[0076]

[0077] Where ZY(τ) represents the resource requirements of the remote sensing image processing tasks that the system needs to perform within time period τ, and t0 is the reference time for the start of the calculation. Let H(t) represent convolution, and H(t) be the mapping between task and resource consumption. This section provides information on the computational resource consumption of the i-th meta-task. `zycpu` represents CPU, `Tci` is the number of CPUs in the i-th meta-task, `zygpu` represents GPU, `Tgi` is the number of GPUs in the i-th meta-task, `zymem` represents memory, `Tmi` is the memory size of the i-th meta-task, and `rtask`... i Let i be the i-th image processing task.

[0078] A third aspect of the present invention discloses another device for dynamic hierarchical scheduling of remote sensing image processing resources, the device comprising:

[0079] Memory containing executable program code;

[0080] A processor coupled to the memory;

[0081] The processor calls the executable program code stored in the memory to execute some or all of the steps in the dynamic hierarchical scheduling method for remote sensing image processing resources disclosed in the first aspect of the present invention.

[0082] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0083] This invention provides a dynamic hierarchical scheduling method for remote sensing image processing resources. Based on the classification and decomposition of the image processing workflow and the analysis of meta-task requirements, it achieves a refined assessment of computing resources. By introducing a task hierarchical scheduling strategy, it realizes dynamic scheduling and allocation of resources for actual remote sensing image processing scenarios, improving the utilization efficiency of computing resources for remote sensing image processing under limited resource conditions. It represents a significant breakthrough in practicality and can be extended to related fields of remote sensing image processing and applications. Attached Figure Description

[0084] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0085] Figure 1 This is a flowchart illustrating a method for dynamic hierarchical scheduling of remote sensing image processing resources disclosed in an embodiment of the present invention;

[0086] Figure 2 This is a flowchart illustrating another method for dynamic hierarchical scheduling of remote sensing image processing resources disclosed in an embodiment of the present invention;

[0087] Figure 3 This is a schematic diagram comparing the number of remote sensing image data receiving tasks and meta-tasks disclosed in an embodiment of the present invention;

[0088] Figure 4 This is a comparative diagram of the consumption of various processing resources disclosed in the embodiments of the present invention;

[0089] Figure 5 This is a schematic diagram of the structure of a dynamic hierarchical scheduling device for remote sensing image processing resources disclosed in an embodiment of the present invention;

[0090] Figure 6 This is a schematic diagram of another remote sensing image processing resource dynamic hierarchical scheduling device disclosed in an embodiment of the present invention. Detailed Implementation

[0091] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0092] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0093] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0094] This invention discloses a method and apparatus for dynamic hierarchical scheduling of remote sensing image processing resources. The method includes: acquiring remote sensing image processing task information; processing the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within a time period τ; mapping the remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms; processing the set of remote sensing image processing algorithms to obtain the computational resource consumption information of each meta-task; processing the number of image processing tasks that the ground system needs to complete within a time period τ based on the computational resource consumption information of each meta-task to obtain the resource requirement information for the system to execute a single remote sensing image processing task and the resource requirement information for the remote sensing image processing tasks that the system needs to execute; and performing dynamic hierarchical scheduling of resources for the remote sensing image processing tasks to be processed based on the priority of the remote sensing image processing tasks, the resource requirement information for the system to execute a single remote sensing image processing task, and the resource requirement information for the remote sensing image processing tasks that the system needs to execute. These steps are described in detail below.

[0095] Example 1

[0096] Please see Figure 1 , Figure 1 This is a flowchart illustrating a dynamic hierarchical scheduling method for remote sensing image processing resources disclosed in an embodiment of the present invention. Figure 1 The described dynamic hierarchical scheduling method for remote sensing image processing resources is applied in the field of remote sensing image processing and applications, improving the utilization efficiency of computing resources for remote sensing image processing under limited resource conditions. This invention does not limit the scope of the embodiments. Figure 1 As shown, the dynamic hierarchical scheduling method for remote sensing image processing resources may include the following operations:

[0097] S1, Obtain remote sensing image processing task information; the remote sensing image processing task information includes sensor type, working mode, working duration, data reception start time, data reception end time, and remote sensing image processing task priority information.

[0098] S2, process the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within the time period τ;

[0099] S3 maps remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms.

[0100] The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer;

[0101] The remote sensing image processing algorithm set includes a preprocessing algorithm set Y, a data cataloging algorithm set S, an image correction algorithm set T, and a model training algorithm set Z; the processing software or plugin in each processing flow of the remote sensing image processing algorithm set is regarded as a meta-task.

[0102] S4, process the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task; the computational resource consumption information of each meta-task includes the number of CPUs, memory size, and number of GPUs;

[0103] S5. Based on the computational resource consumption information of each meta-task, the number of image processing tasks that the ground system needs to complete within the time period τ is processed to obtain the resource requirement information for the system to execute a single remote sensing image processing task and the resource requirement information for the remote sensing image processing tasks that the system needs to execute.

[0104] S6. Based on the priority of the remote sensing image processing task, the resource requirement information of the system to execute a single remote sensing image processing task, and the resource requirement information of the remote sensing image processing tasks to be executed by the system, the remote sensing image processing task to be processed is dynamically scheduled in a hierarchical manner.

[0105] Optionally, the step of processing the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within the time period τ includes:

[0106] S21, The remote sensing image processing task information is processed to obtain a single image processing task;

[0107] S22, process the single image processing task to obtain the number of image processing tasks that the ground system needs to complete at time t;

[0108] S23, process the number of image processing tasks that the ground system needs to complete at time t to obtain the number of image processing tasks that the ground system needs to complete within time period τ.

[0109] Optionally, the expression for the single image processing task is:

[0110] rtask={Sensor,Mode,Period,St,Et}

[0111] Where rtask is a single image processing task, Sensor is the sensor type, Mode is the working mode, Period is the working duration, St is the data reception start time, and Et is the data reception end time.

[0112] Optionally, the expression for the number of image processing tasks that the ground system needs to complete at time t is:

[0113]

[0114] Where f(t) is the number of image processing tasks that the ground system needs to complete at time t, and rtask i For the i-th image processing task, St i Let Et be the start time for data reception of the i-th image processing task. i Let be the end time of data reception for the i-th image processing task.

[0115] Optionally, the step of processing the number of image processing tasks that the ground system needs to complete at time t to obtain the number of image processing tasks that the ground system needs to complete within time period τ includes:

[0116] Using an image processing task calculation model, the number of image processing tasks that the ground system needs to complete at time t is processed to obtain the number of image processing tasks that the ground system needs to complete within time period τ.

[0117] The image processing task calculation model expression is:

[0118]

[0119] Where F(τ) is the number of image processing tasks that the ground system needs to complete within time period τ, f(t) is the number of image processing tasks that the ground system needs to complete at time t, h(t) is the processing response function, and t0 is the reference time for the start of the calculation. This represents convolution.

[0120] Optionally, the mapping of remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms includes:

[0121] By using a remote sensing image processing task mapping model, remote sensing image processing tasks are mapped to obtain a set of remote sensing image processing algorithms.

[0122] The expression for the remote sensing image processing task mapping model is:

[0123] f:rtask i →E i (Y i ,S i ,T i Z i )

[0124] Where f represents the mapping relationship, rtask i For the i-th image processing task, Y i =(y1,y2,…,y m () represents the set of algorithms for the i-th preprocessing process. Let S be the i1-th element in the i-th preprocessing algorithm set, where i1 = 1, 2, ..., m, and m is the number of elements in the i-th preprocessing algorithm set. i =(s1,s2,...,s n () represents the set of algorithms for the i-th data cataloging process. Let T be the i2th element in the i-th data cataloging process algorithm set, where i2 = 1, 2, ..., n, and n is the number of elements in the i-th data cataloging process algorithm set. i =(t1,t2,…,t p () represents the set of algorithms for the i-th image correction process. Z represents the i-th element in the i-th image correction process algorithm set, where i3 = 1, 2, ..., p, and p is the number of elements in the i-th image correction process algorithm set. i =(z1,z2,…,z) q () represents the set of algorithms for the training process of the i-th model. Let m be the 4th element in the i-th model training process algorithm set, where i4 = 1, 2, ..., q, and q is the number of elements in the i-th model training process algorithm set. The values ​​of m, n, p, and q are related to the actual business system and can be directly obtained from the system configuration. The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer. One processing software or plugin from the preprocessing process algorithm set Y, the data cataloging process algorithm set S, the image correction process algorithm set T, and the model training process algorithm set Z is considered as one meta-task. i Let be the i-th set of remote sensing image processing algorithms.

[0125] Optionally, the step of processing the remote sensing image processing algorithm set to obtain the computational resource consumption information for each meta-task includes:

[0126] Statistical analysis is performed on the historical execution of each meta-task to obtain information on the computational resource consumption of each meta-task;

[0127] The expression for the computational resource consumption information of each meta-task is as follows:

[0128]

[0129] in, This section contains the computational resource consumption information for the i-th meta-task. zycpu represents CPU, Tci represents the number of CPUs in the i-th meta-task, zygpu represents GPU, Tgi represents the number of GPUs in the i-th meta-task, zymem represents memory, and Tmi represents the memory size of the i-th meta-task.

[0130] Optionally, the resource requirement information expression for the remote sensing image processing task to be performed by the system is as follows:

[0131]

[0132] Where ZY(τ) represents the resource requirements of the remote sensing image processing tasks that the system needs to execute within the time period τ, zy(t) represents the resource requirements of the system executing a single remote sensing image processing task (obtained by classifying and accumulating the resource consumption of the corresponding meta-tasks for a single task; a single processing task (such as a single data reception) can be decomposed into multiple meta-tasks, and the resource requirements of a single processing task can be obtained by classifying and accumulating the resource requirements of the corresponding cloud tasks; this embodiment does not impose any restrictions), and t0 represents the reference time for the start of the calculation. Represents convolution. Here, H(t) represents the computational resource consumption information for the i-th meta-task, and H(t) is the mapping between the task and resource consumption. zycpu represents CPU, Tci is the number of CPUs in the i-th meta-task, zygpu represents GPU, Tgi is the number of GPUs in the i-th meta-task, zymem represents memory, Tmi is the memory size of the i-th meta-task, and rtask i Let i be the i-th image processing task.

[0133] As can be seen, the present invention provides a dynamic hierarchical scheduling method for remote sensing image processing resources. Based on the classification and decomposition of the image processing process and the analysis of meta-task requirements, it achieves a refined evaluation of computing resources. By introducing a task hierarchical scheduling strategy, it realizes dynamic scheduling and allocation of resources for actual remote sensing image processing scenarios, improving the utilization efficiency of computing resources for remote sensing image processing under limited resource conditions. It has made a significant breakthrough in practicality and can be extended to related fields of remote sensing image processing and applications.

[0134] Example 2

[0135] Please see Figure 2 , Figure 2 This is a flowchart illustrating another method for dynamic hierarchical scheduling of remote sensing image processing resources disclosed in an embodiment of the present invention. Figure 2The described dynamic hierarchical scheduling method for remote sensing image processing resources is applied in the field of remote sensing image processing and applications, improving the utilization efficiency of computing resources for remote sensing image processing under limited resource conditions. This invention does not limit the scope of the embodiments. Figure 2 As shown, the dynamic hierarchical scheduling method for remote sensing image processing resources may include the following operations:

[0136] Step 1: Obtain Task Information

[0137] Retrieve detailed information about remote sensing image processing tasks, including sensor type, operating mode, duration, data reception start time, data reception end time, and task priority. A single image processing task (rtask) is described as a function of the sensor type (Sensor), operating mode (Mode), duration (Period), data reception start time (St), and end time (Et):

[0138] rtask={Sensor,Mode,Period,St,Et} (1)

[0139] For a given time t, the number of image processing tasks f(t) that the ground system needs to complete can be initially calculated using the sensor workload plan:

[0140]

[0141] Furthermore, the number of image processing tasks that the ground system needs to complete within any time period τ can be calculated as F(τ), which is the convolution of the processing tasks and the processing response function h(t). Here, the processing response function h(t) is a function of the actual reception time of the system with respect to the planned processing task time, representing the data transmission time delay under actual conditions, and can be obtained through statistical analysis of historical system information or actual testing.

[0142]

[0143] Step 2: Process Breakdown

[0144] Based on the computational resource requirements of remote sensing image processing, the remote sensing image processing workflow is categorized into four types: preprocessing, data cataloging, image correction, and model training. The processing software or plugins within each workflow are considered as a meta-task. For a single image processing task (rtask),... i Image processing tasks can be further mapped to sets of image processing algorithms based on the image processing workflow, namely, preprocessing workflow algorithm set Y, data cataloging workflow algorithm set S, image correction workflow algorithm set T, and model training workflow algorithm set Z. Each algorithm set represents the set of processing algorithms that the ground system actually needs to call to complete the image processing task.

[0145]

[0146] In Formula 4, the values ​​of m, n, p, and q are related to the actual business system and can be obtained directly from the system configuration.

[0147] Step 3: Meta-task Requirements Analysis

[0148] Statistical analysis is performed based on the historical execution data of the software or plugins corresponding to each meta-task to determine the CPU, memory, and GPU requirements of each meta-task. Building upon step two, the processing algorithm set E... i By conducting a preliminary assessment of the resource requirements of each processing algorithm, the resource requirement res for each processing algorithm can be obtained. i Based on image processing computing power analysis, the resource requirement is res i The main requirements include three categories: CPU, memory, and GPU.

[0149]

[0150] Among them, Tc, Tg, and Tm correspond to the time consumed by CPU, GPU, and memory, respectively.

[0151] Step 4: Resource Requirements Calculation

[0152] Based on the resource consumption of each meta-task, calculate the resource requirement zy(t) for a single remote sensing image processing task to be executed by the system (obtained by summing the resource consumption of the corresponding meta-tasks; a single processing task (such as a single data reception) can be decomposed into multiple meta-tasks, and the resource requirement of a single processing task can be obtained by summing the resource requirements of the corresponding cloud tasks), and the resource requirement for the remote sensing image processing tasks to be executed by the system. Based on step three, the total resource requirement ZY(τ) for the image processing task F(τ) to be completed by the ground system within any time period τ is:

[0153]

[0154] H(t) is the mapping between processing tasks and resource consumption, which represents the delay between the actual resource consumption time of the system and the evaluation time. It can be obtained by statistically analyzing the historical processing information of the system or by actual testing.

[0155]

[0156] Step 5: Hierarchical Task Scheduling

[0157] Dynamic resource scheduling is implemented based on the priority of remote sensing image processing tasks and the resource requirements of individual tasks. High-priority tasks are executed first through resource reservation and preemption, while tasks with the same priority are queued for execution. Without loss of generality, remote sensing image processing tasks can be categorized into three types: "urgent," "important," and "general," and the processing flow can be further refined and configured according to specific application needs. When scheduling resources based on priority, the following factors should also be considered: 1) Urgent and important tasks have relatively abundant computing resources to ensure success rates; 2) Tasks with the same priority should be scheduled and allocated based on maximizing the overall efficiency of resources; 3) From the perspective of system robustness, the frequency of secondary or higher resource allocation should be minimized. Based on the above factors, three resource conflict handling methods are set: reservation, preemption, and queuing. The specific execution process is as follows:

[0158] 1) Set up task queues according to task attributes: task_yj, task_zy, task_yb;

[0159] 2) At time T0, query the tasks currently being executed by the system and those to be executed within the time interval (T1-T0), along with their priority attributes, denoted as task_tmp;

[0160] 3) When task_tmp is empty, the system sleeps (T1-T0); otherwise, steps 4-6 are executed.

[0161] 4) If the task attribute in task_tmp is yj, proceed to step 5; otherwise, proceed to step 7.

[0162] 5) Calculate the resource requirement res_n and compare it with the existing resource res_k. If res_n <= res_k, reserve processing resources; otherwise, proceed to step 6.

[0163] 6) If task_tmp is not empty, then end the currently executing task in the task_yb queue and jump to step 4; if task_tmp is empty, then change task_tmp->task_yj.

[0164] 7) If the task attribute in task_tmp is equal to zy, proceed to step 8; otherwise, skip to step X.

[0165] 8) Calculate the resource requirement res_n and compare it with the existing resource res_k. If res_n <= res_k, reserve processing resources; otherwise, proceed to step 9.

[0166] 9) If task_tmp is not empty, then end the currently executing task in the task_yb queue and jump to step 7; if task_tmp is empty, then queue task_tmp->task_zy.

[0167] 10) Jump to step 1 and repeat.

[0168] To avoid frequent adjustments to resource allocation, this example only preempts resources for general tasks. Emergency tasks can be configured to preempt resources for important tasks based on actual task requirements. Figure 3 This is a schematic diagram comparing the number of remote sensing image data receiving tasks and meta-tasks disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing the consumption of various processing resources disclosed in the embodiments of the present invention.

[0169] Example 3

[0170] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a dynamic hierarchical scheduling device for remote sensing image processing resources disclosed in an embodiment of the present invention. Figure 5 The described dynamic hierarchical scheduling device for remote sensing image processing resources is applied in the field of remote sensing image processing and applications, improving the utilization efficiency of computing resources for remote sensing image processing under limited resource conditions. This invention is not limited to any particular embodiment. Figure 5 As shown, the remote sensing image processing resource dynamic hierarchical scheduling device may include the following operations:

[0171] S301, Information Acquisition Module, used to acquire remote sensing image processing task information; the remote sensing image processing task information includes sensor type, working mode, working duration, data reception start time, data reception end time, and remote sensing image processing task priority information.

[0172] S302, Information processing module, used to process the remote sensing image processing task information to obtain the number of image processing tasks that the ground system needs to complete within the time period τ;

[0173] S303, Task Mapping Module, is used to map remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms;

[0174] The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer;

[0175] The remote sensing image processing algorithm set includes a preprocessing algorithm set Y, a data cataloging algorithm set S, an image correction algorithm set T, and a model training algorithm set Z.

[0176] S304, Meta-task processing module, is used to process the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task; the computational requirement information of each meta-task includes the number of CPUs, memory size, and number of GPUs.

[0177] S305, Resource Requirement Calculation Module, is used to process the number of image processing tasks that the ground system needs to complete within the time period τ based on the computational resource consumption information of each meta-task, and obtain the resource requirement information of the system executing a single remote sensing image processing task and the resource requirement information of the remote sensing image processing tasks that the system needs to execute.

[0178] S306, a hierarchical scheduling module, is used to perform dynamic hierarchical scheduling of resources for the remote sensing image processing tasks to be processed based on the priority of the remote sensing image processing tasks, the resource requirement information of the system to execute a single remote sensing image processing task, and the resource requirement information of the remote sensing image processing tasks to be executed by the system.

[0179] Example 4

[0180] Please see Figure 6 , Figure 6 This is a schematic diagram of another dynamic hierarchical scheduling device for remote sensing image processing resources disclosed in an embodiment of the present invention. Figure 6 The described dynamic hierarchical scheduling device for remote sensing image processing resources is applied in the field of remote sensing image processing and applications, improving the utilization efficiency of computing resources for remote sensing image processing under limited resource conditions. This invention is not limited to any particular embodiment. Figure 6 As shown, the remote sensing image processing resource dynamic hierarchical scheduling device may include the following operations:

[0181] Memory 401 storing executable program code;

[0182] Processor 402 coupled to memory 401;

[0183] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the dynamic hierarchical scheduling method for remote sensing image processing resources described in Embodiment 1 and Embodiment 2.

[0184] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0185] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0186] Finally, it should be noted that the method and apparatus for dynamic hierarchical scheduling of remote sensing image processing resources disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for dynamic hierarchical scheduling of remote sensing image processing resources, characterized in that, The method includes: S1, Obtain remote sensing image processing task information; the remote sensing image processing task information includes sensor type, working mode, working duration, data reception start time, data reception end time, and remote sensing image processing task priority information. The remote sensing image processing task priority is used for hierarchical task scheduling. Resources are dynamically scheduled according to the priority of the remote sensing image processing task and the resource requirements of a single remote sensing image processing task. Through resource reservation and preemption, high-priority tasks are guaranteed to be executed first, and tasks with the same priority are executed in a queue order. S2, process the remote sensing image processing task information to obtain the time period. The number of image processing tasks that the inland ground system needs to complete includes: S21, The remote sensing image processing task information is processed to obtain a single image processing task; S22, process the single image processing task to obtain the time. The number of image processing tasks that the ground system needs to complete; S23, regarding the time... The number of image processing tasks that the ground system needs to complete is processed to obtain the time period. The number of image processing tasks that the inland ground system needs to complete; S3 maps remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms. The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer; The remote sensing image processing algorithm set includes a preprocessing algorithm set Y, a data cataloging algorithm set S, an image correction algorithm set T, and a model training algorithm set Z; the processing software or plugin in each processing flow of the remote sensing image processing algorithm set is regarded as a meta-task. S4, process the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task; the computational resource consumption information of each meta-task includes the number of CPUs, memory size, and number of GPUs; S5, based on the computational resource consumption information of each meta-task, the time period... The number of image processing tasks that the inland surface system needs to complete is processed to obtain the resource requirements information for the system to execute a single remote sensing image processing task and the resource requirements information for the remote sensing image processing tasks that the system needs to execute. The expression for the single image processing task is: in, For a single image processing task, For sensor type, For work mode, For working hours, The start time for data reception. This is the end time for data reception; time The expression for the number of image processing tasks that the ground system needs to complete is: in, For a moment The number of image processing tasks that the ground system needs to complete. For the first An image processing task For the first Start time of data reception for each image processing task For the first End time of data reception for each image processing task; S6. Based on the priority of the remote sensing image processing task, the resource requirement information of the system to execute a single remote sensing image processing task, and the resource requirement information of the remote sensing image processing tasks to be executed by the system, the remote sensing image processing task to be processed is dynamically scheduled in a hierarchical manner.

2. The method for dynamic hierarchical scheduling of remote sensing image processing resources according to claim 1, characterized in that, The time The number of image processing tasks that the ground system needs to complete is processed to obtain the time period. The number of image processing tasks that the inland ground system needs to complete includes: Using an image processing task calculation model, for the time... The number of image processing tasks that the ground system needs to complete is processed to obtain the time period. The number of image processing tasks that the inland ground system needs to complete; The image processing task calculation model expression is: in, For time period The number of image processing tasks that the inland ground system needs to complete. For a moment The number of image processing tasks that the ground system needs to complete. To process the response function, To calculate the reference time for the start, This represents convolution.

3. The method for dynamic hierarchical scheduling of remote sensing image processing resources according to claim 1, characterized in that, The mapping of remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms includes: By using a remote sensing image processing task mapping model, remote sensing image processing tasks are mapped to obtain a set of remote sensing image processing algorithms. The expression for the remote sensing image processing task mapping model is: in, Indicates the mapping relationship. For the first An image processing task, For the first A set of preprocessing algorithms, For the first The first preprocessing algorithm set One element, , For the first The number of elements in the preprocessing algorithm set. For the first A set of data cataloging process algorithms, For the first The first data cataloging process algorithm set One element, , For the first The number of elements in a data cataloging process algorithm set. For the first A set of image correction process algorithms, For the first The first image correction process algorithm set One element, , For the first The number of elements in the set of image correction process algorithms. For the first A set of algorithms for model training processes. For the first The first model training process algorithm set One element, , For the first The number of elements in the algorithm set for each model training process. , , , The value is related to the actual business system and is obtained directly from the system configuration. The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer. One processing software or plugin from the preprocessing process algorithm set Y, the data cataloging process algorithm set S, the image correction process algorithm set T, and the model training process algorithm set Z is regarded as one meta-task. For the first A set of remote sensing image processing algorithms.

4. The method for dynamic hierarchical scheduling of remote sensing image processing resources according to claim 1, characterized in that, The process of processing the remote sensing image processing algorithm set to obtain computational resource consumption information for each meta-task includes: Statistical analysis is performed on the historical execution of each meta-task to obtain information on the computational resource consumption of each meta-task; The expression for the computational resource consumption information of each meta-task is as follows: in, For the first Information on the computational resource consumption of each meta-task. Represents CPU. For the first Number of CPUs for each meta-task Represents GPU, For the first The number of GPUs for each meta-task Represents memory. For the first The memory size of each metatask.

5. The method for dynamic hierarchical scheduling of remote sensing image processing resources according to claim 1, characterized in that, The resource requirement information expression for the remote sensing image processing task that the system needs to perform is as follows: in, For time period Resource requirements for the remote sensing image processing tasks that the internal system needs to perform. This represents the resource requirements for a single remote sensing image processing task that the system needs to perform. To calculate the reference time for the start, Represents convolution. Mapping tasks to resource consumption. , For the first Information on the computational resource consumption of each meta-task. Represents CPU. For the first Number of CPUs for each meta-task Represents GPU, For the first The number of GPUs for each meta-task Represents memory. For the first The memory size of each metatask For the first Image processing tasks.

6. A dynamic hierarchical scheduling device for remote sensing image processing resources, characterized in that, The device includes: The information acquisition module is used to acquire remote sensing image processing task information; the remote sensing image processing task information includes sensor type, working mode, working duration, data reception start time, data reception end time, and remote sensing image processing task priority information. The remote sensing image processing task priority is used for hierarchical task scheduling. Resources are dynamically scheduled according to the priority of the remote sensing image processing task and the resource requirements of a single remote sensing image processing task. Through resource reservation and preemption, high-priority tasks are guaranteed to be executed first, and tasks with the same priority are executed in a queue order. The information processing module is used to process the remote sensing image processing task information to obtain time periods. The number of image processing tasks that the inland ground system needs to complete includes: S21, The remote sensing image processing task information is processed to obtain a single image processing task; S22, process the single image processing task to obtain the time. The number of image processing tasks that the ground system needs to complete; S23, regarding the time... The number of image processing tasks that the ground system needs to complete is processed to obtain the time period. The number of image processing tasks that the inland ground system needs to complete; The task mapping module is used to map remote sensing image processing tasks to obtain a set of remote sensing image processing algorithms. The remote sensing image processing algorithm set includes N meta-tasks, where N is a positive integer; The remote sensing image processing algorithm set includes a preprocessing algorithm set Y, a data cataloging algorithm set S, an image correction algorithm set T, and a model training algorithm set Z. The meta-task processing module is used to process the remote sensing image processing algorithm set to obtain the computational resource consumption information of each meta-task; the computational requirement information of each meta-task includes the number of CPUs, memory size, and number of GPUs. The resource requirement calculation module is used to calculate the time period based on the computational resource consumption information of each meta-task. The number of image processing tasks that the inland surface system needs to complete is processed to obtain the resource requirements information for the system to execute a single remote sensing image processing task and the resource requirements information for the remote sensing image processing tasks that the system needs to execute. The expression for the single image processing task is: in, For a single image processing task, For sensor type, For work mode, For working hours, The start time for data reception. This is the end time for data reception; time The expression for the number of image processing tasks that the ground system needs to complete is: in, For a moment The number of image processing tasks that the ground system needs to complete. For the first An image processing task, For the first Start time of data reception for each image processing task For the first End time of data reception for each image processing task; The hierarchical scheduling module is used to dynamically schedule resources for the remote sensing image processing tasks to be processed based on the priority of the remote sensing image processing tasks, the resource requirements of the system to execute a single remote sensing image processing task, and the resource requirements of the remote sensing image processing tasks to be executed by the system.

7. A dynamic hierarchical scheduling device for remote sensing image processing resources, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the dynamic hierarchical scheduling method for remote sensing image processing resources as described in any one of claims 1-5.

Images