Two-stage imaging satellite task scheduling method, storage medium and computer device

Abstract

The application discloses a two-stage imaging satellite task scheduling method, a storage medium and computer equipment, relates to the technical field of aerospace engineering, and mainly comprises the following steps: constructing an imaging task planning model according to imaging tasks and satellite resources; obtaining a pre-scheduling task set, a backtracking task list, a to-be-processed task list and a resource allocation scheme by using a conflict-aware heuristic scheduling strategy according to the imaging task planning model; and obtaining a scheduling scheme by using a RainbowDQN algorithm. The two-stage imaging satellite task scheduling method, the storage medium and the computer equipment can improve the response efficiency and flexibility of satellite task scheduling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of aerospace engineering, and more particularly, to a two-stage imaging satellite task scheduling method, a storage medium and a computer device. BACKGROUND

[0002] With the rapid development of aerospace technology, imaging satellites have become an indispensable observation tool in the field of remote sensing. Relying on the high-precision imaging equipment carried, imaging satellites can perform high-resolution observation of the earth's surface, and are widely used in agricultural monitoring, forestry protection, urban planning and marine environmental monitoring, etc. and play an important role in promoting the country's economic development and social progress. When imaging satellites effectively perform remote sensing tasks, with the rapid growth of user demand under the condition of limited satellite resources, how to plan a large number of user tasks to develop a reasonable and effective task observation sequence is crucial to fully exert the advantages of imaging satellites and provide high-quality space-based information services. In the algorithm research of imaging satellite task planning, there are many kinds of algorithms for solving complex task scheduling problems. From classical algorithms to modern intelligent optimization methods, researchers have proposed a variety of solutions, but these methods usually face the problem of high computational complexity when dealing with complex task scheduling. Especially in the case of large task size and multiple resource constraints, the dilemma of "solution quality" and "solving efficiency" is difficult to balance. Some methods sacrifice the quality of the solution in exchange for a shorter time to get the result, while other methods need to repeatedly search for iterations to find a high-quality solution, resulting in huge time consumption. With the development of artificial intelligence technology and satellite platform computing power, the demand for autonomous planning and decision-making technology in imaging satellites is becoming more and more urgent. SUMMARY

[0003] The purpose of the present application is to provide a two-stage imaging satellite task scheduling method, a storage medium and a computer device, which can improve the response efficiency and flexibility of satellite task scheduling.

[0004] The present application provides a two-stage imaging satellite task scheduling method, comprising the following steps: S1: constructing an imaging task planning model according to the imaging task and the satellite resources; S2: obtaining a pre-scheduling task set, a backtracking task list, a to-be-processed task list and a resource allocation scheme by using a conflict-aware heuristic scheduling strategy according to the imaging task planning model; S3: obtaining a scheduling scheme by using a RainbowDQN algorithm according to the pre-scheduling task set, the backtracking task list, the to-be-processed task list and the resource allocation scheme.

[0005] Further, step S1 specifically comprises: constructing an imaging task planning model according to the imaging task and the satellite resources, such as the formula:

[0006] ,

[0007] ,

[0008] ,

[0009] , ,

[0010] ,

[0011] ,

[0012] ,

[0013] ,

[0014] ,

[0015] ,

[0016] ,

[0017] ,

[0018] ,

[0019] wherein, is a set of tasks, is the th observed task, denotes the number of tasks in the set, is a set of resources, is the th resource, denotes the number of satellite resources available in the scenario, is a set of allowed visibility time windows for a task to occupy a resource , denotes the th allowed visibility time window for a task to occupy a resource , denotes the number of allowed visibility time windows for a task to occupy a resource , and denote the start time and the end time of the th allowed visibility time window for a task to occupy a resource , is a task reward value, The execution time required for the task. Storage consumption for tasks Battery consumption for the task. The duration of posture transitions required for the task. For all that can be arranged in resources The set of tasks to be executed. To meet the observation task The required set of resources; The objective function for planning the model for the imaging task is... Indicates task In resources The first A Boolean variable representing the execution status over a visible time window. To maximize, and Representing tasks The specific start and end times when selected for scheduling planning; and Representing tasks The start and end times of the scene's time window. Indicates the first Maximum data storage capacity of each resource Indicates and the The maximum total power storage limit for each resource.

[0020] Further, step S2 specifically includes: S21: Based on the imaging task planning model, obtain the task time window, map the task time window to the corresponding available resources, and obtain the spatiotemporal distribution characteristics of the task; S22: Based on the spatiotemporal distribution characteristics of the task, use a pre-scheduling strategy to obtain a pre-scheduled task set; S23: Based on the spatiotemporal distribution characteristics of the task and the pre-scheduled task set, use a backtracking search method to obtain a backtracking task list and a list of tasks to be processed; S24: Calculate the conflict degree and set the conflict degree threshold, and obtain a resource allocation scheme based on the conflict degree, the conflict degree threshold, the backtracking task list, and the list of tasks to be processed.

[0021] Further, step S24 specifically includes: S241: Calculating the conflict degree of the task, as shown in the formula:

[0022] ,

[0023] ,

[0024] ,

[0025] ,

[0026] , , ,

[0027] in, For all possible and visible time windows on the same resource The set of time windows that can generate potential resource contention conflicts. Visible time window and Conflicting time slices where all time windows in the set intersect. The required pose transition time for the corresponding task. For the task Visible time window Time-series conflict indicators For the task The degree of conflict To retrieve the task list, This is a list of tasks to be processed. S242: Obtain the average temporal conflict index and its corresponding standard deviation and conflict threshold based on the conflict degree of the tasks; S243: Obtain the temporal conflict degree of the time window using a greedy strategy based on the conflict degree threshold, task execution time, weight that can be obtained after execution, and temporal conflict index of the visible time window; S244: Select the time window with the minimum conflict degree as the task allocation time window based on the time window conflict degree, and filter out the tasks with the minimum time window conflict degree exceeding the conflict degree threshold from the set of tasks to be scheduled to obtain the resource allocation scheme.

[0028] Further, step S242 specifically includes: obtaining the average temporal conflict index and its corresponding standard deviation and conflict threshold based on the conflict degree of the task, as shown in the formula:

[0029] ,

[0030] ,

[0031] ,

[0032] in, This is the average time-series conflict index. For the number of conflicting tasks, The corresponding standard deviation is... This is the conflict threshold.

[0033] Further, step S243 specifically includes: based on the conflict degree threshold, task execution time, the weights obtainable after execution, and the temporal conflict index of the visible time window, a greedy strategy is used to obtain the time window conflict degree, as shown in the formula:

[0034] ,

[0035] in, For time windows The degree of conflict.

[0036] Further, step S3 specifically includes: S31: Constructing empirical priority, empirical sampling probability, empirical weighting coefficient, and loss function; S32: Based on the pre-scheduled task set, backtracked task list, pending task list, and resource allocation scheme, using empirical priority, empirical sampling probability, empirical weighting coefficient, loss function, and RainbowDQN algorithm to train the Markov decision process model, obtaining a trained Markov decision process model; S33: Using the trained Markov decision process model, obtaining the scheduling scheme.

[0037] Further, step S31 specifically includes: constructing the empirical priority, empirical sampling probability, empirical weighting coefficient, and loss function, as shown in the formula:

[0038] ,

[0039] ,

[0040] ,

[0041] ,

[0042] ,

[0043] in, For the first Prioritize the experience points; For the first The time difference error of the experience; It is a small constant used to avoid cases with zero priority; For the first The sampling probability of the empirical evidence, It is the total number of experience points stored in the experience pool. The expected total return of the Markov decision process model; As a discount factor, and These are the Q values ​​of the target network and the current network, respectively; and The first The environmental state at each time step and state-based pass The chosen action For the maximum value index function, For the first a weighting coefficient of a piece of experience; is a hyperparameter used to control the degree of importance adjustment of the sample, which gradually increases during the training process to reduce the bias of the sample; is a loss function.

[0044] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the two-stage imaging satellite task scheduling method.

[0045] The application further provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the two-stage imaging satellite task scheduling method when executing the program.

[0046] The two-stage imaging satellite task scheduling method, the storage medium and the computer device provided by the application have the following beneficial effects:

[0047] The application provides a two-stage imaging satellite task scheduling framework based on conflict awareness and Rainbow DQN (hereinafter referred to as "CAS-D2RL framework"), in the first stage, through the heuristic scheduling strategy based on conflict awareness, the spatiotemporal distribution characteristics of the task are analyzed in depth, the potential risk of resource contention and task conflict is accurately identified and quantified, so as to realize effective inhibition and optimization of the conflict in the resource allocation stage, avoid the occurrence of resource bottlenecks in advance, and reduce the conflict between tasks as much as possible, provide a more stable and reliable basis for the subsequent task scheduling and planning stage; in the second stage, the Rainbow DQN algorithm in deep reinforcement learning (DRL) is introduced, the dynamic decision-making process is simulated, the historical data and the current environmental information are used, the real-time state of the task and its network decision mechanism are used, the dynamic changes of the environment in the task scheduling process are comprehensively considered, through the adaptive mechanism in reinforcement learning, the strategy is continuously learned and updated, the scheduling strategy is adjusted in real time in the complex and changing multi-task scene and environment, the decision-making process is optimized, the dynamic optimal scheduling decision is realized, the deficiencies of traditional methods in complexity and real-time performance are overcome, the complex resource allocation and task conflict in the imaging satellite task planning process are alleviated, the continuous optimization and dynamic adaptation of task scheduling under different conditions are realized, the dynamic real-time response to observation task demand is realized, the response efficiency and flexibility of satellite task scheduling are improved, and the autonomy and resource utilization efficiency of the satellite are improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] The application will be further described below with reference to the drawings and examples, and the drawings show:

[0049] Figure 1is a flow chart of the two-stage imaging satellite task scheduling method provided by the present application;

[0050] Figure 2 is a distribution diagram of the task visibility time window provided by the present application;

[0051] Figure 3 is a time window diagram of resource contention conflict provided by the present application;

[0052] Figure 4 is a Rainbow DQN training structure diagram provided by the present application;

[0053] Figure 5 is a structure diagram of the imaging satellite task scheduling framework provided by the present application;

[0054] Figure 6 is a structure diagram of the computer equipment provided by the present application. DETAILED DESCRIPTION

[0055] In order to have a clearer understanding of the technical features, objectives and effects of the present application, the specific embodiments of the present application will be described in detail with reference to the drawings.

[0056] Figure 1 The present embodiment shows a schematic diagram of the two-stage imaging satellite task scheduling method. In the present embodiment, the two-stage imaging satellite task scheduling method comprises the following steps:

[0057] S1: constructing an imaging task planning model according to the imaging task and the satellite resource;

[0058] In an exemplary embodiment, step S1 specifically comprises: constructing an imaging task planning model according to the imaging task and the satellite resource, such as the formula:

[0059] ,

[0060] ,

[0061] ,

[0062] , ,

[0063] ,

[0064] ,

[0065] ,

[0066] ,

[0067] ,

[0068] ,

[0069] ,

[0070] ,

[0071] ,

[0072] where, is a set of tasks, is the th observation task, denotes the number of tasks in the set, is a set of resources, is the th resource, denotes the number of satellite resources available in the scenario, is a set of allowed visibility time windows for a task to occupy a resource , denotes the th allowed visibility time window for a task to occupy a resource , denotes the number of allowed visibility time windows for a task to occupy a resource , and denote the start time and end time of the th allowed visibility time window for a task to occupy a resource , is a task reward value, is a task execution duration, is a task storage consumption, is a task power consumption, is a task attitude translation duration, is a set of all tasks that can be scheduled to be executed on a resource , is a set of resources that satisfy the requirement of an observation task ; is an objective function of the imaging task planning model, is a Boolean variable representing the execution status of a task in the th visibility time window on a resource , is to maximize, and respectively represent the start time and the end time of the task respectively represent the start time and the end time of the specific start time and the end time of the scheduling plan when selected and respectively represent the start time and the end time of the task the start time and the end time of the scenario time window to which it belongs, represent the maximum data storage capacity of the first resource, represent the total upper limit of the power storage of the first resource;

[0073] S2: According to the imaging task planning model, a conflict-aware heuristic scheduling strategy is used to obtain a pre-scheduled task set, a backtracking task list, a to-be-processed task list, and a resource allocation scheme;

[0074] In an exemplary embodiment, step S2 specifically comprises:

[0075] S21: According to the imaging task planning model, a task time window is obtained, the task time window is mapped to a corresponding available resource, and the spatiotemporal distribution characteristics of the task are obtained;

[0076] S22: According to the spatiotemporal distribution characteristics of the task, a pre-scheduling strategy is used to obtain a pre-scheduled task set;

[0077] In an exemplary embodiment, according to the spatiotemporal distribution characteristics of the task, a pre-scheduling strategy is used to obtain a pre-scheduled task set, as shown in the formula:

[0078] ,

[0079] wherein, is the pre-scheduled task set;

[0080] S23: According to the spatiotemporal distribution characteristics of the task and the pre-scheduled task set, a backtracking search method is used to obtain a backtracking task list and a to-be-processed task list;

[0081] As an exemplary embodiment, in step S23, the tasks are arranged in descending order of priority, and the tasks with higher weights are given priority to find the best time period. The tasks successfully allocated to the time window are put into the backtracking task list , and the time periods of these tasks meet the constraint conditions and do not conflict with other tasks in the backtracking list. In this process, a backtracking search method is used to explore possible solutions. When a resource conflict or a time window that does not meet the conditions is encountered, the previous task can be returned to, and the allocation of the resource time window is adjusted to find a better allocation scheme. For tasks that fail to solve resource contention conflicts, they are put into the to-be-processed task list , so as to be further processed in the subsequent stage; these tasks often fail to find a suitable allocation scheme immediately due to fierce resource competition or strong time window constraints; the union of the backtracking tasks and the pending tasks is denoted as for subsequent conflict reduction processing;

[0082] S24: calculating the conflict degree and setting the conflict degree threshold, and obtaining a resource allocation scheme according to the conflict degree, the conflict degree threshold, the backtracking task list and the pending task list;

[0083] In an exemplary embodiment, step S24 specifically comprises:

[0084] S241: calculating the conflict degree of a task, as shown in the following formula:

[0085] ,

[0086] ,

[0087] ,

[0088] ,

[0089] , , ,

[0090] wherein, is a set of time windows that can potentially cause a resource contention conflict with the visible time window of all tasks, is the visible time window of the task , is a conflict time slice intersected with all time windows in the set, is a posture transformation duration required by the corresponding task, is a time sequence conflict indicator of the visible time window of the task, is the conflict degree of the task , is the backtracking task list, is the pending task list, is the union of the backtracking tasks and the pending tasks;

[0091] S242: obtaining an average time sequence conflict indicator, a corresponding standard deviation and a conflict degree threshold according to the conflict degree of a task;

[0092] In an exemplary embodiment, step S242 specifically comprises: obtaining an average time sequence conflict indicator, a corresponding standard deviation and a conflict degree threshold according to the conflict degree of a task, as shown in the following formula:

[0093] ,

[0094] ,

[0095] ,

[0096] wherein, is the average time sequence conflict index, is the number of conflicting tasks, is the corresponding standard deviation, is the conflict threshold value;

[0097] S243: obtaining the time window conflict degree according to the conflict threshold value, the task execution time, the obtainable weight after execution, and the time sequence conflict index of the visible time window by using the greedy strategy;

[0098] In an exemplary embodiment, step S243 specifically comprises: obtaining the time window conflict degree according to the conflict threshold value, the task execution time, the obtainable weight after execution, and the time sequence conflict index of the visible time window by using the greedy strategy, as the formula:

[0099] ,

[0100] wherein, is the conflict degree of the time window ;

[0101] S244: selecting the time window with the minimum conflict degree as the allocation time window of the task according to the time window conflict degree, and filtering the task whose conflict degree in the minimum time window exceeds the conflict threshold value from the set of tasks to be scheduled, to obtain a resource allocation scheme;

[0102] S3: obtaining a scheduling scheme by using the RainbowDQN algorithm according to the pre-scheduled task set, the backtracking task list, the task list to be processed, and the resource allocation scheme;

[0103] In an exemplary embodiment, step S3 specifically comprises:

[0104] S31: constructing an experience priority, an experience sampling probability, an experience weighting coefficient, and a loss function;

[0105] In an exemplary embodiment, step S31 specifically comprises: constructing an experience priority, an experience sampling probability, an experience weighting coefficient, and a loss function, as the formula:

[0106] ,

[0107] ,

[0108] ,

[0109] ,

[0110] ,

[0111] in, For the first Prioritize the experience points; For the first The time difference error of the experience; It is a small constant used to avoid cases with zero priority; For the first The sampling probability of the empirical evidence, It is the total number of experience points stored in the experience pool. The expected total return of the Markov decision process model; As a discount factor, and These are the Q values ​​of the target network and the current network, respectively; and The first The environmental state at each time step and state-based pass The chosen action For the maximum value index function, For the first The weighting coefficients of the empirical values; It is a hyperparameter used to control the degree of importance adjustment of samples. It is gradually increased during training to reduce sampling bias. The loss function;

[0112] S32: Based on the pre-scheduled task set, backtracked task list, pending task list and resource allocation scheme, the Markov decision process model is trained using empirical priority, empirical sampling probability, empirical weighting coefficient, loss function and RainbowDQN algorithm to obtain the trained Markov decision process model.

[0113] S33: Using a trained Markov decision process model, obtain the scheduling scheme.

[0114] In one exemplary embodiment, the two-stage imaging satellite mission scheduling method can be implemented as follows:

[0115] 1、Construction of CAS-D2RL framework: The primary purpose of this embodiment is to construct a CAS-D2RL scheduling framework that combines conflict awareness and Rainbow DQN; this framework integrates heuristic strategies with conflict awareness and deep reinforcement learning methods to optimize resource allocation and conflict resolution during task scheduling, thereby improving the performance of the overall scheduling system;

[0116] 2、Resource allocation method based on conflict awareness: In the first stage of scheduling, for the resource allocation problem in imaging satellite tasks, this embodiment proposes a heuristic scheduling strategy based on conflict awareness; this strategy accurately identifies and quantifies the potential risks of resource contention and task conflicts by deeply analyzing the temporal and spatial distribution characteristics of the task, thereby effectively suppressing and optimizing conflicts during resource allocation; through this method, resource bottlenecks can be avoided in advance, and conflicts between tasks can be minimized, providing a more stable and reliable foundation for subsequent task scheduling and planning stages;

[0117] 3、DRL-driven task scheduling decision: In the second stage of scheduling, this embodiment introduces the Rainbow DQN algorithm in deep reinforcement learning to achieve continuous optimization and dynamic adaptation of task scheduling; Rainbow DQN can fully consider the dynamic changes in the environment during task scheduling based on the real-time state of the task and its network decision mechanism; through the adaptive mechanism in reinforcement learning, the system can adjust the scheduling strategy in real time in a complex and changing environment, optimize the decision-making process, and ensure efficient task scheduling and resource utilization under different conditions, thereby improving the overall scheduling performance and robustness of the system.

[0118] In an exemplary embodiment, the two-stage imaging satellite task scheduling method includes the following specific steps:

[0119] I、Problem decomposition: The imaging satellite task planning problem is divided into two sub-problems, namely the resource allocation problem and the task planning problem; this helps to simplify the complex scheduling problem and make it more operational;

[0120] II. Conflict-aware resource allocation: Utilize conflict-aware heuristic scheduling strategies for resource allocation; By comprehensively analyzing the spatio-temporal characteristics of tasks, accurately identify the time window and resource demand of each task, to achieve more efficient resource scheduling; Specifically, first calculate the conflict degree between tasks, evaluate the potential task conflict risk, and according to the priority and resource demand of the task, construct an optimized resource allocation scheme; In order to ensure the optimality of task scheduling, this method uses backtracking search technology, gradually adjusts and verifies different task allocation strategies, to preferentially meet high-priority tasks and minimize subsequent task conflicts; In order to further improve resource utilization efficiency, the system sets a conflict threshold to filter out tasks with high conflict and low overall system benefit, thereby maximizing resource utilization efficiency while ensuring task scheduling quality;

[0121] III. DRL-driven task scheduling decision: This embodiment introduces the Rainbow DQN algorithm, which further improves the adaptability and flexibility of task scheduling through the dynamic learning and optimization capabilities of deep reinforcement learning; The Rainbow DQN algorithm is based on the real-time state of the task, including the priority of the task, the availability of resources, and the changes in the external environment, and uses a deep reinforcement learning framework for decision-making and scheduling, so that it can adaptively adjust in complex and dynamic environments; Through continuous learning and self-optimization, the algorithm adjusts the task scheduling strategy in real time to cope with the time-varying relationship between tasks and resource constraints, ensuring that the system can maintain efficient scheduling performance and resource utilization under various changing conditions, thereby achieving long-term optimization of task scheduling.

[0122] In an exemplary embodiment, the two-stage imaging satellite task scheduling method has the following specific steps:

[0123] Step 1: Determine the imaging task planning model to be solved :

[0124] Set definition: task set , represents the number of tasks in the set; resource set , represents the number of satellite resources available in the scene; task occupies resources allowed visible time window set:

[0125] { ,…, }, =[ , ], ,

[0126] in, Indicates task Resource consumption The number of visible time windows allowed at that time; and They represent the tasks respectively Resource consumption The first allowed time The start and end times of each visible time window; each task It can be represented as:

[0127] ,

[0128] Among them, task reward value Task execution time Task storage consumption Task power consumption , Task required posture transition time ;definition For all that can be arranged in resources The set of tasks to be executed. To meet the observation task The required set of resources and Representing tasks The specific start and end times when selected for scheduling planning;

[0129] Objective function: Describe the task In resources The first The execution status over each visible time window is a Boolean variable. =1 indicates a task Assigned to the visible time window Because tasks differ in importance, different tasks are assigned different weights. Treating the task weights as the rewards for successful execution, this embodiment sets the optimization objective as maximizing these rewards during the scheduling and planning process. Therefore, the model... The objective function is:

[0130] ,

[0131] Model The following constraints need to be met:

[0132]

[0133]

[0134]

[0135]

[0136]

[0137]

[0138] Constraint (1) represents the task uniqueness constraint: each task can only be executed at most once in the planning schedule and can only be allocated within one visibility time window; constraint (2) represents the observation window and resource availability constraint: for each task in the planning schedule, if the task is allocated with a resource and the corresponding visibility time window, the observation time period of the task must fall completely within the allocated visibility time window; constraint (3) represents the minimum transition duration constraint: the same satellite resource needs to complete the transition from one observation attitude to another observation attitude within a certain time when executing two consecutive observation tasks, therefore the start time between the two tasks should satisfy the minimum transition duration constraint and the execution time segments do not overlap; constraint (4) represents the task visibility constraint: the specific start time and end time of the task planned to be scheduled must fall within the range of the corresponding scenario time window ; constraints (5) (6) represent the resource capability constraint: the storage capacity and energy consumption required by the task sequence allocated on each resource must not exceed the maximum data storage capacity and the upper limit of total power storage of the resource , to ensure that the resource will not be overloaded;

[0139] Step two, conflict-aware phase based on conflict reduction strategy:

[0140] In the first phase, the conflict-aware heuristic algorithm combines backtracking search and greedy strategy to ensure the balance of resource allocation; first, calculate all the visibility time windows of the task sequence to clarify the feasibility of each task in different time periods; map the calculated task time windows to the corresponding available resources, as shown in Figure 2 , in this way, analyze the spatiotemporal distribution characteristics of the task (whether the task is concentrated or dispersed); the analysis of this distribution characteristic is crucial for subsequent optimization decisions, which helps to assess the degree of resource conflict of all potential observation time periods in each time window, thereby providing guidance for further scheduling optimization; then, schedule the tasks that have no time conflict with other tasks through pre-scheduling strategy, i.e. for task and resource , if the task a certain time window in the set of visible time windows [ , ] with other tasks on the same resource , indicates that it can be scheduled in advance; scheduling tasks that meet the above conditions in advance can obtain a pre-scheduled task set

[0141] ,

[0142] Because these tasks have been determined not to conflict with other tasks in resource contention, they do not need to be considered in subsequent scheduling processes, which can reduce the search space of subsequent time window allocation; next, the tasks are arranged in descending order of priority, and the tasks with higher weights are given priority to find the best time period, and the tasks successfully allocated to the time window are put into the backtracking task list , the time period of these tasks meets the constraint condition and does not conflict with other tasks in the backtracking list; in this process, a backtracking search method is used to explore possible solutions, and when a resource conflict or a time window that does not meet the condition is encountered, the previous task can be returned to readjust the allocation of resource time windows to find a better allocation scheme; for tasks that fail to solve resource contention conflicts, they are put into the to-be-processed task list for further processing in subsequent stages; these tasks often fail to find a suitable allocation scheme immediately due to fierce resource competition or strong time window constraints; the union of the backtracking tasks and the to-be-processed tasks is denoted as for subsequent conflict reduction processing;

[0143] As shown in Figure 3 , when the task selects to execute in the visible time window , under the constraint of the task conversion duration, it may simultaneously have potential conflict impact on the visible time window of and ; therefore, for each visible time window , , , calculate the set of all time windows that may have potential resource contention conflicts with on the same resource:

[0144] ,

[0145] The conflict time slice of the visible time window with all time windows in the set can be represented as:​

[0146] ,

[0147] the visible time window of the task The timing conflict indicator of the task

[0148] ,

[0149] The conflict degree of the task is defined as the sum of timing conflicts of all time windows:

[0150] ,

[0151] The average timing conflict indicator and its corresponding standard deviation In order to realize the conflict reduction strategy, a threshold value is introduced, which can be adjusted according to the distribution of the task timing conflict indicator, and better reflect the timing correlation in task scheduling. The threshold value can be represented as:

[0152] ,

[0153] ,

[0154] ,

[0155] After calculating the threshold value, a greedy strategy based on task execution time, the weight that can be obtained after execution, and the timing conflict indicator is selected for comprehensive consideration, and the conflict degree of the time window is calculated:

[0156] ,

[0157] The time window with the minimum conflict degree is selected as the allocation time window of the task. In addition, conflict reduction decisions need to be made according to the preset conflict degree threshold. If the conflict degree of the minimum time window of a task exceeds the threshold, the task will be filtered from the set of tasks to be scheduled.

[0158] Step three, DRL driven task scheduling decision:

[0159] The resource allocation in the first stage provides the initial conditions for the task planning in the second stage. In the second stage, the task planning problem is abstracted as a Markov decision process (MDP). MDP is a continuous decision-making process of a random dynamic system based on Markov theory. It is a theoretical framework that realizes the goal through interactive learning. The Markov decision process with Markov property is the mathematical theoretical basis of reinforcement learning. The Markov decision process can be represented by a four-tuple​​ denotes, where S denotes the state space, denotes the action space, denotes the state transition function, denotes the reward function; the MDP decision process can be described by the following formula:

[0160] ,

[0161] where is the current state, is the next state, and are the action decisions taken in the state and respectively; is the immediate reward obtained by choosing the action in the current state ; denotes the probability of transitioning from the state to after choosing the action ; is the discount factor, indicating the degree of emphasis on future rewards; is the value function of the state , indicating the expected cumulative reward obtained by executing the task scheduling policy from the state ;

[0162] A process in which the agent interacts with the environment is called a time step, and the set of time steps is defined as ; at each time step, the agent observes the current state , selects an action according to the current state, and performs the action to interact with the environment; then, the agent receives the reward value and the next state returned by the environment, enters the next time step, and through continuous time step interaction, the agent can gradually learn and improve its decision policy until the termination condition is reached or the task is completed;

[0163] (1) State: The state space S contains all possible states, which is a description of the environment information, denoted as follows, denotes the size of the state space (i.e., the number of possible states):

[0164] ,

[0165] Each state consists of the attributes of the task and the resources:

[0166] , The task arriving at the first time step

[0167] is the start time, end time and duration of the time window on the resource to which the task is assigned; is the reward value obtained after the task is scheduled; is the remaining energy ratio and the remaining storage capacity ratio of the resource after the task is the task set after the first stage of processing, if the task is the last task, or the remaining energy ratio and the remaining storage capacity ratio of all resources have reached the upper limit at a certain time step is the termination state: (2) State transition (transition): let be the task set after the first stage of processing, if the task

[0168] is the last task, or the remaining energy ratio and the remaining storage capacity ratio of all resources have reached the upper limit at a certain time step is defined as the termination state: ,

[0169] ,

[0170] (3) Action: based on the state, the action represents the decision made at a certain state ; the action space contains all the available actions, usually represented as , where represents the size of the action space; the action is defined as the decision made for the task at the first time step:

[0171] ,

[0172] (4) Reward: the reward is the immediate reward obtained when the environment moves to a new state after performing the action , defined as ; the expected reward at the time step is:

[0173] ,

[0174] The expected total reward is:

[0175] ,

[0176] (3) Model training: based on the constructed MDP model, the multi-layer feedforward neural network is used to extract key features from the input state and generate the corresponding task scheduling strategy; the model training process is as follows:​​​​Figure 4 The core steps of training include, for each state The Q values of each possible action are calculated by the neural network as the input of the network, which represent the expected future return of taking each action in a given state; according to the calculated Q values, the agent will select an action according to a certain policy And interact with the environment to obtain the immediate reward And the information of the next state After each interaction with the environment, the agent will store the current experience sample Into the experience replay pool to provide data support for subsequent learning process;

[0177] Unlike traditional experience replay methods, the embodiment introduces a priority experience replay technique. Considering the importance of experience samples, when the buffer reaches a certain capacity, the system will assign a priority to each experience using the priority experience replay technique, which reflects the error of the current network in estimating the value function, usually evaluated by the time difference error (TD error). The calculation formula of TD error is as follows:

[0178]

[0179] Where, is the discount factor, and are the Q values of the target network and the current network respectively; TD error can measure the importance of experience samples in the current training; generally, experience samples with larger TD error have greater impact on the network's value estimation error, so they need to be sampled more frequently; define the priority As the absolute TD error size of each experience sample:

[0180] ,

[0181] Where is a small constant to avoid zero priority; through this calculation method, experience with larger TD error will get higher priority; next, sampling is performed according to the priority of the experience; the sampling probability of each experience Is determined by its priority and normalized by the following formula:

[0182] ,

[0183] Where is the total number of experiences stored in the experience pool, is the priority of the i-th experience, is the priority of the i-th experience, represents the sum of experienced priority; through normalization processing, it is ensured that all sampling probabilities sum to 1; in priority experience replay, sampling not only selects experience samples according to priority, but also considers the relative importance of experience; for this purpose, a weighting coefficient is introduced , which is used to adjust the influence of each experience on network training; the calculation formula of the weighting coefficient is as follows:

[0184] ,

[0185] wherein, is a hyperparameter for controlling the degree of importance adjustment of samples, which gradually increases during the training process to reduce the bias of sampling; at this time, the experience with higher priority will be sampled first, and the experience with lower priority will be gradually replaced, thereby maintaining the effectiveness and diversity of experience samples in the buffer pool; through this priority experience replay mechanism, the experience with larger TDerror will have a higher sampling probability, thereby guiding the agent to focus on important experience and accelerating the learning process and improving training efficiency; at the same time, through the weighted sampling mechanism, the priority of different experiences can be effectively balanced, avoiding excessive attention to certain specific samples, thereby ensuring the stability and convergence of the training process;

[0186] Next, the loss function is calculated, and the gradient descent method is used to update the parameters of the current network; the loss function is defined as the square of TD error, that is:

[0187] ,

[0188] The optimization goal of the loss function is to minimize the square of TD error, so that the Q value of the current network is as close as possible to the estimated value of the target network, thereby improving the decision accuracy and efficiency of the agent in the given task scheduling; through the back propagation algorithm and gradient update, the parameters of the network will be gradually optimized, so that the agent can generate more accurate and efficient scheduling decisions in complex satellite scheduling problems; as Figure 5 shown in Fig. 1 is a structure diagram of an imaging satellite task scheduling framework.

[0189] Through the training process, the imaging satellite is efficiently scheduled in resource allocation and task planning. The CAS-D2RL framework effectively combines the resource allocation phase and the task execution phase, can ensure the success rate of the task, and improve the adaptability of the system in a complex environment through a dynamic optimization mechanism. The framework not only optimizes the allocation of resources and improves the efficiency of resource utilization, but also maximizes the benefits of the task through accurate task scheduling strategies. Based on this, the scheduling framework proposed by the present application provides autonomous decision-making capabilities for imaging satellites, enabling them to make real-time and accurate scheduling decisions when faced with dynamic and uncertain environments, thereby enhancing the autonomy and adaptability of the system in a changing environment.

[0190] The embodiment provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the two-stage imaging satellite task scheduling method. The storage medium can be a disk, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0191] The embodiment provides a computer device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the two-stage imaging satellite task scheduling method are implemented.

[0192] As Figure 6As shown, the computer device 120 can include at least one processor 121, such as a central processing unit (CPU), at least one communication interface 123, a memory 124, and at least one communication bus 122. The communication bus 122 is configured to enable communication between these components. The communication interface 123 can include a display, a keyboard, and optionally the communication interface 123 can further include a standard wired interface, a wireless interface. The memory 124 can be a high-speed random access memory (RAM), and can also be a non-volatile memory, such as at least one disk memory. The memory 124 can also be at least one storage device located away from the aforementioned processor 121. The memory 124 stores an application program, and the processor 121 invokes the program code stored in the memory 124 to execute any of the above method steps. The communication bus 122 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 122 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6The bus 123 is used to connect the above-mentioned elements in the system 120, and only one bus is represented, but it does not mean that there is only one bus or only one type of bus. Among them, the memory 124 can include volatile memory such as random-access memory (RAM); the memory can also include non-volatile memory such as flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 124 can also include a combination of the above-mentioned types of memory. Among them, the processor 121 can be a central processing unit (CPU), a network processor (NP) or a combination of CPU and NP. The processor 121 can further include a hardware chip. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. Alternatively, the memory 124 is also used to store program instructions. The processor 121 can call the program instructions to implement the two-stage imaging satellite task scheduling method of the embodiment.

[0193] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative but not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection of the present application.

Claims

1. A two-phase imaging satellite mission scheduling method, characterized in that, The method comprises the following steps: S1: constructing an imaging task planning model according to imaging tasks and satellite resources; S2: obtaining a pre-scheduling task set, a backtracking task list, a to-be-processed task list and a resource allocation scheme by using a conflict-aware heuristic scheduling strategy according to the imaging task planning model; S3: obtaining a scheduling scheme by using a RainbowDQN algorithm according to the pre-scheduling task set, the backtracking task list, the to-be-processed task list and the resource allocation scheme; Step S2 specifically comprises: S21: obtaining a task time window according to the imaging task planning model, mapping the task time window to a corresponding available resource to obtain a spatiotemporal distribution characteristic of the task; S22: obtaining a pre-scheduling task set by using a pre-scheduling strategy according to the spatiotemporal distribution characteristic of the task; S23: obtaining a backtracking task list and a to-be-processed task list by using a backtracking search method according to the spatiotemporal distribution characteristic of the task and the pre-scheduling task set; S24: calculating a conflict degree and setting a conflict degree threshold, and obtaining a resource allocation scheme according to the conflict degree, the conflict degree threshold, the backtracking task list and the to-be-processed task list; Step S3 specifically comprises: S31: constructing an experience priority, an experience sampling probability, an experience weighting coefficient and a loss function; S32: training a Markov decision process model by using the experience priority, the experience sampling probability, the experience weighting coefficient, the loss function and the RainbowDQN algorithm according to the pre-scheduling task set, the backtracking task list, the to-be-processed task list and the resource allocation scheme, to obtain a trained Markov decision process model; S33: obtaining a scheduling scheme by using the trained Markov decision process model.

2. The two-phase imaging satellite task scheduling method according to claim 1, characterized in that, Step S1 specifically comprises: constructing an imaging task planning model according to imaging tasks and satellite resources, such as the formula: ， ， ， ， ， ， ， ， ， ， ， ， ， ， wherein, is a set of tasks, is a th observation task, represents the number of tasks in the set, is a set of resources, is a th resource, represents the number of satellite resources available in the scenario, is a task occupying a resource allowed visible time window set, denotes the th visible time window allowed for a task occupying a resource , denotes the number of visible time windows allowed for a task occupying a resource , and denote the start time and end time of the th visible time window allowed for a task occupying a resource , is a task reward value, is a task execution duration, is a task storage consumption, is a task power consumption, is a task attitude translation duration, is a set of all tasks that can be scheduled to be executed on a resource , is a set of resources satisfying the observation task requirements; is an objective function of the imaging task planning model, is a Boolean variable indicating the execution status of a task on the th visible time window on a resource , denotes the maximization, and denote the specific start time and end time of a task when it is selected for scheduling planning; and denote the start time and end time of the scenario time window to which a task belongs, denotes the maximum data storage capacity of the th resource, denotes the total power storage upper limit of the th resource.

3. The two-phase imaging satellite task scheduling method according to claim 2, characterized in that, Step S24 specifically comprises: S241: calculating a conflict degree of a task, such as the formula: ， ， ， ， ， ， ， wherein, is the set of all possible visible time windows on the same resource is the set of time windows that generate potential resource contention conflicts, is the visible time window for is the set of all time windows in is the conflict time slice intersecting all time windows in is the pose translation duration required for the corresponding task, is the task is the timing conflict indicator for the visible time window of the task is the conflict degree of the task is the backtracking task list, is the pending task list, is the union of the backtracking task and the pending task.​ S242: obtaining an average timing conflict index and a corresponding standard deviation and a conflict degree threshold according to the conflict degree of the task; S243: obtaining a time window conflict degree by using a greedy strategy according to the conflict degree threshold, a task execution time, an obtainable weight after execution and a timing conflict index of a visible time window; S244: selecting a time window with the minimum conflict degree as an allocation time window of the task according to the time window conflict degree, and filtering tasks with a conflict degree of the minimum time window exceeding the conflict degree threshold from a to-be-scheduled task set to obtain a resource allocation scheme.

4. The two-phase imaging satellite task scheduling method according to claim 3, characterized in that, Step S242 specifically comprises: obtaining an average timing conflict index and a corresponding standard deviation and a conflict degree threshold according to the conflict degree of the task, such as the formula: ， ， ， wherein, is an average timing conflict indicator, is a number of conflicting tasks, is a corresponding standard deviation, is a conflict degree threshold value.

5. The two-phase imaging satellite task scheduling method according to claim 3, characterized in that, Step S243 specifically comprises: obtaining a time window conflict degree by using a greedy strategy according to the conflict degree threshold, a task execution time, an obtainable weight after execution and a timing conflict index of a visible time window, such as the formula: ， wherein, is the time window of the conflict.

6. The two-phase imaging satellite task scheduling method according to claim 1, characterized in that, Step S31 specifically comprises: constructing an experience priority, an experience sampling probability, an experience weighting coefficient and a loss function, such as the formula: ， ， ， ， ， in, For the first Prioritize the experience points; For the first The time difference error of the experience; It is a small constant used to avoid cases with zero priority; For the first The sampling probability of the empirical evidence, It is the total number of experience points stored in the experience pool. The expected total return of the Markov decision process model; As a discount factor, and These are the Q values ​​of the target network and the current network, respectively; and The first The environmental state at each time step and state-based pass The chosen action and The first The environmental state at each time step and state-based pass The chosen action For the maximum value index function, For the first The weighting coefficients of the empirical values; It is a hyperparameter used to control the degree of importance adjustment of samples. It is gradually increased during training to reduce sampling bias. This is the loss function.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the steps of the two-stage imaging satellite task scheduling method according to any one of claims 1-6.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the two-stage imaging satellite task scheduling method according to any one of claims 1-6.

Images