Two-stage imaging satellite task scheduling method, storage medium and computer equipment

Abstract

The invention discloses a two-stage imaging satellite task scheduling method, a storage medium and computer equipment, and relates to the technical field of aerospace engineering.The two-stage imaging satellite task scheduling method mainly comprises the steps that an imaging task planning model is constructed according to imaging tasks and satellite resources; according to the imaging task planning model, obtaining a pre-scheduling task set, a backtracking task list, a to-be-processed task list and a resource allocation scheme by using a conflict-aware heuristic scheduling strategy; and obtaining a scheduling scheme by using a RainbowDQN algorithm. By implementing the two-stage imaging satellite task scheduling method, the storage medium and the computer equipment provided by the invention, the response efficiency and flexibility of satellite task scheduling can be improved.

Description

Technical Field

[0001] The present invention relates to the field of aerospace engineering technology, and more specifically to a two-stage imaging satellite task scheduling method, storage medium and computer equipment. Background Art

[0002] With the rapid development of aerospace technology, imaging satellites have become an indispensable observation tool in the field of remote sensing. Relying on the high-precision imaging equipment on board, imaging satellites can perform high-resolution observations of the earth's surface. They are widely used in agricultural monitoring, forestry protection, urban planning, and marine environment monitoring, and have played an important role in promoting the country's economic development and social progress. When imaging satellites effectively perform remote sensing tasks, with limited satellite resources and the rapid growth of user demand, how to plan a large number of user tasks to formulate a reasonable and effective task observation sequence is crucial to give full play to the advantages of imaging satellites and provide high-quality space-based information services. In the algorithm research of imaging satellite task planning, there are many types of algorithms for solving complex task scheduling problems. From classical algorithms to modern intelligent optimization methods, researchers have proposed a variety of solutions, but these methods usually face the problem of high computational complexity when dealing with complex task scheduling. Especially when the task scale is large and there are many resource constraints, they face the dilemma of "solution quality" and "solution efficiency". Some methods can get results in a shorter time at the expense of the quality of the solution; while other methods require repeated iterative searches to find high-quality solutions, resulting in huge time consumption. With the improvement of artificial intelligence technology and the computing power of satellite platforms, the demand for the application of autonomous planning and decision-making technologies in imaging satellites is becoming more and more urgent. Summary of the invention

[0003] The purpose of the present invention is to provide a two-stage imaging satellite mission scheduling method, storage medium and computer equipment, which can improve the response efficiency and flexibility of satellite mission scheduling.

[0004] The present invention provides a two-stage imaging satellite task scheduling method, comprising the following steps: S1: constructing an imaging task planning model according to imaging tasks and satellite resources; S2: obtaining a pre-scheduled task set, a backtracking task list, a to-be-processed task list and a resource allocation scheme according to the imaging task planning model by using a conflict-aware heuristic scheduling strategy; S3: obtaining a scheduling scheme according to the pre-scheduled task set, the backtracking task list, the to-be-processed task list and the resource allocation scheme by using a RainbowDQN algorithm.

[0005] Furthermore, step S1 specifically includes: constructing an imaging mission planning model according to the imaging mission and satellite resources, such as the formula: , , , , , , , , , , , , , , in, For the task set, For the Observation tasks, Represents the number of tasks in the collection, For resource collection, For the resources, Represents the number of satellite resources available in the scene, For the task Occupied resources The set of allowed visible time windows, Representation Task Occupied resources The allowed Visible time window, Representation Task Occupied resources The number of visible time windows allowed when and Respectively represent the tasks Occupied resources The first The start and end time of the visible time window, is the task benefit value, is the execution time required for the task, For task storage consumption, The power consumption of the task, The time required for the task to transform. For all resources that can be arranged The set of tasks executed on To meet the observation task The set of resources required; The objective function of the imaging task planning model is, Representation Task In Resources On A Boolean variable representing the execution status over a visible time window, represents maximization, and Respectively represent tasks The specific start and end time when selected for scheduling planning; and Respectively represent tasks The start and end time of the scene time window. Indicates The maximum data storage capacity of a resource, Indicates and The total power storage limit of each resource.

[0006] Furthermore, step S2 specifically includes: S21: according to the imaging task planning model, the task time window is obtained, and the task time window is mapped to the corresponding available resources to obtain the spatiotemporal distribution characteristics of the task; S22: according to the spatiotemporal distribution characteristics of the task, a pre-scheduling strategy is used to obtain a pre-scheduled task set; S23: according to the spatiotemporal distribution characteristics of the task and the pre-scheduled task set, a backtracking task list and a pending task list are obtained using a backtracking search method; S24: the degree of conflict is calculated and a conflict degree threshold is set, and a resource allocation plan is obtained according to the degree of conflict, the conflict degree threshold, the backtracking task list and the pending task list.

[0007] Furthermore, step S24 specifically includes: S241: calculating the conflict degree of the task, such as formula: , , , , , , , in, For all possible and visible time windows on the same resource The set of time windows that generate potential resource contention conflicts, Visible time window and The conflicting time slices where all time windows in the set intersect, is the posture conversion time required for the corresponding task, For the task Visible time window The timing violation indicator, For the task The degree of conflict, To backtrack the task list, A list of pending tasks. is the union of the backtracking task and the tasks to be processed; S242: according to the task conflict degree, the average timing conflict index and its corresponding standard deviation and conflict degree threshold are obtained; S243: according to the conflict degree threshold, the task execution time, the weight that can be obtained after execution and the timing conflict index of the visible time window, the time window conflict degree is obtained by using the greedy strategy; S244: according to the time window conflict degree, the time window with the minimum conflict degree is selected as the allocation time window of the task, and the tasks whose conflict degree of the minimum time window in the task exceeds the conflict degree threshold are filtered from the set of tasks to be scheduled to obtain a resource allocation plan.

[0008] Further, step S242 specifically includes: according to the conflict degree of the task, obtaining the average timing conflict index and its corresponding standard deviation and conflict degree threshold, such as the formula: , , , in, is the average timing conflict indicator, is the number of conflicting tasks, is the corresponding standard deviation, is the conflict threshold.

[0009] Further, step S243 specifically includes: according to the conflict degree threshold, the task execution time, the weight that can be obtained after execution and the timing conflict index of the visible time window, the greedy strategy is used to obtain the time window conflict degree, such as the formula: , in, For time window degree of conflict.

[0010] Furthermore, step S3 specifically includes: S31: constructing empirical priority, empirical sampling probability, empirical weighting coefficient and loss function; S32: according to the pre-scheduled task set, the backtracking task list, the pending task list and the resource allocation plan, using the empirical priority, empirical sampling probability, empirical weighting coefficient, loss function and RainbowDQN algorithm, the Markov decision process model is trained to obtain a trained Markov decision process model; S33: using the trained Markov decision process model to obtain a scheduling plan.

[0011] Further, step S31 specifically includes: constructing experience priority, experience sampling probability, experience weighting coefficient and loss function, such as formula: , , , , , in, For the The priority of the experience; For the The time difference error of the experience; is a small constant used to avoid the case of zero priority; For the The sampling probability of an experience, is the total amount of experience stored in the experience pool, is the expected total return of the Markov decision process model; is the discount factor, and are the Q values ​​of the target network and the current network respectively; and Respectively The environment state at time step and the state-based pass Select the action, is the maximum value index function, For the The weighting coefficient of the experience; It is a hyperparameter used to control the degree of sample importance adjustment, which gradually increases during training to reduce sampling bias; is the loss function.

[0012] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned two-stage imaging satellite task scheduling method are implemented.

[0013] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned two-stage imaging satellite task scheduling method are implemented.

[0014] The two-stage imaging satellite task scheduling method, storage medium and computer device provided by the present invention have the following beneficial effects: The present invention proposes a two-stage imaging satellite task scheduling framework based on conflict-aware heuristic algorithm and Rainbow DQN (hereinafter referred to as "CAS-D2RL framework"). In the first stage, through the conflict-aware heuristic scheduling strategy, the temporal and spatial distribution characteristics of the task are deeply analyzed, and the potential risks of resource contention and task conflict are accurately identified and quantified, so as to effectively suppress and optimize the conflict in the resource allocation stage, avoid the occurrence of resource bottlenecks in advance, and minimize the conflicts between tasks, providing a more stable and reliable foundation for the subsequent task scheduling and planning stages. In the second stage, Rainbow in Deep Reinforcement Learning (DRL) is introduced. The DQN algorithm simulates the dynamic decision-making process, uses historical data and current environmental information, and comprehensively considers the dynamic changes of the environment in the task scheduling process based on the real-time status of the task and its network decision-making mechanism. Through the adaptive mechanism in reinforcement learning, it continuously learns and updates strategies, adjusts the scheduling strategy in real time in complex and changing multi-task scenarios and environments, optimizes the decision-making process, and realizes dynamic optimal scheduling decisions. It overcomes the shortcomings of traditional methods in complexity and real-time performance, alleviates the complex resource allocation and task conflicts faced in the imaging satellite mission planning process, realizes continuous optimization and dynamic adaptation of task scheduling under different conditions, realizes dynamic real-time response to observation mission requirements, improves the response efficiency and flexibility of satellite task scheduling, and improves the autonomy and resource utilization efficiency of satellites. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 is a flow chart of a two-stage imaging satellite task scheduling method provided by the present invention; Figure 2 It is a distribution diagram of the visible time window of the task provided by the present invention; Figure 3 It is a schematic diagram of the time window of resource contention conflict provided by the present invention; Figure 4 It is a diagram of the Rainbow DQN training structure provided by the present invention; Figure 5 This is a diagram of the imaging satellite task scheduling framework provided by the present invention; Figure 6 It is a structural block diagram of the computer device provided by the present invention. DETAILED DESCRIPTION

[0016] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0017] Figure 1A schematic diagram of a two-stage imaging satellite task scheduling method of this embodiment is shown. In this embodiment, the two-stage imaging satellite task scheduling method includes the following steps: S1: Construct an imaging mission planning model based on imaging missions and satellite resources; In an exemplary embodiment, step S1 specifically includes: constructing an imaging mission planning model according to the imaging mission and satellite resources, such as formula: , , , , , , , , , , , , , , in, For the task set, For the Observation tasks, Represents the number of tasks in the collection, For resource collection, For the resources, Represents the number of satellite resources available in the scene, For the task Occupied resources The set of allowed visible time windows, Representation Task Occupied resources The allowed Visible time window, Representation Task Occupied resources The number of visible time windows allowed when and Respectively represent the tasks Occupied resources The first The start and end time of the visible time window, is the task benefit value, is the execution time required for the task, For task storage consumption, The power consumption of the task, The time required for the task to transform. For all resources that can be arranged The set of tasks executed on To meet the observation task The set of resources required; The objective function of the imaging task planning model is, Representation Task In Resources On A Boolean variable representing the execution status over a visible time window, represents maximization, and Respectively represent tasks The specific start and end time when selected for scheduling planning; and Respectively represent tasks The start and end time of the scene time window. Indicates The maximum data storage capacity of a resource, Indicates and The total power storage limit of each resource; S2: According to the imaging task planning model, the conflict-aware heuristic scheduling strategy is used to obtain the pre-scheduled task set, the backtracking task list, the pending task list and the resource allocation plan; In an exemplary embodiment, step S2 specifically includes: S21: According to the imaging task planning model, the task time window is obtained, and the task time window is mapped to the corresponding available resources to obtain the spatiotemporal distribution characteristics of the task; S22: according to the temporal and spatial distribution characteristics of the tasks, a pre-scheduling strategy is used to obtain a pre-scheduling task set; In an exemplary embodiment, according to the temporal and spatial distribution characteristics of the tasks, a pre-scheduling strategy is used to obtain a pre-scheduling task set, such as the formula: , in, A collection of pre-scheduled tasks; S23: according to the temporal and spatial distribution characteristics of the tasks and the pre-scheduled task set, a backtracking search method is used to obtain a backtracking task list and a to-be-processed task list; As an exemplary embodiment, in step S23, the tasks are arranged in descending order of priority, the best time period is found for the tasks with higher weights first, and the tasks successfully assigned to the time window are put into the backtracking task list. , the time periods of these tasks meet the constraints and do not conflict with other tasks in the backtracking list; in this process, the backtracking search method is used to explore possible solutions. When encountering resource conflicts or time windows that do not meet the conditions, you can return to the previous task and readjust the allocation of resource time windows to find a better allocation solution; for tasks that fail to resolve resource contention conflicts, put them into the pending task list , so that they can be further processed in the subsequent stages; these tasks often fail to find a suitable allocation solution immediately due to fierce resource competition or strong time window constraints; the union of the backtracking task and the pending task is recorded as For subsequent conflict mitigation processing; S24: Calculate the conflict degree and set the conflict degree threshold, and obtain a resource allocation plan according to the conflict degree, the conflict degree threshold, the backtracking task list and the pending task list; In an exemplary embodiment, step S24 specifically includes: S241: Calculate the conflict degree of the task, such as the formula: , , , , , , , in, For all possible and visible time windows on the same resource The set of time windows that generate potential resource contention conflicts, Visible time window and The conflicting time slices where all time windows in the set intersect, is the posture conversion time required for the corresponding task, For the task Visible time window The timing violation indicator, For the task The degree of conflict, To backtrack the task list, A list of pending tasks. It is the union of the backtracking tasks and the tasks to be processed; S242: according to the conflict degree of the task, obtaining an average timing conflict index and its corresponding standard deviation and conflict degree threshold; In an exemplary embodiment, step S242 specifically includes: obtaining an average timing conflict index and its corresponding standard deviation and conflict threshold according to the conflict degree of the task, such as the formula: , , , in, is the average timing conflict indicator, is the number of conflicting tasks, is the corresponding standard deviation, is the conflict degree threshold; S243: Obtaining the time window conflict degree using a greedy strategy according to the conflict degree threshold, the task execution time, the weight that can be obtained after execution, and the timing conflict index of the visible time window; In an exemplary embodiment, step S243 specifically includes: according to the conflict degree threshold, the task execution time, the weight that can be obtained after execution and the timing conflict index of the visible time window, using a greedy strategy, obtaining the time window conflict degree, such as the formula: , in, For time window degree of conflict; S244: According to the time window conflict degree, the time window with the minimum conflict degree is selected as the allocation time window of the task, and the tasks whose conflict degree of the minimum time window exceeds the conflict degree threshold are filtered from the set of tasks to be scheduled to obtain a resource allocation plan; S3: Based on the pre-scheduled task set, the backtracking task list, the pending task list and the resource allocation plan, the RainbowDQN algorithm is used to obtain the scheduling plan; In an exemplary embodiment, step S3 specifically includes: S31: Construct experience priority, experience sampling probability, experience weighting coefficient and loss function; In an exemplary embodiment, step S31 specifically includes: constructing experience priority, experience sampling probability, experience weighting coefficient and loss function, such as formula: , , , , , in, For the The priority of the experience; For the The time difference error of the experience; is a small constant used to avoid the case of zero priority; For the The sampling probability of an experience, is the total amount of experience stored in the experience pool, is the expected total return of the Markov decision process model; is the discount factor, and are the Q values ​​of the target network and the current network respectively; and Respectively The environment state at time step and the state-based pass Select the action, is the maximum value index function, For the The weighting coefficient of the experience; It is a hyperparameter used to control the degree of sample importance adjustment, which gradually increases during training to reduce sampling bias; is the loss function; S32: According to the pre-scheduled task set, the backtracking task list, the pending task list and the resource allocation plan, the Markov decision process model is trained using the empirical priority, the empirical sampling probability, the empirical weighting coefficient, the loss function and the RainbowDQN algorithm to obtain a trained Markov decision process model; S33: Use the trained Markov decision process model to obtain a scheduling plan.

[0018] In an exemplary embodiment, the two-stage imaging satellite task scheduling method can be implemented in the following manner: 1. Constructing a CAS-D2RL framework: The primary purpose of this embodiment is to construct a CAS-D2RL scheduling framework that combines conflict perception with RainbowDQN. This framework aims to optimize resource allocation and conflict resolution in the task scheduling process and improve the performance of the overall scheduling system by integrating conflict-aware heuristic strategies and deep reinforcement learning methods. 2. Resource allocation method based on conflict perception: In the first stage of scheduling, this embodiment proposes a heuristic scheduling strategy based on conflict perception for the resource allocation problem in imaging satellite missions. This strategy analyzes the spatiotemporal distribution characteristics of tasks in depth, accurately identifies and quantifies the potential risks of resource contention and task conflicts, and thus effectively suppresses and optimizes conflicts in the resource allocation stage. This method can avoid the emergence of resource bottlenecks in advance and minimize conflicts between tasks, providing a more stable and reliable foundation for the subsequent task scheduling and planning stages. 3. DRL-driven task scheduling decision: In the second stage of scheduling, this embodiment introduces the Rainbow DQN algorithm in deep reinforcement learning to achieve continuous optimization and dynamic adaptation of task scheduling; Rainbow DQN can comprehensively consider the dynamic changes of the environment during task scheduling based on the real-time status of the task and its network decision-making mechanism; through the adaptive mechanism in reinforcement learning, the system can adjust the scheduling strategy in real time in a complex and changing environment, optimize the decision-making process, ensure efficient task scheduling and resource utilization under different conditions, and thus improve the overall scheduling performance and robustness of the system.

[0019] In an exemplary embodiment, the two-stage imaging satellite task scheduling method specifically includes the following steps: 1. Problem decomposition: Divide the imaging satellite mission planning problem into two sub-problems, namely the resource allocation problem and the mission planning problem; this helps to simplify the complex scheduling problem and make it more operational; 2. Conflict-aware resource allocation: Utilize conflict-aware heuristic scheduling strategies to allocate resources; By comprehensively analyzing the spatiotemporal characteristics of tasks, accurately identify the time window and resource requirements of each task to achieve more efficient resource scheduling; Specifically, firstly, by calculating the degree of conflict between tasks, evaluate the potential risk of task conflict, and build an optimized resource allocation plan based on the priority and resource requirements of tasks; To ensure the optimality of task scheduling, this method uses backtracking search technology to gradually adjust and verify different task allocation strategies to give priority to high-priority tasks and minimize conflicts between subsequent tasks; To further improve resource utilization efficiency, the system sets a conflict threshold to filter out tasks with high conflict and small contribution to the overall system benefits, thereby maximizing resource utilization efficiency while ensuring the quality of task scheduling; 3. DRL-driven task scheduling decision: This embodiment introduces the Rainbow DQN algorithm, which further improves the adaptability and flexibility of task scheduling through the dynamic learning and optimization capabilities of deep reinforcement learning; the Rainbow DQN algorithm is based on the real-time status of the task, including the priority of the task, resource availability, and changes in the external environment, and uses a deep reinforcement learning framework for decision-making and scheduling, so that it can make adaptive adjustments in complex and dynamic environments; the algorithm adjusts the task scheduling strategy in real time through continuous learning and self-optimization to cope with the time-varying relationship and resource constraints between tasks, ensuring that the system can still maintain efficient scheduling performance and resource utilization under various changing conditions, thereby achieving long-term optimization of task scheduling.

[0020] In an exemplary embodiment, the two-stage imaging satellite task scheduling method includes the following specific steps: Step 1: Determine the imaging task planning model to be solved : Collection definition: Task collection , Represents the number of tasks in a collection; resource collection , Represents the number of satellite resources available in the scenario; Mission Occupied resources The set of allowed visible time windows: { ,…, }, =[ , ], , in, Representation Task Occupied resources The number of visible time windows allowed when and Respectively represent the tasks Occupied resources The first The start and end time of the visible time window; each task can be expressed as: , Among them, the task benefit value , the time required to execute the task , Task storage consumption , Task power consumption ,The time required for the posture transformation of the task ;definition For all resources that can be arranged The set of tasks executed on To meet the observation task The requested resource set, and Respectively represent tasks The specific start and end time when selected for scheduling planning; Objective function: remember the task In Resources On The execution status on the visible time window is a Boolean variable. =1 indicates task Assigned to visible time window ; Due to the differences in the importance of tasks, different tasks are assigned different weights; the weight of the task is regarded as the benefit brought by successful execution. In this embodiment, the optimization goal is set to maximize the benefit in the scheduling planning process, then the model The objective function is: , Model The constraints that need to be met are as follows:

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] Constraint (1) represents the task uniqueness constraint: each task It can only be executed once during the planning and scheduling process, and can only be allocated within a visible time window; Constraint (2) represents the observation window and resource availability constraints: For each task in the planning and scheduling process, if it is a task Resources allocated and the corresponding visible time window, the observation time period of the task must fall completely within the allocated visible time window; constraint (3) represents the minimum transition time constraint: when the same satellite resource performs two consecutive observation tasks, it needs to complete the transition from one observation attitude to another within a certain period of time. Therefore, the start time between the two tasks should meet the minimum transition time constraint, and there is no overlap in the execution time segments; constraint (4) represents the task visibility constraint: the task The specific start time of the scheduled schedule and end time Must fall within the scene time window [ ]; Constraints (5) and (6) represent resource capacity constraints: the storage capacity and energy consumption required by the task sequence assigned to each resource must not exceed the maximum data storage capacity of the resource and total power storage limit , ensuring that resources are not overloaded; Step 2: Conflict perception phase based on conflict reduction strategies: In the first stage, a conflict-aware heuristic algorithm is combined with backtracking search and greedy strategy to ensure the balance of resource allocation. First, all visible time windows of the task sequence are calculated to clarify the feasibility of each task in different time periods. The calculated task time windows are mapped to the corresponding available resources, such as Figure 2 As shown in the figure, in this way, the temporal and spatial distribution characteristics of tasks (whether the tasks are concentrated or discretely distributed) are analyzed; the analysis of such distribution characteristics is crucial for subsequent optimization decisions, and helps to evaluate the degree of resource conflict in all potential observation time periods in each time window, thereby providing guidance for further scheduling optimization; then, the pre-scheduling strategy is used to schedule tasks that have no time conflicts with other tasks, that is, for tasks and resources , if the task A time window in the set of visible time windows =[ , ] and other tasks In the same resource All time windows on do not overlap, Indicates that it can be scheduled in advance; by scheduling the tasks that meet the above conditions in advance, you can get a set of pre-scheduled tasks : , Because these tasks have been determined not to conflict with other tasks in resource contention, they do not need to be considered in the subsequent scheduling process, which can reduce the search space for subsequent time window allocations. Next, the tasks are sorted in descending order of priority, and the best time period is found for tasks with higher weights first. Tasks that are successfully assigned to time windows are placed in the backtracking task list. , the time periods of these tasks meet the constraints and do not conflict with other tasks in the backtracking list; in this process, the backtracking search method is used to explore possible solutions. When encountering resource conflicts or time windows that do not meet the conditions, you can return to the previous task and readjust the allocation of resource time windows to find a better allocation solution; for tasks that fail to resolve resource contention conflicts, put them into the pending task list , so that they can be further processed in the subsequent stages; these tasks often fail to find a suitable allocation solution immediately due to fierce resource competition or strong time window constraints; the union of the backtracking task and the pending task is recorded as For subsequent conflict mitigation processing; like Figure 3 As shown, when the task Select in the visible time window When executing on the same task, under the constraint of task switching time, Visible time window and Potentially conflicting effects occur; so for each visible time window , , , calculate all possible A set of time windows that may cause potential resource contention conflicts: , Visible time window and The conflicting time slices of all time windows in the set that intersect can be expressed as: , Task Visible time window The timing violation index can be expressed as: , Task Degree of conflict Defined as the sum of timing violations in all time windows: , Calculate the average timing violation metric and its corresponding standard deviation ,In order to implement the conflict reduction strategy, a threshold is introduced. The threshold can be adjusted according to the distribution of task timing conflict indicators to better reflect the timing association in task scheduling. The threshold can be expressed as: , , , After calculating the threshold, a greedy strategy is selected based on the task execution time, the weight that can be obtained after execution, and the timing conflict index to calculate the time window. Degree of conflict: , The time window with the minimum conflict degree is selected as the time window for task allocation. In addition, a conflict reduction decision is made based on a preset conflict degree threshold. That is, if the conflict degree of the minimum time window of a task exceeds the threshold, the task is filtered out from the set of tasks to be scheduled. Step 3: DRL-driven task scheduling decision: The resource allocation in the first stage provides the initial conditions for the task planning in the second stage. In the second stage, this embodiment abstracts the task planning problem into a Markov decision process (MDP). MDP is a continuous decision process for random dynamic systems based on Markov theory. It is a theoretical framework for achieving goals through interactive learning. The Markov decision process with Markov properties is the mathematical theoretical basis of reinforcement learning. The Markov decision process can be composed of a four-tuple express, where denotes the state space, represents the action space, represents the state transition function, Represents the reward function; the MDP decision process can be described by the following formula: , in is the current state, is the next state, and In the state and Decisions on actions to take; is in the current state Select Action Immediate benefits gained; Indicates the choice of action Back from state Transfer to probability; is the discount factor, which indicates the importance attached to future earnings; Yes Status The value function of The expected cumulative benefits from starting to execute the task scheduling strategy; A process of interaction between the agent and the environment is called a time step. The set of time steps is defined as ; At each time step, the agent observes the current state , select an action based on the current state , execute the action to interact with the environment; then, the agent receives the benefit value returned by the environment and the next state , enter the next time step. Through continuous time step interactions, the agent can gradually learn and improve its decision-making strategy until it reaches the termination condition or completes the task; (1) State: The state space S contains all possible states and is a description of the environment information, which is expressed as follows: Represents the size of the state space (i.e. the number of possible states): , Each state consists of properties of tasks and resources: , It is in The task that arrives at time steps In the allocated resources The start time, end time and duration of the task in the time window above; It's a task The reward value that can be obtained after being scheduled; It's a task After that, the remaining energy ratio and remaining storage capacity ratio of the resource; (2) State transition: is the set of tasks processed in the first stage. If the task is the last task, or at a certain time step When the remaining energy ratio and remaining storage capacity ratio of all resources have reached the upper limit, it is defined as the terminal state: , (3) Action: based on state, action Indicates the decision taken in a certain state. ; The action space contains all optional actions, usually expressed as ,in represents the size of the action space; the action is defined as time step for task Decisions made: , (4) Benefits: Rewards It means in the state Execute an action After that, the environment moves to a new state The immediate reward obtained when ;exist time steps, the expected return is: , The expected total revenue is: , (3) Model training: Based on the constructed MDP model, this embodiment uses a multi-layer feedforward neural network to extract key features from the input state and generate corresponding task scheduling strategies; the model training process is as follows: Figure 4 As shown; the core steps of training include: As the input of the network, the Q value of each possible action is calculated through the neural network. These Q values ​​represent the future rewards expected from taking each action in a given state. Based on the calculated Q value, the agent will choose an action according to a certain strategy. , and interact with the environment to get immediate rewards and the next state After each interaction with the environment, the agent will Stored in the experience replay pool to provide data support for subsequent learning processes; Different from the traditional experience replay method, this embodiment introduces the priority experience replay technology. Considering the importance of experience samples, when the buffer pool reaches a certain capacity, the system will assign a priority to each experience using the priority experience replay technology. This priority reflects the error of the current network's estimation of the value function, which is usually evaluated by the time difference error (TDerror); the calculation formula of TD error is as follows:

[0027] in, is the discount factor, and are the Q values ​​of the target network and the current network respectively; TDerror can measure the importance of the experience sample in the current training; usually, the experience sample with a larger TD error has a greater impact on the value estimation error of the network, so it needs to be sampled more frequently; define the priority is the absolute TD error size of each experience sample: , in is a small constant used to avoid zero priority. Through this calculation method, experiences with larger TD error will get higher priority. Next, sampling is performed according to the priority of the experience. The sampling probability of each experience is Determined by its priority and normalized by the following formula: , in is the total amount of experience stored in the experience pool, It is The priority of the experience, Represents the sum of the experience priorities; through normalization, it is ensured that the sum of all sampling probabilities is 1; in the priority experience playback, sampling not only selects experience samples based on priority, but also needs to consider the relative importance of experience; for this purpose, the weighting coefficient is introduced , this coefficient is used to adjust the impact of each experience on network training; the calculation formula of the weighted coefficient is as follows: , in, is a hyperparameter used to control the degree of sample importance adjustment. It gradually increases during training to reduce sampling bias. At this time, experiences with higher priorities will be sampled first, while experiences with lower priorities will be gradually replaced, thereby maintaining the effectiveness and diversity of experience samples in the buffer pool. Through this priority experience playback mechanism, experiences with larger TDerror will have a higher sampling probability, thereby guiding the agent to focus on important experiences, accelerate the learning process and improve training efficiency. At the same time, through the weighted sampling mechanism, the priorities of different experiences can be effectively balanced to avoid excessive attention to certain specific samples, thereby ensuring the stability and convergence of the training process. Next, the parameters of the current network are updated using the gradient descent method by calculating the loss function; the loss function is defined as the square of the TD error, that is: , The optimization goal of the loss function is to minimize the square of the TD error, so that the Q value of the current network is as close as possible to the estimated value of the target network, thereby improving the decision-making accuracy and efficiency of the agent in a given task scheduling; through the back propagation algorithm and gradient update, the parameters of the network will be gradually optimized, enabling the agent to generate more accurate and efficient scheduling decisions in complex satellite scheduling problems; Figure 5 Shown is the imaging satellite mission scheduling framework diagram.

[0028] Through the above-mentioned training process, the present invention realizes efficient scheduling of imaging satellites in resource allocation and mission planning. The CAS-D2RL framework can improve the adaptability of the system in complex environments through a dynamic optimization mechanism while ensuring the success rate of the mission by effectively combining the resource allocation stage and the mission execution stage. This framework not only optimizes the allocation of resources and improves the efficiency of resource utilization, but also maximizes the benefits of the mission through precise task scheduling strategies. Based on this, the scheduling framework proposed in the present invention provides imaging satellites with autonomous decision-making capabilities, enabling them to make real-time and accurate scheduling decisions when facing dynamic and uncertain environments, thereby enhancing the autonomy and adaptability of the system in a changing environment.

[0029] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned two-stage imaging satellite task scheduling method are implemented. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0030] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned two-stage imaging satellite task scheduling method are implemented.

[0031] like Figure 6As shown, the computer device 120 may include: at least one processor 121, such as a central processing unit (CPU), at least one communication interface 123, a memory 124, and at least one communication bus 122. Among them, the communication bus 122 is used to realize the connection and communication between these components. Among them, the communication interface 123 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 123 may also include a standard wired interface and a wireless interface. The memory 124 may be a high-speed random access memory (Random Access Memory, RAM), or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 124 may also be at least one storage device located away from the aforementioned processor 121. Among them, the memory 124 stores an application program, and the processor 121 calls the program code stored in the memory 124 to perform any of the above method steps. Among them, the communication bus 122 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 122 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6Only one line is used to represent it, but it does not mean that there is only one bus or one type of bus. Among them, the memory 124 may include a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); the memory may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a hard disk drive (hard disk drive, HDD) or a solid-state drive (solid-state drive, SSD); the memory 124 may also include a combination of the above-mentioned types of memory. Among them, the processor 121 may be a central processing unit (central processing unit, CPU), a network processor (network processor, NP) or a combination of CPU and NP. Among them, the processor 121 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (application-specific integrated circuit, ASIC), a programmable logic device (programmable logic device, PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a generic array logic (generic array logic, GAL) or any combination thereof. Optionally, the memory 124 is also used to store program instructions. The processor 121 can call the program instructions to implement the two-stage imaging satellite task scheduling method of this embodiment.

[0032] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A two-stage imaging satellite task scheduling method, characterized in that: The following steps are involved: S1: Construct an imaging mission planning model based on imaging missions and satellite resources; S2: According to the imaging task planning model, using the conflict-aware heuristic scheduling strategy, a pre-scheduled task set, a backtracking task list, a pending task list and a resource allocation plan are obtained; S3: According to the pre-scheduled task set, the backtracking task list, the pending task list and the resource allocation plan, a scheduling plan is obtained using the RainbowDQN algorithm.

2. The two-stage imaging satellite task scheduling method according to claim 1, characterized in that: Step S1 specifically includes: constructing an imaging mission planning model according to the imaging mission and satellite resources, such as the formula: ， ， ， ， ， ， ， ， ， ， ， ， ， ， in, For the task set, For the Observation tasks, Represents the number of tasks in the collection, For resource collection, For the resources, Represents the number of satellite resources available in the scene, For the task Occupied resources The set of allowed visible time windows, Indicates the task Occupied resources The allowed Visible time window, Indicates the task Occupied resources The number of visible time windows allowed when and Respectively represent the tasks Occupied resources The first The start and end time of the visible time window, is the task benefit value, is the execution time required for the task, It is the task storage consumption, The power consumption of the task, The time required for the task to transform. For all resources that can be arranged The set of tasks executed on To meet the observation task The set of resources required; The objective function of the imaging task planning model is, Indicates the task In Resources On A Boolean variable representing the execution status over a visible time window, represents maximization, and Respectively represent tasks The specific start and end time when selected for scheduling planning; and Respectively represent tasks The start and end time of the scene time window. Indicates The maximum data storage capacity of a resource, Indicates and The total power storage limit of each resource.

3. The two-stage imaging satellite task scheduling method according to claim 1, characterized in that: Step S2 specifically includes: S21: obtaining a task time window according to the imaging task planning model, mapping the task time window to corresponding available resources, and obtaining a temporal and spatial distribution characteristic of the task; S22: obtaining a pre-scheduled task set by using a pre-scheduling strategy according to the temporal and spatial distribution characteristics of the task; S23: according to the temporal and spatial distribution characteristics of the tasks and the pre-scheduled task set, using a backtracking search method, obtaining a backtracking task list and a to-be-processed task list; S24: Calculate the conflict degree and set the conflict degree threshold, and obtain a resource allocation plan according to the conflict degree, the conflict degree threshold, the backtracking task list and the pending task list.

4. The two-stage imaging satellite task scheduling method according to claim 3 is characterized in that: Step S24 specifically includes: S241: Calculate the conflict degree of the task, such as the formula: ， ， ， ， ， ， ， in, For all possible and visible time windows on the same resource The set of time windows that generate potential resource contention conflicts, Visible time window and The conflicting time slices where all time windows in the set intersect, is the posture conversion time required for the corresponding task, For the task Visible time window The timing violation indicator, For the task The degree of conflict, To backtrack the task list, A list of pending tasks. It is the union of the backtracking tasks and the tasks to be processed; S242: according to the conflict degree of the task, obtaining an average timing conflict index and its corresponding standard deviation and conflict degree threshold; S243: Obtaining the time window conflict degree by using a greedy strategy according to the conflict degree threshold, the task execution time, the weight that can be obtained after execution, and the timing conflict index of the visible time window; S244: According to the time window conflict degree, the time window with the minimum conflict degree is selected as the allocation time window of the task, and the tasks whose conflict degree of the minimum time window exceeds the conflict degree threshold are filtered from the set of tasks to be scheduled to obtain a resource allocation plan.

5. The two-stage imaging satellite task scheduling method according to claim 4, characterized in that: Step S242 specifically includes: according to the conflict degree of the task, obtaining an average timing conflict index and its corresponding standard deviation and conflict degree threshold, such as the formula: ， ， ， in, is the average timing conflict indicator, is the number of conflicting tasks, is the corresponding standard deviation, is the conflict threshold.

6. The two-stage imaging satellite task scheduling method according to claim 4, characterized in that: Step S243 specifically includes: according to the conflict degree threshold, task execution time, the weight that can be obtained after execution and the timing conflict index of the visible time window, using a greedy strategy, obtaining the time window conflict degree, such as the formula: ， in, For time window degree of conflict.

7. The two-stage imaging satellite task scheduling method according to claim 1, characterized in that: Step S3 specifically includes: S31: Construct experience priority, experience sampling probability, experience weighting coefficient and loss function; S32: According to the pre-scheduled task set, the backtracking task list, the pending task list and the resource allocation plan, the Markov decision process model is trained using the empirical priority, the empirical sampling probability, the empirical weighting coefficient, the loss function and the RainbowDQN algorithm to obtain a trained Markov decision process model; S33: Obtain a scheduling plan using the trained Markov decision process model.

8. The two-stage imaging satellite task scheduling method according to claim 7, characterized in that: Step S31 specifically includes: constructing experience priority, experience sampling probability, experience weighting coefficient and loss function, such as formula: ， ， ， ， ， in, For the The priority of the experience; For the The time difference error of the experience; is a small constant used to avoid the case of zero priority; For the The sampling probability of an experience, is the total amount of experience stored in the experience pool, is the expected total return of the Markov decision process model; is the discount factor, and are the Q values ​​of the target network and the current network respectively; and Respectively The environment state at time step and the state-based pass Select the action, is the maximum value index function, For the The weighting coefficient of the experience; It is a hyperparameter used to control the degree of sample importance adjustment, which gradually increases during training to reduce sampling bias; is the loss function.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the two-stage imaging satellite task scheduling method as described in any one of claims 1 to 8 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the two-stage imaging satellite task scheduling method as described in any one of claims 1-8 are implemented.

Images