Agile optical imaging satellite scheduling method and device based on deep reinforcement learning

Abstract

The present application relates to an agile optical imaging satellite scheduling method and apparatus based on deep reinforcement learning. The method comprises: obtaining satellite status data and first task data; encoding the satellite status data and the first task data to obtain dynamic encoded data; performing task screening on the first static encoded data based on a local attention mechanism to obtain a task list; obtaining a task probability distribution based on the dynamic encoded data and the task list based on the local attention mechanism; determining a second task and determining the execution order of the second task based on the task list, the task probability distribution, and a preset constraint satisfaction model; deleting a task from the first static encoded data based on the second task and the satellite status data to obtain second static encoded data; repeating the above steps with the second task as the first task and the second static encoded data as the first static encoded data until the execution order of all tasks in the task list is determined; and determining a satellite scheduling plan based on the execution order in the task list.

Description

Technical Field

[0001] This application relates to the field of satellite scheduling, specifically to an agile optical imaging satellite scheduling method and device based on deep reinforcement learning. Background Art

[0002] Agile optical imaging satellites, as a new generation of satellite platforms, possess flexible attitude maneuvering capabilities, capable of roll, pitch, and yaw. Due to the complexity of these satellite attitude maneuvers and the current surge in demand for Earth observation, the scheduling of agile optical imaging satellites has become a critical issue in space mission planning. Reasonable planning methods are key to ensuring the proper allocation of Earth observation missions and the efficient use of in-orbit satellite resources.

[0003] Existing agile satellite scheduling methods include exact algorithms, heuristic algorithms, and deep reinforcement learning algorithms. Exact algorithms search the entire solution space to ensure the global optimal solution. They mainly include dynamic programming, branch-and-bound, and integer programming, and have high algorithmic complexity. Heuristic algorithms rely on iterative search to obtain a better solution at a relatively low computational time. The main algorithms used include genetic algorithms, particle swarm algorithms, ant colony algorithms, and bee colony algorithms. The solution performance depends on the capabilities of the designed search operator. Because the agile satellite scheduling problem can be viewed as a sequential decision-making problem, some research has explored the use of deep reinforcement learning to solve it. The main methods include deep Q-network (DQN) and end-to-end network models based on encoder-decoder structures. Preliminary verification of the feasibility of deep reinforcement learning for solving this problem is provided.

[0004] However, deep reinforcement learning in existing technologies does not fully consider the intrinsic connections between tasks, which easily generates low-quality solutions and increases computational complexity. Summary of the Invention

[0005] In response to the above problems, the purpose of this application is to provide an agile optical imaging satellite scheduling method and device based on deep reinforcement learning.

[0006] According to a first aspect of an embodiment of the present application, a method for agile optical imaging satellite scheduling based on deep reinforcement learning is provided, comprising: obtaining satellite status data and first task data, wherein the satellite executes a first task based on the first task data and generates corresponding satellite status data; encoding the satellite status data and the first task data to obtain dynamic encoded data; performing task screening on the first static encoded data based on a local attention mechanism to obtain a task list; the first static encoded data is a set of tasks to be executed; obtaining a task probability distribution based on the dynamic encoded data and the task list based on the local attention mechanism; the task probability distribution is used to characterize the probability of each task in the task list being executed; determining a second task based on the task list, the task probability distribution, and a preset constraint satisfaction model, and determining an execution order of the second task; deleting a task from the first static encoded data based on the second task and the satellite status data to obtain second static encoded data; repeating the above steps with the second task as the first task and the second static encoded data as the first static encoded data until the execution order of all tasks in the task list is determined; and determining a satellite scheduling plan based on the execution order of each task in the task list.

[0007] Furthermore, the method is implemented based on a preset adaptive learning rate strategy, and the adaptive learning rate strategy includes:

[0008]

[0009] Where, lr k+1 is the adaptive learning rate for the next training, k is the number of training times, lr k is the current learning rate, γ is the decay rate of the learning rate, F(lr k ) is the preset learning rate formula.

[0010] Furthermore, the second task is determined based on the task list, the task probability distribution and the preset constraint satisfaction model, and the execution order of the second task is determined, including: based on the task probability distribution, determining the task to be executed with the highest probability of being executed in the task list, and based on the preset constraint satisfaction model, judging whether the task to be executed with the highest probability of being executed complies with the constraint satisfaction model; if not, removing the determined task to be executed from the task list; repeating the above operations until the task to be executed with the highest probability of being executed is determined from the task list to comply with the constraint satisfaction model; if it complies, determining that the determined task to be executed is the second task.

[0011] Furthermore, the constraint satisfaction model includes at least one or more of: an optimization target formula, a uniqueness constraint of task execution, an illumination constraint, a minimum imaging quality constraint, a posture conversion time constraint, a memory capacity constraint, and an energy constraint.

[0012] Furthermore, the first static coded data is deleted according to the second task and the satellite status data to obtain the second static coded data, including: generating a total task list according to the first static coded data; determining the remaining scheduling cycle data according to the second task data and the satellite status data; for each of the tasks to be executed, judging whether the current task to be executed has been executed; if executed, deleting the current task to be executed from the total task list; if not executed, judging whether there is a visible time window for the current task to be executed in the remaining scheduling cycle data; if not, deleting the current task to be executed from the total task list; if so, adding the current task to be executed to the second static coded data.

[0013] Furthermore, the method also includes: obtaining preset task data, performing data processing on the task data based on the static embedding layer to obtain static embedded data; inputting the static embedded data, extracting features of the task embedded data based on a static encoder to obtain first static encoded data.

[0014] According to the second aspect of the embodiment of the present application, an agile optical imaging satellite scheduling device based on deep reinforcement learning is provided, including: a dynamic data acquisition module for acquiring satellite status data and first task data, wherein the satellite performs a first task and generates corresponding satellite status data according to the first task data; a dynamic data processing module for encoding the satellite status data and the first task data to obtain dynamic coded data; a preliminary screening module for performing task screening on the first static coded data based on a local attention mechanism to obtain a task list; the first static coded data is a set of tasks to be executed; a probability calculation module for obtaining a task probability based on the dynamic coded data and the task list based on a local attention mechanism. The task probability distribution is used to characterize the probability of each task in the task list being executed; a task selection module is used to determine the second task and the execution order of the second task according to the task list, the task probability distribution and the preset constraint satisfaction model; a task deletion module is used to delete the first static coded data according to the second task and the satellite status data to obtain second static coded data; a loop module is used to use the second task as the first task and the second static coded data as the first static coded data, and repeatedly execute the above steps until the execution order of all tasks in the task list is determined; a plan generation module is used to determine the satellite scheduling plan according to the execution order of each task in the task list.

[0015] Furthermore, the device is implemented based on a preset adaptive learning rate strategy, and the adaptive learning rate strategy includes:

[0016]

[0017] Where, lr k+1 is the adaptive learning rate for the next training, k is the number of training times, lr κ is the current learning rate, γ is the decay rate of the learning rate, F(lr k ) is the preset learning rate formula.

[0018] According to a third aspect of an embodiment of the present application, a neural network model is provided to execute an agile optical imaging satellite scheduling method based on deep reinforcement learning, including: a static embedding layer, a static encoder, a dynamic embedding layer, a dynamic encoder and a decoder; wherein, the static embedding layer includes a fully connected network and a long short-term memory network; the static encoder includes a multi-head attention layer, a fully connected feedforward layer, a cross-layer connection structure and a layer normalization operator; the dynamic embedding layer includes a fully connected network; the dynamic encoder includes a long short-term memory network, a splicing operator and a fully connected network; the decoder includes a long short-term memory network, a local attention layer and a single-step scheduler.

[0019] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of an agile optical imaging satellite scheduling method based on deep reinforcement learning provided in the first aspect of the present application are implemented.

[0020] The technical solutions provided by the embodiments of the present application may have the following beneficial effects:

[0021] This application encodes the data of tasks and satellites, which helps to obtain the second task based on the task and satellite encoding. The encoded data of satellites and tasks is used to reduce the complexity of calculations through task screening. Based on the local attention mechanism, the probability of each task being executed is calculated to improve the quality of the generated solution. Based on the preset constraint satisfaction model, tasks that cannot be completed at the moment are eliminated to obtain an implementable execution order. Through task deletion, unfinished tasks are deleted, and the subsequent loop calculation cost is reduced to facilitate the generation of high-quality satellite scheduling solutions. This application effectively improves the solution efficiency by providing a solution method that does not require iterative search. By obtaining high-quality solutions, it provides a high-yield agile optical imaging satellite scheduling solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 This is a flowchart illustrating an agile optical imaging satellite scheduling method based on deep reinforcement learning according to an exemplary embodiment;

[0024] Figure 2 is a schematic diagram of a network model of an agile optical imaging satellite scheduling method based on deep reinforcement learning according to an exemplary embodiment;

[0025] Figure 3 1 is a schematic diagram of a visible window of an agile optical imaging satellite scheduling method based on deep reinforcement learning according to an exemplary embodiment;

[0026] Figure 4 is a diagram illustrating a construction process of an agile optical imaging satellite scheduling method based on deep reinforcement learning according to an exemplary embodiment;

[0027] Figure 5 This is a device diagram showing an agile optical imaging satellite scheduling method based on deep reinforcement learning according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this document.

[0029] Exemplary Method 1

[0030] like Figure 1 As shown, the embodiment of the present application provides an agile optical imaging satellite scheduling method based on deep reinforcement learning, comprising the following steps:

[0031] In step S101 , satellite status data and first mission data are acquired.

[0032] The satellite performs the first mission according to the first mission data and generates corresponding satellite status data.

[0033] In an embodiment of the present application, the satellite status data consists of the orbit number of the current schedule, the start time of the satellite idleness, i.e., the end time of the observation action, the satellite pitch angle after the observation action is executed, the roll angle of the satellite after the observation action is executed, the remaining memory capacity after the observation action is executed, and the remaining energy after the observation action is executed.

[0034] The first mission is the mission to be performed by the satellite. Satellite status data does not represent the actual satellite status, but rather the status obtained after executing the first mission. For example, satellite A will perform both missions 1 and 2. Based on satellite A's current state, the state of satellite A after executing mission 1 can be inferred. In this case, mission 1 is the first mission. Subsequently, based on satellite A's state after mission 1, mission 2 is executed, and the state of satellite A after mission 2 is determined. In this case, mission 2 is the first mission.

[0035] It should be noted that step S101 is equivalent to predicting in advance what effect the satellite will obtain after performing the mission. Subsequently, the decision on whether to perform the corresponding mission is made based on the simulated effect. In this way, the relationship between tasks can be associated with the mathematical model.

[0036] The satellite status data is normalized, and the original data is mapped to between 0 and 1 using a deviation normalization method, so that the data is limited to a certain range, thereby eliminating the adverse effects caused by singular sample data.

[0037] In step S102, the satellite status data and the first mission data are encoded to obtain dynamic encoded data.

[0038] In the embodiment of the present application, an effective solution to the agile optical imaging satellite scheduling problem is achieved by inputting data and based on a neural network model. Figure 2 As shown in Figure 1, the neural network model consists of five parts: a static embedding layer, a static encoder, a dynamic embedding layer, a dynamic encoder, and a decoder. Static coded data is obtained through the static embedding layer and the static encoder, while dynamic coded data is obtained through the dynamic embedding layer and the dynamic encoder. Based on the dynamic and static coded data, the decoder selects the second task and determines the satellite scheduling plan. The dynamic embedding layer uses a fully connected network W e Embed satellite status data into a high-dimensional vector space.

[0039] The dynamic encoder is used to extract dynamic features. The input of each step of the dynamic encoder includes the embedded form of the current satellite state data and the encoded form of the first mission data. The satellite state information has the following characteristics: first, dynamic changes; second, the current state is affected by the previous state; third, the state sequence has time series characteristics. Therefore, the dynamic encoder is composed of an LSTM network L d , concatenation operator and fully connected network W d constitute.

[0040] The process of generating dynamic codes consists of three steps: first, the dynamic embedding data is input and converted into a feature vector through an LSTM network; then, the feature vector is concatenated with the first task code using a concatenation operator; finally, a fully connected network extracts further features to produce the final dynamic code. For example, if Task 1 is selected as the execution task, the satellite state 1 of Satellite A after executing Task 1 can be inferred. The dynamic code result is then concatenated by concatenating the feature vector of Satellite State 1 with the code data of Task 1.

[0041] In step S103, based on the local attention mechanism, the first statically encoded data is task screened to obtain a task list.

[0042] In an embodiment of the present application, the first statically encoded data is a set of tasks to be performed. The first statically encoded data is obtained based on a static embedding layer and a static encoder.

[0043] Specifically, task data is input, where the task data includes task list time window information and requirement information.

[0044] The task data is normalized and mapped to between 0 and 1 using a deviation normalization method, so that the task data is limited to a certain range, thereby eliminating the adverse effects caused by singular sample data.

[0045] The static embedding layer is primarily used to embed task time window information and demand information into a high-dimensional vector space. Task information has the following characteristics: first, the number of tasks is uncertain; second, the order of tasks during input has no effect on the scheduling results; that is, the input task sequence does not have temporal characteristics. Furthermore, the characteristics of time window information and demand information are different. Different tasks may have different numbers of visible time windows, and the visible time window information of each task has temporal characteristics. Meanwhile, the demand information of each task is a vector with a fixed dimension. Therefore, the static embedding layer must be able to handle these two different types of data.

[0046] In the static embedding layer, the task data time window set passes through a fully connected network W w and a long short-term memory network (LSTM) L wThe demand information of the task data passes through a fully connected network W e Processing, get the required result vector. Then concatenate the two result vectors and pass them through another fully connected network W s After further processing, static embedded data is obtained. The static embedding layer processes the two data with different characteristics to obtain static embedded data, which is then input into the static encoder for encoding.

[0047] The static encoder further extracts features from the static embedded data after preliminary processing. Since the order of the input task set has no effect, the position encoding and recurrent neural network commonly used in deep reinforcement learning are not suitable for encoding the current data. Therefore, the static encoder chooses to use a multi-head attention layer W m and a fully connected feed-forward layer W f To extract the feature information of static input. A cross-layer connection structure and a layer normalization operator are added between layers.

[0048] The static encoder adopts network structures such as fully connected networks, long short-term memory networks, and multi-head attention mechanisms, which can process input data with different dimensions. Therefore, the network model can be applied to agile optical imaging satellite scheduling problems of different scales and has good universality and generalization capabilities.

[0049] Through the above process, both satellite status data and mission data are converted into feature-extracted data to facilitate subsequent calculations. For example, satellite status data is a dynamically changing data set with time series characteristics and requires a specific input order. Mission data is a data set without time series characteristics, with an indefinite number of data and no specific input order. Since the values and formats of the two are different, direct calculations cannot be performed on them. Therefore, the mission data is converted into statically encoded data and the satellite status data into dynamically encoded data. The features of both data are extracted to facilitate subsequent calculations.

[0050] In an embodiment of the present application, there may be multiple visible time windows for an observation task, and these visible time windows are distributed on different orbits. The visible time windows of different tasks may also be distributed on the same orbit. On the same orbit, the execution of the previous task will affect the completion of the next task. The final state of the satellite on the previous orbit is also the initial state of the satellite on the next orbit, so the scheduling result of the satellite on the previous orbit will affect the scheduling of the next orbit. Based on this characteristic, the local attention mechanism no longer calculates the probability distribution of all unplanned tasks, but only considers the unplanned tasks with visible time windows on the current orbit and the next orbit. Therefore, based on this screening, only the conditional probabilities of some unplanned tasks need to be calculated to obtain a task list, reducing the number of tasks that need to be calculated and reducing the complexity of the calculation.

[0051] The visible time window is as follows Figure 3 As shown, the agile optical imaging satellite operates in orbit, and when passing the target, the observation area covers the observation window to obtain a visible time window.

[0052] In step S104, based on the local attention mechanism, the task probability distribution is obtained according to the dynamic encoding data and the task list.

[0053] The task probability distribution is used to represent the probability of each task in the task list being executed. For example, the task list contains Task 1, Task 2, and Task 3. Corresponding to Task Code 1, Task Code 2, and Task Code 3. Based on the local attention mechanism, the conditional probability of executing Task 1 is calculated to be 0.4, the conditional probability of executing Task 2 is 0.5, and the conditional probability of executing Task 3 is 0.2 under the current dynamic encoding data.

[0054] In this embodiment, the task probability distribution is composed of the conditional probabilities corresponding to each task in the task list. The conditional probabilities represent the degree of fit between each task in the task list and the current satellite state and trajectory. The more suitable the task is for the current satellite state, the higher the conditional probability of the next task to be executed. This intuitively demonstrates the selection order with data, laying the foundation for obtaining a high-quality solution.

[0055] Because the results of dynamic encoding have time series characteristics, an LSTM network is used for processing. The conditional probability is calculated based on the processed dynamic encoding data and the task list.

[0056] In step S105 , a second task is determined according to the task list, the task probability distribution and a preset constraint satisfaction model, and an execution order of the second task is determined.

[0057] Based on the probability distribution, determining the to-be-executed task with the highest probability of being executed in the task list, and based on a preset constraint satisfaction model, determining whether the to-be-executed task with the highest probability of being executed satisfies the constraint satisfaction model;

[0058] If not, removing the determined task to be executed from the task list; repeating the above operation until a task to be executed with the highest probability of being executed is determined from the task list to meet the constraint satisfaction model;

[0059] If so, the determined task to be executed is determined to be the second task.

[0060] The constraint satisfaction model includes an optimization objective formula and six constraint formulas: task execution uniqueness constraint, illumination constraint, minimum image quality constraint, attitude transition time constraint, memory capacity constraint, and energy constraint. This improves the quality of the generated solution. The constraint satisfaction model pre-determines task feasibility. Based on task information and satellite status, unfinishable tasks are identified and removed from the task list, ensuring task completion. This improves the feasibility of the resulting satellite scheduling plan and ensures its quality.

[0061] For example, in a task list, task 1 has a conditional probability of 0.5, task 2 has a conditional probability of 0.1, and task 3 has a conditional probability of 0.9. Task 3, which has the highest conditional probability, is selected and its compliance with the constraint satisfaction model is determined. If it is, task 3 is made the second task. If not, task 3 is removed from the task list and task 1, which has the highest conditional probability, is selected from the remaining tasks. The constraint satisfaction model is determined to be satisfied until the second task is confirmed or there are no remaining tasks in the task list.

[0062] In step S106, the first static coded data is globally filtered according to the second mission and the satellite status data to obtain second static coded data.

[0063] In an embodiment of the present application, based on a single-step scheduler, after one round of task selection, the remaining scheduling period data is determined according to the first task data and the satellite state element;

[0064] In the embodiment of the present application, when inputting tasks to be executed, it is ensured that all tasks are executable. After a round of task selection, the satellite status changes, and the task execution may also change accordingly. The feasibility of the remaining tasks to be executed is screened.

[0065] For each pending task, determine whether the current pending task has been executed. If not, determine whether there is a visible time window for the current pending task within the remaining scheduling period data. If so, add the current pending task to the task list. For example, if there are visible time windows a, b, and c in the remaining scheduling period, and Task 1 requires visible time window a, then Task 1 will be added to the task list. If Task 2 requires visible time window d, then Task 1 will not be added to the task list. Only tasks that are likely to be executed will continue to be calculated to determine the execution order, reducing the amount of calculation and obtaining high-quality results.

[0066] In step S107 , the second task is used as the first task, and the second statically encoded data is used as the first statically encoded data, and steps S101 - S106 are repeatedly executed until the execution order of all tasks in the task list is determined.

[0067] When there are no remaining tasks to be planned, the current scheduling ends.

[0068] In step S108, a satellite scheduling plan is determined according to the execution order of each task in the task list.

[0069] The execution order of each task in the task list is used as the task sorting order to obtain the satellite scheduling plan, and the satellite executes the task according to the satellite scheduling plan.

[0070] Exemplary Method 2

[0071] In this embodiment, if Figure 4 , provides a construction process of an agile optical imaging satellite scheduling method based on deep reinforcement learning.

[0072] Step S401: convert the real-world agile optical imaging satellite scheduling problem into a mathematical problem.

[0073] Use mathematical language to describe the mission information and satellite status in the agile optical imaging satellite scheduling problem, and use mathematical symbols to represent the parameters and variables involved in the problem:

[0074] The agile optical imaging satellite scheduling problem can be expressed as a quaternion SAOSSP = {S, T, VTW, A}, where:

[0075] S=(op,O,M,E,rm,reo,rea,r max , r min , p max , p min ) represents the satellite's attribute parameters. op represents the satellite's orbital parameters; O = {o l |1≤l≤|O|} represents the orbital set of the satellite, |O| represents the total number of orbital circles in the scheduling period, l represents the orbit number, and o l represents the lth orbit; M represents the maximum memory capacity for storing observation images; E represents the maximum energy used for observation and attitude adjustment; rm represents the memory consumption rate during satellite observation; reo represents the energy consumption rate during satellite observation; rea represents the energy consumption rate during satellite attitude adjustment; r max and r min Represent the maximum roll angle and minimum roll angle of the satellite, r min =-r max ;p max and p min Respectively represent the maximum and minimum pitch angles of the satellite, p min =-p max .

[0076] T={t i|i=1,…,|T|) represents the task set, |T| represents the number of tasks, t i =(loc i ,q i , d i , p i ) represents task t i Attribute parameters of loc i =(lat i ,lon i ) represents the coordinates of the task, lat i is latitude, lon i is the longitude; q i Indicates the minimum imaging quality requirement for the task, and the imaging quality is divided into ten levels from 1 to 10; d i Indicates the continuous observation time required for the task; p i It represents the priority of the task and also the observed benefit of the task.

[0077] VTW={W i |i=1,…,|T|} represents the set of visible time windows of all tasks, Represents task t i The visible time window set, |W i | is the total number of its visible time windows, Represents the attribute parameters of the kth visible time window of the task, Indicates the track number where the window is located; and Respectively represent the start time and end time of the visible time window; Indicates the scroll angle corresponding to the window.

[0078] A represents the satellite's observation action sequence, Indicates that the satellite is in the visible time window Execute task t i Observation action. and Respectively represent the start time and end time of the observation action, Indicates the corresponding pitch angle; mr i and er i They represent the remaining memory capacity and energy after the action is executed.

[0079] The attitude transition time tranT(·) during the attitude maneuver of the satellite around the pitch axis or roll axis can be expressed as:

[0080]

[0081] Where ΔA is the adjustment angle, a is the angular acceleration, ω is the maximum angular velocity, and ST is the attitude stabilization time. If the satellite's roll angle is adjusted by Δr and its pitch angle by Δp, the total attitude transition time is ΔT = tranT(Δr) + tranT(Δp).

[0082] In the visible time window Inside, pitch angle and observation start time The relationship between can be expressed by a linear function:

[0083]

[0084] Where,

[0085] The imaging quality is determined by the start time of observation. Inside, and They represent the earliest observation start time and the latest observation start time for the mission to meet the minimum imaging quality requirements.

[0086] The expressions of the three decision variables are as follows:

[0087]

[0088]

[0089]

[0090] Step S402: Establish a constraint satisfaction model.

[0091] According to the description of the agile optical imaging satellite scheduling problem, taking the maximization of the rate of return of completing the mission as the optimization goal, and fully considering the uniqueness constraint of mission execution, illumination constraint, minimum imaging quality constraint, attitude conversion time constraint, memory capacity constraint and energy constraint, a constraint satisfaction model of the agile optical imaging satellite scheduling problem is established as follows:

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099] Among them, formula 1 is the optimization objective formula, and formulas 2 to 7 are the constraints. l ) represents track o l The time period during which the satellite is in the illuminated area. Formula 2 represents the uniqueness constraint for task execution, meaning each task can only be executed once at most. Formula 4 represents the minimum imaging quality constraint, meaning the satellite's observation start time for the task must meet the minimum imaging quality requirement. Formula 5 represents the attitude transition time constraint, meaning when the satellite executes tasks u and v sequentially, the interval between the two tasks must be sufficient to allow for attitude transitions. Formula 6 represents the memory capacity constraint, meaning the total storage space for observation images on each orbit cannot exceed the maximum memory capacity. Formula 7 represents the energy constraint, meaning the energy consumed by observations and attitude maneuvers on each orbit cannot exceed the maximum energy. This improves the quality of the generated solution.

[0100] Step S403: Establish a neural network model to solve the problem.

[0101] The neural network model consists of five parts: static embedding layer, static encoder, dynamic embedding layer, dynamic encoder and decoder.

[0102] The input of the network model can be expressed as X = {tx i |i=1,…,|T|}. tx i Represents the input data as a sequence of tuples Indicates that sx i =(WX i , R i ) indicates task information, WX i Represents the time window information, R i Indicates demand information, Indicates the satellite status data at the time of decoding in step n.

[0103] in, R i =[d i , p i ]. express The corresponding maximum pitch angle that meets the minimum imaging quality requirements is, express The corresponding minimum pitch angle that meets the minimum imaging quality requirements.

[0104] In the static embedding layer, input sx i =(WX i , Ri ), after being processed by the fully connected network and the long short-term memory network (LSTM), the embedded result static embedded data is obtained. The static embedded data is expressed as SE = {se i |i=1,…,|T|), the process is as follows:

[0105] we i =L w (W w (WX i ));

[0106] re i =W r (R i );

[0107] se i =W s (Concat(we i ,re i ));

[0108] Where W w , L w and W s Represents the corresponding network parameters, Concat(·,·) represents the concatenation function, we i Represents time window data, re i Represents the required data. The static embedding layer processes data with different characteristics and obtains information of the same dimension for subsequent calculations.

[0109] The static encoder further extracts features from the preliminarily processed information.

[0110] In the static encoder, the input embedding result SE passes through the multi-head attention layer, the fully connected feedforward layer, the cross-layer connection structure and the layer normalization operator LN(·), and the static encoded data SC is obtained, SC={sc i The output of the multi-head attention layer is represented by HM, and the final encoding result is represented by SC.

[0111] The calculation process of static encoding is as follows:

[0112] HM=LN(SE+W m (SE));

[0113] SC=LN(HM+W f (HM));

[0114] Where W m is the network parameter of the multi-head attention layer, W f are the network parameters of the fully connected feed-forward layer.

[0115] In the dynamic embedding layer, the satellite status data s is input n , After the fully connected network, the dynamic embedded data is obtained n , the calculation process is as follows:

[0116] de n =W e (s n );

[0117] Among them, W e represents the parameters of the fully connected network, and the dynamic embedding layer embeds the satellite status data into a high-dimensional vector space.

[0118] The dynamic encoder is used to extract dynamic features.

[0119] In the dynamic encoder, the input embedding result de n and the corresponding task code Get dc through LSTM network n , through the splicing operator and the fully connected network, the dynamic encoding result c is output n , W c Indicates the corresponding network parameters.

[0120] The process is as follows:

[0121] dc n =L d (de n );

[0122]

[0123] The decoder can make decisions based on the outputs of the static encoder and the dynamic encoder. It consists of three parts: an LSTM network, a local attention layer, and a single-step scheduler.

[0124] Because the result of dynamic coding has time series characteristics, LSTM network is used for processing to obtain the processed data h n , the formula is as follows:

[0125] h n =L c (c n );

[0126] Where, L c Represents the network parameters of LSTM.

[0127] A local attention mechanism is used to optimize solution generation.

[0128] Based on the local attention mechanism, the static coding data is filtered to obtain a task list. According to the dynamic coding data and the task list, the task probability distribution is obtained. The coding set of the static elements of the task that meets the n-th step decoding of the visible time window on the current track and the next track is LS n The calculation process of conditional probability is as follows:

[0129] u n =v T tanh(W a (h n )+W b (sc i )), sc i ∈LS n ;

[0130] P(·|Y n , X n )=softmax(u n );

[0131] Where, v T 、W a and W b are all learnable network parameters of the local attention layer, u n is the intermediate result of the local attention layer. n represents the state set of all inputs at the time of decoding in step n. n |n=0,…,N} represents the final output sequence, where ty n Y represents the mission information selected in the nth step decoding, and ty0 represents the initial virtual tuple composed of virtual static elements and the initial state of the satellite. n ={ty0, ..., ty n ) represents the output sequence at the n-th step decoding, and P(·) represents the conditional probability.

[0132] In each step of decoding, the network model first generates X n The probability distribution of each input in the previous step is then determined by the ty selected in the previous step. n Pointing to X n The input with the highest probability can be selected as the next ty n+1 And used for the next step of decoding, at the same time ty n+1 is put into the sequence Y generated by decoding n+1 Then, according to the constraints in the actual problem, X n Update to X n+1 The above process can be expressed by the following formula:

[0133]

[0134] X n+1 =f(ty n+1 , X n );

[0135] Where f(·) represents the state transition function that updates the input state set, θ represents the learnable network parameters, Indicates a function that returns the maximum value index.

[0136] In the embodiment of the present application, the optimal parameter θ of the network model * It can be obtained by maximizing the above conditional probability, expressed by the formula:

[0137]

[0138] The single-step scheduler selects the task with the highest probability as the next task to be executed and selects the earliest feasible time as the observed start time of the task. The generated single-step scheduling result must satisfy the constraint satisfaction model in step S402.

[0139] Once a task is successfully scheduled, it is necessary to determine whether the remaining unscheduled tasks have visible time windows within the remaining scheduling period. Unscheduled tasks that do not exist will be removed from the current unscheduled task sequence.

[0140] When there are no remaining unplanned tasks, the current scheduling ends and the satellite scheduling plan is determined.

[0141] Exemplary Method Three

[0142] The actual neural network model is trained using an actor-critic training algorithm based on an adaptive learning rate. The training process involves initializing the network parameters and learning rates of the actor and critic networks and performing multiple training cycles using datasets of varying task sizes. Each training cycle consists of four steps: First, the actor network cyclically decodes the mission's static information and satellite status information to construct a satellite scheduling plan and calculate the reward for the plan; second, the critic network calculates an evaluation value based on the mission's static information; third, the reward and evaluation values are used to calculate the gradients of the two networks and optimize the network parameters; and fourth, an adaptive learning rate strategy is used to update the learning rate of the actor network.

[0143] In the above training algorithm, the Actor network is the policy network used to generate scheduling results, that is, the neural network model constructed according to step S403; the Critic network is the evaluation network used to evaluate the input reward value. It consists of an embedding layer, an encoder, and a decoder. The network structures of the Critic network's embedding layer and encoder are consistent with the network structures of the Actor network's static embedding layer and static encoder, respectively. The decoder consists of two one-dimensional convolutional networks. Both networks require training. The specific network structure parameter settings for the Actor network and Critic network are shown in Table 1. In the Actor network's static embedding layer, the first fully connected network (Linear) has an input dimension (input_size) of 2 and an output dimension (out_size) of 128; the LSTM network (LSTMcell) has an input dimension (input_size) of 6 and an output dimension (hidden_size) of 128; the second fully connected network (Linear) has an input dimension (input_size) of 256 and an output dimension (out_size) of 128. In the static encoder of the actor network, the input and output dimensions (embeded_dim) of the multi-head attention (MultiHeadAttention) are both 128, the number of heads (num_heads) is 2, and the dropout rate (dropout) is 0.1. The input dimension (input_size) of the one-dimensional convolutional network (Conv1d) is 128, the output dimension (output_size) is 128, and the convolution kernel (kernel_size) size is 1. The input dimension (input_size) of the fully connected network (Linear) in the dynamic embedding layer of the actor network is 6, and the output dimension (out_size) is 128. In the dynamic decoder of the actor network, the input dimension (input_size) of the LSTM network (LSTMcell) is 256, and the output dimension (hidden_size) is 128; the input dimension (input_size) of the fully connected network (Linear) is 256, and the output dimension (out_size) is 128. In the decoder of the Actor network, the LSTM network (LSTMcell) has an input dimension (input_size) of 6 and an output dimension (hidden_size) of 128; the local attention mechanism does not require setting network parameters.The embedding layer and encoder of the Critic network are consistent with the static embedding layer and static encoder network structures of the Actor network respectively; in its decoder, the input dimension (input_size) of the first one-dimensional convolutional network (Conv1d) is 128, and the output dimension (output_size) is 64. The input dimension (input_size) of the second one-dimensional convolutional network (Conv1d) is 64, and the output dimension (output_size) is 16. The input dimension (input_size) of the third one-dimensional convolutional network (Conv1d) is 64, and the output dimension (output_size) is 1. The convolution kernel (kernel_size) of the three convolutional networks is 1.

[0144] Table 1 Specific composition and parameter settings of Actor network and Critic network

[0145]

[0146] In the above training algorithm, an adaptive learning rate strategy is proposed to improve the training effect of the policy network. The learning rate decays exponentially at a fixed rate overall, but is appropriately increased every ten training runs based on the trend of the reward curve and the evaluation value of the previous ten training runs. If the trend of the reward curve does not reach the expected value after ten training runs, the learning rate is appropriately increased. The specific method is as follows:

[0147]

[0148] Where, lr k+1 is the adaptive learning rate for the next training, k is the number of training times, lr κ is the current learning rate, γ is the decay rate of the learning rate, F(lr k ) is the preset learning rate formula.

[0149]

[0150] Where, is the mean of the evaluation values output by the evaluation network, is the evaluation value of the network output, e κ is the expected slope value, is the average reward, sl κ Indicates the slope of the reward curve for the first 10 trainings. If sl κ <e κ , then record the current κ as Slope sl κ The expression is as follows:

[0151]

[0152] In the formula, the reward value of the first ten trainings is given Represents the mean value from 1 to 10.

[0153] The ideal reward curve can rise rapidly in the middle of training and converge steadily in the late stage of training, so the sine function is used to represent the baseline value of the slope. The expected slope value e κ The expression is as follows:

[0154]

[0155] Where, express K represents the total number of training times.

[0156] Exemplary devices

[0157] In the embodiments of this application, Figure 5 , including a dynamic data acquisition module 501, a dynamic data processing module 502, a preliminary screening module 503, a probability calculation module 504, a task selection module 505, a task deletion module 506, a loop module 507 and a solution generation module 508.

[0158] A dynamic data acquisition module 501 is configured to acquire satellite status data and first mission data, wherein the satellite performs a first mission according to the first mission data and generates corresponding satellite status data;

[0159] A dynamic data processing module 502 is configured to encode the satellite status data and the first mission data to obtain dynamic encoded data;

[0160] The initial screening module 503 is configured to perform task screening on the first statically encoded data based on a local attention mechanism to obtain a task list; the first statically encoded data is a set of tasks to be executed;

[0161] A probability calculation module 504 is configured to obtain a task probability distribution based on the dynamic encoding data and the task list based on a local attention mechanism; the task probability distribution is used to represent the probability of each task in the task list being executed;

[0162] The task selection module 505 is configured to determine the second task based on the task list, the task probability distribution, and the preset constraint satisfaction model, and determine the execution order of the second task; determine the task to be executed with the highest probability of being executed in the task list based on the task probability distribution, and determine whether the task to be executed with the highest probability of being executed complies with the constraint satisfaction model based on the preset constraint satisfaction model; if not, remove the determined task to be executed from the task list; repeat the above operations until the task to be executed with the highest probability of being executed is determined to comply with the constraint satisfaction model from the task list; if so, determine that the determined task to be executed is the second task. The constraint satisfaction model includes at least one or more of: an optimization objective formula, a uniqueness constraint for task execution, an illumination constraint, a minimum imaging quality constraint, a posture transition time constraint, a memory capacity constraint, and an energy constraint.

[0163] The task deletion module 506 is used to delete the first static coded data according to the second task and the satellite status data to obtain the second static coded data; generate a total task list according to the first static coded data; determine the remaining scheduling cycle data according to the second task data and the satellite status data; for each of the tasks to be executed, determine whether the current task to be executed has been executed; if executed, delete the current task to be executed from the total task list; if not executed, determine whether there is a visible time window for the current task to be executed in the remaining scheduling cycle data; if not, delete the current task to be executed from the total task list; if so, add the current task to be executed to the second static coded data.

[0164] A loop module 507 is configured to repeatedly execute the above steps with the second task as the first task until the execution order of all tasks in the task list is determined;

[0165] The plan generating module 508 is configured to determine a satellite scheduling plan according to the execution order of each task in the task list.

[0166] Exemplary electronic devices

[0167] This embodiment proposes an electronic device, comprising: one or more processors, and an internal memory and an external memory, wherein the internal memory stores instructions. When the instructions are executed by the one or more processors, the one or more processors execute the agile optical imaging satellite scheduling method based on deep reinforcement learning described in any of the aforementioned embodiments.

[0168] The processor is configured to execute all or part of the steps of the method for agile optical imaging satellite scheduling based on deep reinforcement learning, as described in the embodiment. The memory is configured to store various types of data, such as instructions for any application or method in the electronic device, as well as data related to the application.

[0169] The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute the agile optical imaging satellite scheduling method based on deep reinforcement learning described in the embodiment.

[0170] Computer storage media

[0171] The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, it implements an agile optical imaging satellite scheduling method based on deep reinforcement learning as described in any of the aforementioned embodiments.

[0172] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0173] In the 1930s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0174] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0175] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0176] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0177] Those skilled in the art will appreciate that one or more embodiments of this specification may be provided as a method, system, or computer program product. Thus, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0181] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0182] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0183] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0184] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0185] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0186] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0187] The foregoing description is merely an example of the present invention and is not intended to limit the present invention. Persons skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims herein.

Claims

1. An agile optical imaging satellite scheduling method based on deep reinforcement learning, characterized in that: include: Acquiring satellite status data and first mission data, wherein the satellite performs a first mission according to the first mission data and generates corresponding satellite status data; Encoding the satellite status data and the first mission data to obtain dynamic encoded data; Based on the local attention mechanism, the first statically encoded data is task-filtered to obtain a task list; the first statically encoded data is a set of tasks to be executed; Based on the local attention mechanism, a task probability distribution is obtained according to the dynamic encoding data and the task list; the task probability distribution is used to represent the probability of each task in the task list being executed; Determining a second task according to the task list, the task probability distribution, and a preset constraint satisfaction model, and determining an execution order of the second task; Deleting the first static coded data according to the second task and the satellite status data to obtain second static coded data; Using the second task as the first task and the second statically coded data as the first statically coded data, repeat the above steps until the execution order of all tasks in the task list is determined; The satellite scheduling plan is determined according to the execution order of each task in the task list.

2. The method according to claim 1, characterized in that The method is implemented based on a preset adaptive learning rate strategy, which includes: Where, lr k+1 is the adaptive learning rate for the next training, k is the number of training times, lr k is the current learning rate, γ is the decay rate of the learning rate, F(lr k ) is the preset learning rate formula.

3. The method according to claim 1, characterized in that The determining of the second task according to the task list, the task probability distribution, and the preset constraint satisfaction model, and determining the execution order of the second task, includes: Determining, based on the task probability distribution, a to-be-executed task with the highest probability of being executed in the task list, and judging, based on a preset constraint satisfaction model, whether the to-be-executed task with the highest probability of being executed satisfies the constraint satisfaction model; If not, removing the determined task to be executed from the task list; repeating the above operation until a task to be executed with the highest probability of being executed is determined from the task list to meet the constraint satisfaction model; If so, the determined task to be executed is determined to be the second task.

4. The method according to claim 3, characterized in that The constraint satisfaction model includes at least one or more of an optimization target formula, a uniqueness constraint of task execution, an illumination constraint, a minimum imaging quality constraint, a posture conversion time constraint, a memory capacity constraint, and an energy constraint.

5. The method according to claim 1, wherein The step of deleting the first static coded data according to the second task and the satellite status data to obtain the second static coded data includes: generating a total task list according to the first statically encoded data; determining remaining scheduling period data according to the second mission data and the satellite status data; For each of the tasks to be executed, determining whether the current task to be executed has been executed; If executed, the current task to be executed will be deleted from the total task list; If not executed, determine whether there is a visible time window for the current task to be executed within the remaining scheduling cycle data; If it does not exist, delete the current task to be executed from the total task list; If so, add the current task to be executed to the second static coded data.

6. The method according to claim 1, characterized in that The method further comprises: Acquire preset task data, and perform data processing on the task data based on the static embedding layer to obtain static embedded data; The static embedding data is input, and based on a static encoder, features of the task embedding data are extracted to obtain first static encoded data.

7. An agile optical imaging satellite scheduling device based on deep reinforcement learning, characterized in that: include: A dynamic data acquisition module is used to acquire satellite status data and first mission data, wherein the satellite performs the first mission and generates corresponding satellite status data according to the first mission data; a dynamic data processing module, configured to encode the satellite status data and the first mission data to obtain dynamic encoded data; A preliminary screening module, configured to perform task screening on the first statically encoded data based on a local attention mechanism to obtain a task list; the first statically encoded data is a set of tasks to be performed; A probability calculation module is used to obtain a task probability distribution based on the dynamic encoding data and the task list based on a local attention mechanism; the task probability distribution is used to represent the probability of each task in the task list being executed; a task selection module, configured to determine a second task and an execution order of the second task based on the task list, the task probability distribution, and a preset constraint satisfaction model; A task deletion module is used to delete the first static coded data according to the second task and the satellite status data to obtain second static coded data; a loop module, configured to repeatedly execute the above steps with the second task as the first task and the second statically coded data as the first statically coded data until the execution order of all tasks in the task list is determined; The plan generation module is used to determine the satellite scheduling plan according to the execution order of each task in the task list.

8. The device according to claim 7, characterized in that The device is implemented based on a preset adaptive learning rate strategy, and the adaptive learning rate strategy includes: Where, lr k+1 is the adaptive learning rate for the next training, k is the number of training times, lr κ is the current learning rate, γ is the decay rate of the learning rate, F(lr k ) is the preset learning rate formula.

9. A neural network model, characterized in that Implementing agile optical imaging satellite scheduling methods based on deep reinforcement learning, including: Static embedding layer, static encoder, dynamic embedding layer, dynamic encoder and decoder; Among them, the static embedding layer includes a fully connected network and a long short-term memory network; the static encoder includes a multi-head attention layer, a fully connected feedforward layer, a cross-layer connection structure and a layer normalization operator; the dynamic embedding layer includes a fully connected network; the dynamic encoder includes a long short-term memory network, a splicing operator and a fully connected network; the decoder includes a long short-term memory network, a local attention layer and a single-step scheduler.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the agile optical imaging satellite scheduling method based on deep reinforcement learning as claimed in any one of claims 1 to 6.

Images