A multi-satellite intelligent collaborative observation method based on model-accelerated reinforcement learning

Abstract

The present invention discloses a multi-satellite collaborative observation method and apparatus. The method includes obtaining a training data set, mission information, and time window information; fusing the training data set, mission information, and time window information to obtain collaborative observation information to be processed; and analyzing and calculating the mission information, time window information, and collaborative observation information to be processed to obtain target collaborative observation information. This application can accelerate the decision-making process for multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of search optimization for satellite mission planning.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-satellite autonomous collaborative observation in land and resources surveys, and in particular to a multi-satellite intelligent collaborative observation method based on model-accelerated reinforcement learning. Background Art

[0002] Satellite remote sensing has become a crucial tool for land and resources surveys and disaster monitoring. With increasing mission demands, the rational scheduling of satellites has become particularly crucial. Existing satellite mission planning research often models imaging satellite mission planning as an optimization problem. However, existing satellite collaborative mission modeling often employs combinatorial optimization or machine scheduling models, and the solution algorithms often employ heuristic search or intelligent optimization algorithms. Satellite mission planning is an NP-Hard problem, and the difficulty and time consumption of search optimization increase dramatically with the scale of the problem. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a multi-satellite intelligent collaborative observation method based on model-accelerated reinforcement learning, which is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0004] In order to solve the above technical problems, a first aspect of an embodiment of the present invention discloses a multi-satellite collaborative observation method, the method comprising:

[0005] S1, obtain training data set, task information and time window information;

[0006] S2, fusing the training data set, the task information, and the time window information to obtain collaborative observation information to be processed;

[0007] S3, analyzing and calculating the task information, the time window information and the collaborative observation information to be processed to obtain target collaborative observation information.

[0008] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the fusing processing of the training data set, the task information, and the time window information to obtain the collaborative observation information to be processed includes:

[0009] S21, preprocessing the training data set to obtain a preprocessed training data set;

[0010] S22, using the preprocessed training data set, training a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model;

[0011] S23: Utilize the multi-satellite collaborative observation optimization model to perform calculations on the mission information and the time window information to obtain collaborative observation information to be processed.

[0012] As an optional implementation manner, in the first aspect of the embodiment of the present invention, using the preprocessed training data set to train a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model includes:

[0013] S221, processing the preprocessed training data set to obtain a target training data set;

[0014] S222: Training the multi-satellite collaborative observation model using the target training data set to obtain training result information and a multi-satellite collaborative observation training model;

[0015] S223, using a loss function to calculate and process the training result information to obtain a loss function value;

[0016] S224, determining whether the loss function value is less than a preset loss function threshold, and obtaining a first determination result;

[0017] When the first judgment result is no, determining that the multi-satellite collaborative observation training model is the multi-satellite collaborative observation model, executing S222;

[0018] When the first judgment result is yes, it is determined that the multi-satellite collaborative observation training model is a multi-satellite collaborative observation optimization model.

[0019] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the loss function is:

[0020]

[0021] Among them, Loss is the loss function value, P(x i,j ′) is the result information of the jth task feature of the i-th sample in the training result information, P(x i,j ) is the true result information corresponding to the jth task feature of the i-th sample, w j is the weight coefficient of the jth task feature, N is the number of samples, M is the number of task features in the samples, and α is the deviation coefficient.

[0022] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the analyzing and calculating the task information, the time window information, and the to-be-processed collaborative observation information to obtain the target collaborative observation information includes:

[0023] S31, determining that the collaborative observation information to be processed is initial state information;

[0024] S32, fusing the task information and the time window information to obtain state space information and action space information;

[0025] S33: Process the state space information, the action space information, and the initial state information to obtain target collaborative observation information.

[0026] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the state space information, the action space information, and the initial state information to obtain target collaborative observation information includes:

[0027] S331, processing the state space information and the action space information to obtain state-action table information;

[0028] S332, determining that the initial state information is state information;

[0029] S333, preset t=1;

[0030] S334, performing calculation processing on the state information and the state-action table information to obtain action information;

[0031] S335, calculating and processing the action information and the state information to obtain a reward value;

[0032] S336, processing the initial state information, the state information, the action information, the state-action table information, and the reward value to obtain updated state information and updated state-action table information, and determining the updated state information as the state information and the updated state-action table information as the state-action table information;

[0033] S337, determining whether t is greater than a preset number of iterations, and obtaining a first determination result;

[0034] When the first judgment result is no, increment t by 1 and execute S334;

[0035] When the first judgment result is yes, it is determined that the state-action table information is the target collaborative observation information.

[0036] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the initial state information, the state information, the action information, the state-action table information, and the reward value to obtain updated state information and updated state-action table information includes:

[0037] S3361: Using a multi-satellite collaborative observation update model, processing the initial state information, the state information, the action information, the state-action table information, and the reward value to obtain a state update action value and updated state information;

[0038] The multi-satellite collaborative observation update model is:

[0039]

[0040] Where Q'(s,a) is the state update action value, Q(s,a) is the state action value corresponding to the action information corresponding to the state information in the state action table information, s is the state information, s0 is the initial state information, a is the action information, s' is the updated state information, a' is the state information corresponding to the maximum state action value obtained after the updated state information executes all action information in the action space information, max a′ Q(s′,a′) is the maximum state-action value obtained after the updated state information executes all action information in the action space information, ∥s-s0∥ is the distance value between the initial state information and the state information, r is the reward value, α and β represent the preset learning rate and the preset weight parameter respectively, and γ is the preset discount factor.

[0041] S3362: Use the state update action value to update the state action table information to obtain updated state action table information.

[0042] A second aspect of an embodiment of the present invention discloses a multi-satellite collaborative observation device, characterized in that the device includes:

[0043] The acquisition module is used to obtain training data sets, task information, and time window information;

[0044] A first computing module is configured to fuse the training data set, the task information, and the time window information to obtain collaborative observation information to be processed;

[0045] The second calculation module is used to analyze and calculate the task information, the time window information and the collaborative observation information to be processed to obtain target collaborative observation information.

[0046] A third aspect of an embodiment of the present invention discloses another multi-satellite collaborative observation device, characterized in that the device includes:

[0047] processor;

[0048] a memory coupled to the processor and storing executable program code;

[0049] The processor calls the executable program code stored in the memory to execute part or all of the steps of the multi-satellite collaborative observation method disclosed in the first aspect of the embodiment of the present invention.

[0050] The fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and when the computer instructions are called, some or all steps of the multi-satellite collaborative observation method disclosed in the first aspect of the embodiment of the present invention are executed.

[0051] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0052] In an embodiment of the present invention, a training data set, mission information, and time window information are obtained; the training data set, mission information, and time window information are fused to obtain collaborative observation information to be processed; and the mission information, time window information, and collaborative observation information to be processed are analyzed and calculated to obtain target collaborative observation information. This application can accelerate the decision-making process for multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of search optimization for satellite mission planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 A schematic diagram of a multi-satellite collaborative observation method disclosed in an embodiment of the present invention;

[0055] Figure 2 Inputting characteristic schematic information for the training samples of the multi-satellite collaborative observation disclosed in the embodiment of the present invention;

[0056] Figure 3 This is a schematic structural diagram of a multi-satellite collaborative observation device disclosed in an embodiment of the present invention;

[0057] Figure 4 This is a schematic structural diagram of another multi-satellite collaborative observation device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.

[0060] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] The present invention discloses a multi-satellite collaborative observation method and apparatus, which facilitates accelerating the decision-making process for multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing for satellite mission planning. These methods are described in detail below.

[0062] Example 1

[0063] See also Figure 1 , Figure 1 This is a flow chart of a multi-satellite collaborative observation method disclosed in an embodiment of the present invention. Figure 1 The multi-satellite collaborative observation method described is applied to a multi-satellite collaborative observation device, such as a local server or cloud server for optimizing and managing multi-satellite collaborative observation, and the embodiments of the present invention do not limit this. Figure 1 As shown, the multi-satellite collaborative observation method may include the following operations:

[0064] S1, obtain training data set, task information and time window information;

[0065] It should be noted that the training data set is a data set that can perform machine learning on multi-satellite collaborative observations. For example, the training data set is artificial experience historical planning data or planning data obtained using genetic algorithms, backtracking algorithms, reinforcement learning algorithms, etc. Specifically, the embodiments of the present invention do not limit this.

[0066] It should be noted that the time window information is the time window information of the task area corresponding to the scanning task information calculated through the open source satellite library, wherein the time window information includes several time windows.

[0067] S2, fusing the training data set, task information, and time window information to obtain the collaborative observation information to be processed;

[0068] S3, analyze and calculate the task information, time window information and collaborative observation information to be processed to obtain target collaborative observation information.

[0069] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0070] In an optional embodiment, the training data set, task information, and time window information are fused to obtain collaborative observation information to be processed, including:

[0071] S21, preprocessing the training data set to obtain a preprocessed training data set;

[0072] It should be noted that the above-mentioned preprocessing is to perform operations such as data cleaning, noise removal, normalization, and data enhancement on the training data set. The specific preprocessing method can be set by the user or obtained by conventional machine learning data set preprocessing methods. The embodiments of the present invention do not make specific limitations.

[0073] S22, using the preprocessed training data set, training the preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model;

[0074] S23, using a multi-satellite collaborative observation optimization model, calculates and processes the mission information and the time window information to obtain collaborative observation information to be processed.

[0075] It should be noted that the above-mentioned use of the multi-satellite collaborative observation optimization model to calculate and process the task information and time window information to obtain the collaborative observation information to be processed is to use the task information and time window information as the input of the multi-satellite collaborative observation optimization model, and the collaborative observation information to be processed is used as the output of the multi-satellite collaborative observation optimization model.

[0076] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0077] In another optional embodiment, a pre-processed training data set is used to train a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model, including:

[0078] S221, processing the preprocessed training data set to obtain a target training data set;

[0079] It should be noted that the process of processing the preprocessed training data set to obtain the target training data set is as follows:

[0080] The preprocessed training data set includes task information and window information for training, and the window information in the preprocessed training data set includes a plurality of time windows;

[0081] In order to facilitate the description of the multi-satellite collaborative mission decision-making problem, the following definitions are made:

[0082] x i =<f1,f2,…,f n >;

[0083] Where x i represents the time window, f * is the task feature in the task information, n is the total number of task features in the task information, and i is the number of the time window.

[0084] X={x i |i∈[1,N],N∈Z +};

[0085] Where X is the task feature data, N is the number of time windows, and (x1, ..., x i-1 ) is x i The predecessor time window sequence, (x i+1 ,…,x N ) is x i The subsequent time window sequence of .

[0086] Y={y i |i∈[1,N],N∈Z +};

[0087] In the formula, Y represents the actual selection result corresponding to X, y i Represents each time window x i The corresponding real selection results.

[0088] Y'={y i'|i∈[1,N],N∈Z +};

[0089] In the formula, Y' represents the prediction result corresponding to X, y i 'Indicates the prediction result corresponding to each time window.

[0090] Train={<X i ,Y i >|i∈[1,M],M∈Z +};

[0091] Where Train represents the target training dataset, and M represents the number of sample batches in the target training dataset.

[0092] Test={X i |i∈[1,m],m∈Z +};

[0093] Where Test represents the sample data information in the target training dataset, and m represents the number of sample batches in the target training dataset.

[0094] The neural network input feature information in the target training data set is shown in Table 1:

[0095] Table 1 Neural network input feature information

[0096]

[0097]

[0098] See Figure 2 , combining task information and time window information as neural network input feature information (excluding the last column, which is the training sample label value). The specific meaning of each field is: train_df = df[["Window Number","Task Type","Task Priority","Point Observation / Regional Observation","Longitude 1","Latitude 1","Longitude 2","Latitude 2","Longitude 3","Latitude 3","Longitude 4","Latitude 4","Weather Cloud Cover","Satellite ID","Payload Type","Resolution","Window Start Time","Window End Time","Task Coverage","Window Score","Whether This Window Is Selected"]].

[0099] S222, using the target training data set, training the multi-satellite collaborative observation model to obtain training result information and the multi-satellite collaborative observation training model;

[0100] It should be noted that the multi-satellite collaborative observation model is a 4-layer LSTM. The input is the target training dataset designed in the previous step, and the output is the training result information. Each LSTM time step is a time window, and each input is task information and window information.

[0101] S223, using the loss function, calculating and processing the training result information to obtain a loss function value;

[0102] S224, determining whether the loss function value is less than a preset loss function threshold, and obtaining a first determination result;

[0103] When the first judgment result is no, determining that the multi-satellite collaborative observation training model is a multi-satellite collaborative observation model, executing S222;

[0104] When the first judgment result is yes, the multi-satellite collaborative observation training model is determined to be a multi-satellite collaborative observation optimization model.

[0105] It should be noted that the preset loss function threshold value range is [0, 0.5], which is not limited in the specific embodiment of the present invention.

[0106] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0107] In yet another optional embodiment, the loss function is:

[0108]

[0109] Among them, Loss is the loss function value, P(x i,j ′) is the result information of the jth task feature of the i-th sample in the training result information, P(x i,j ) is the real result information corresponding to the jth task feature of the i-th sample, w j is the weight coefficient of the jth task feature, N is the number of samples, M is the number of task features in the sample, and α is the bias coefficient.

[0110] It should be noted that w j (1≤j≤M) and the deviation coefficient may be set by the user or obtained based on historical data, and the embodiment of the present invention does not limit this.

[0111] It should be noted that P(x i,j )(1≤i≤N,1≤j≤M) is set by the user or obtained through historical data, and is not limited in the embodiment of the present invention.

[0112] It should be noted that the target training dataset includes several samples, each of which contains several task features.

[0113] It should be noted that the value of the sample number N is at least 20,000, and the embodiment of the present invention does not limit this.

[0114] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0115] In an optional embodiment, the task information, the time window information, and the collaborative observation information to be processed are analyzed and calculated to obtain the target collaborative observation information, including:

[0116] S31, determining that the collaborative observation information to be processed is initial state information;

[0117] S32, fusing the task information and the time window information to obtain state space information and action space information;

[0118] It should be noted that the task information and time window information are fused to obtain state space information and action space information, including:

[0119] In the context of multi-satellite collaborative observation, state information refers to the time window selection of the mission information. For example, selecting the first five time windows from all time windows in the time window information is one type of state information, and selecting the first ten time windows is another type of state information. Here, s is used to represent the state information. In the above example, the state information is s = [0, 1, 2, 3, 4], [0, 1, 2, ..., 9]. The collection of all state information in the mission information is the state space information.

[0120] Action information refers to the actions that the task information can take under each state information. The possible operations for each state information are not necessarily the same. For example, in this embodiment of the present invention, for each state information s, there are (the total number of time windows in the time window information - the time windows included in s) action information. Using a to represent the action information, the set of all actions that the task information can take under all state information is called the action space information.

[0121] S33, processing the state space information, action space information and initial state information to obtain target collaborative observation information.

[0122] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0123] In an optional embodiment, the state space information, the action space information, and the initial state information are processed to obtain target collaborative observation information, including:

[0124] S331, processing the state space information and the action space information to obtain state-action table information;

[0125] It should be noted that all state information in the state space information is used as the row attribute of the state-action table information, and all action information in the action space information is used as the column attribute of the state-action table information. The value obtained from any column corresponding to any row in the state-action table information is used as the state-action value of the action information in the column corresponding to the state information of the row. In the initial state, all state-action values ​​are set to 0, as shown in Table 2 below, where s0-sn are the state information in the state-action table information, and a1-am are the action information in the state-action table information. The value of the action information a1 corresponding to the state information s0 is the state-action value, which is initially 0:

[0126] Table 2 Initial value of state action table information

[0127]

[0128] S332, determining that the initial state information is state information;

[0129] S333, preset t=1;

[0130] S334, calculating and processing the state information and the state action table information to obtain action information;

[0131] It should be noted that the action information is obtained by calculating and processing the state information and the state-action table information, which is obtained by using the collaborative observation probability model;

[0132] Among them, the multi-satellite collaborative observation probability model is:

[0133]

[0134] Where DZ is the action information, s is the state information, argmaxQ(s, DZ) is the action information corresponding to the maximum action state value corresponding to the state information in the state-action table information, DS is any action information except argmaxQ(s, DZ) corresponding to the state information in the state-action table, p is the preset decay rate, st is the preset decay step number, and δ is the preset first weight parameter.

[0135] It should be noted that the preset attenuation rate is set between [0.01, 0.1], and the preset number of attenuation steps is set between [10, 50]. Specifically, the embodiment of the present invention does not limit this.

[0136] It should be noted that the above-mentioned collaborative observation probability model indicates that DZ has The possibility of choosing argmaxQ(s,DZ) as action information is The possibility of selecting DS as action information.

[0137] It should be noted that the above calculation and processing of the state information and the state-action table information to obtain the action information may also be performed using an exponential decay function or a linear decay function, which is not limited in the embodiment of the present invention.

[0138] S335, calculating and processing the action information and state information to obtain a reward value;

[0139] It should be noted that the calculation and processing of action information and state information to obtain a reward value is performed by determining whether the executed action information satisfies the constraint information corresponding to the state information when the state information executes the action information. If so, a preset value D is obtained, which is a number greater than 0 (specifically, this is not limited in this embodiment of the present invention). If not, a preset value F is obtained, which is a number less than 0 (specifically, this is not limited in this embodiment of the present invention), and the obtained value D or F is used as the reward value. The specific constraint information and judgment process are not specifically limited in this embodiment of the present invention.

[0140] It should be noted that the above calculation and processing of the action information and state information to obtain the reward value can be performed using the OpenAI Gym tool, and this embodiment of the present invention does not limit this.

[0141] It should be noted that the above reward value can also be obtained using the following methods.

[0142] When performing tasks, agents typically consume resources, such as time, energy, or computational resources. Reward values ​​can be set based on resource savings or consumption.

[0143] Resource Optimization Rewards:

[0144] Actions that reduce resource consumption are rewarded, while actions that reduce resource consumption are penalized. For example, if a satellite selects fewer time windows but still completes its mission, it will be rewarded because it consumes fewer resources.

[0145] r=Initial_resource-Used_resource;

[0146] Where r is the reward value, Initial_resource is the initial resource, and Used_resource is the used resource.

[0147] Resource overconsumption penalty:

[0148] If the agent consumes too many resources during task execution, a negative reward is given.

[0149] r=-CF·(Used_resource-Expected_resource)

[0150] Among them, CF is a penalty coefficient representing the severity of resource overconsumption, and expected_resource is the expected resource;

[0151] It should be noted that the penalty coefficient has a value range of [0.01, 0.05], which is not limited in the embodiment of the present invention.

[0152] It should be noted that the specific values ​​of initial resources, used resources, and expected resources may be obtained from a monitoring tool (such as Prometheus) or based on historical data, and the embodiment of the present invention does not limit this.

[0153] S336, processing the initial state information, the state information, the action information, the state-action table information, and the reward value to obtain updated state information and updated state-action table information, and determining the updated state information as the state information and the updated state-action table information as the state-action table information;

[0154] S337, determining whether t is greater than a preset number of iterations, and obtaining a first determination result;

[0155] When the first judgment result is no, t is increased by 1 and S334 is executed;

[0156] When the first judgment result is yes, the state-action table information is determined to be target collaborative observation information.

[0157] It should be noted that the preset number of iterations is in the range of more than 1000, and is not specifically limited in the embodiment of the present invention.

[0158] It should be noted that when the first judgment result is yes, the following operations may also be performed to obtain target collaborative observation information;

[0159] Step 1: Obtain a historical state action table information set; the historical state action table information set includes several historical state action table information;

[0160] Step 2: Using the first calculation model of multi-satellite collaborative observation, the historical state-action table information set is calculated and processed to obtain pre-processed state-action table information;

[0161] Among them, the first calculation model of multi-satellite collaborative observation is:

[0162]

[0163] Where, QS is the pre-processing state action table information, LS i is the i-th historical state action table information in the historical state action table information set, G is the number of historical state action table information in the historical state action table information set;

[0164] Step 3: Using the second calculation model of multi-satellite collaborative observation, the pre-processed state-action table information and the state-action table information are calculated and processed to obtain target collaborative observation information;

[0165] Among them, the second calculation model of multi-satellite collaborative observation is:

[0166] MB=δ2·QS+δ3·QP 0<δ2, δ3<1;

[0167] δ2+δ3=1;

[0168] Where MB is the target collaborative observation information, QS is the pre-processed state action table information, QP is the state action table information, δ2 and δ3 are the second and third weight parameters;

[0169] It should be noted that the second weight parameter and the third weight parameter may be set by the user or obtained based on historical data, and the embodiment of the present invention does not make any specific limitation thereto.

[0170] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0171] In an optional embodiment, the initial state information, state information, action information, state-action table information, and reward value are processed to obtain updated state information and updated state-action table information, including:

[0172] S3361, using a multi-satellite collaborative observation update model, processes the initial state information, state information, action information, state-action table information, and reward value to obtain a state update action value and updated state information;

[0173] Among them, the multi-satellite collaborative observation update model is:

[0174]

[0175] In the formula, Q'(s,a) is the state update action value, Q(s,a) is the state action value corresponding to the action information corresponding to the state information in the state action table information, s is the state information, s0 is the initial state information, a is the action information, s' is the updated state information, a' is the state information corresponding to the maximum state action value obtained after the updated state information executes all the action information in the action space information, max a′ Q(s′,a′) is the maximum state-action value obtained after the updated state information executes all action information in the action space information, ∥s-s0∥ is the distance value between the initial state information and the state information, r is the reward value, α and β represent the preset learning rate and the preset weight parameter respectively, and γ is the preset discount factor.

[0176] It should be noted that the action information a is added to the state information s to obtain the updated state information.

[0177] It should be noted that ||s-s0|| can be calculated using the Euclidean distance calculation formula, and this embodiment of the present invention does not specifically limit this.

[0178] It should be noted that the above-mentioned multi-satellite collaborative observation update model is used to process the initial state information, state information, action information, state action table information and reward value to obtain the state update action value and the updated state information. It can also be implemented using the Q-table update operation in the Q-learning algorithm. Specifically, the embodiment of the present invention does not limit this.

[0179] S3362: Use the state update action value to update the state action table information to obtain updated state action table information.

[0180] It should be noted that the above update process is to set the state update action value to the state action value of the action information corresponding to the state information in the state action table information.

[0181] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0182] As shown in Table 3, Table 3 is an analysis and comparison table of the embodiment of the present invention and the traditional multi-satellite collaborative observation method.

[0183] Table 3 Analysis and comparison of the embodiment of the present invention and the traditional multi-satellite collaborative observation method

[0184]

[0185]

[0186] As can be seen, under the same score, the implementation of the embodiment of the present invention significantly outperforms the traditional multi-satellite collaborative observation method in terms of time consumption. Implementing the multi-satellite collaborative observation method described in the embodiment of the present invention helps accelerate the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of search optimization for satellite mission planning.

[0187] Example 2

[0188] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a multi-satellite collaborative observation device disclosed in an embodiment of the present invention. Figure 3 The multi-satellite collaborative observation device described is applied to a multi-satellite collaborative observation optimization system, such as a local server or cloud server for multi-satellite collaborative observation, etc., which is not limited in the embodiment of the present invention. Figure 3 As shown, the multi-satellite collaborative observation device includes:

[0189] Acquisition module 201, used to obtain training data set, task information and time window information;

[0190] The first computing module 202 is configured to perform a fusion process on the training data set, task information, and time window information to obtain collaborative observation information to be processed;

[0191] The second calculation module 203 is used to analyze and calculate the task information, the time window information and the collaborative observation information to be processed to obtain the target collaborative observation information.

[0192] It can be seen that the implementation of the multi-satellite collaborative observation device described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0193] Example 3

[0194] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of another multi-satellite collaborative observation device disclosed in an embodiment of the present invention. Figure 4 The multi-satellite collaborative observation device described is applied to a multi-satellite collaborative observation optimization system, such as a local server or cloud server for multi-satellite collaborative observation, etc., which is not limited in the embodiment of the present invention. Figure 4 As shown, the multi-satellite collaborative observation device includes:

[0195] Processor 301;

[0196] A memory 302 coupled to the processor 301 and storing executable program code;

[0197] The processor 301 calls the executable program code stored in the memory 302 to execute part or all of the steps of the multi-satellite collaborative observation method of the first embodiment.

[0198] Example 4

[0199] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which, when called, are used to execute some or all steps of the multi-satellite collaborative observation method of embodiment one.

[0200] Example 5

[0201] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute some or all of the steps in the multi-satellite collaborative observation method described in Example 1.

[0202] The system embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network modules. Some or all of these modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0203] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0204] Finally, it should be noted that the multi-satellite collaborative observation method and device disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multi-satellite collaborative observation method, characterized in that: The method comprises: S1, obtain training data set, task information and time window information; S2, fusing the training data set, the task information, and the time window information to obtain collaborative observation information to be processed; S3, analyzing and calculating the task information, the time window information, and the collaborative observation information to be processed to obtain target collaborative observation information; The analyzing and calculating the task information, the time window information, and the collaborative observation information to be processed to obtain target collaborative observation information includes: S31, determining that the collaborative observation information to be processed is initial state information; S32, fusing the task information and the time window information to obtain state space information and action space information; S33, processing the state space information, the action space information, and the initial state information to obtain target collaborative observation information; The processing of the state space information, the action space information, and the initial state information to obtain target collaborative observation information includes: S331, processing the state space information and the action space information to obtain state-action table information; S332, determining that the initial state information is state information; S333, preset t=1; S334, performing calculation processing on the state information and the state-action table information to obtain action information; S335, calculating and processing the action information and the state information to obtain a reward value; S336, processing the initial state information, the state information, the action information, the state-action table information, and the reward value to obtain updated state information and updated state-action table information, and determining the updated state information as the state information and the updated state-action table information as the state-action table information; S337, determining whether t is greater than a preset number of iterations, and obtaining a first determination result; When the first judgment result is no, increment t by 1 and execute S334; When the first judgment result is yes, obtaining a historical state action table information set; the historical state action table information set includes a plurality of historical state action table information; Using a first calculation model of multi-satellite collaborative observation, the historical state-action table information set is calculated and processed to obtain pre-processed state-action table information; The first calculation model of multi-satellite collaborative observation is: Where, QS is the pre-processing state action table information, LS i is the i-th historical state action table information in the historical state action table information set, and G is the number of historical state action table information in the historical state action table information set; Using a second calculation model for multi-satellite collaborative observation, the pre-processed state-action table information and the state-action table information are calculated and processed to obtain target collaborative observation information; The second calculation model of multi-satellite collaborative observation is: MB=δ2·QS+δ3·QP 0<δ2, δ3<1; δ2+δ3=1; Where MB is the target collaborative observation information, QS is the pre-processed state action table information, QP is the state action table information, δ2 and δ3 are the second weight parameter and the third weight parameter; The calculating and processing the state information and the state action table information to obtain the action information includes: Utilizing a multi-satellite collaborative observation probability model, calculating and processing the state information and the state-action table information to obtain action information; Among them, the multi-satellite collaborative observation probability model is: Where DZ is the action information, s is the state information, argmaxQ(s, DZ) is the action information corresponding to the maximum action state value corresponding to the state information in the state-action table information, DS is any action information except argmaxQ(s, DZ) corresponding to the state information in the state-action table, p is the preset decay rate, st is the preset decay step number, and δ is the preset first weight parameter.

2. The multi-satellite collaborative observation method according to claim 1, characterized in that: The fusing the training data set, the task information, and the time window information to obtain the collaborative observation information to be processed includes: S21, preprocessing the training data set to obtain a preprocessed training data set; S22, using the preprocessed training data set, training a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model; S23: Utilize the multi-satellite collaborative observation optimization model to perform calculations on the mission information and the time window information to obtain collaborative observation information to be processed.

3. The multi-satellite collaborative observation method according to claim 2, characterized in that: The method of using the pre-processed training data set to train a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model includes: S221, processing the preprocessed training data set to obtain a target training data set; S222: Training the multi-satellite collaborative observation model using the target training data set to obtain training result information and a multi-satellite collaborative observation training model; S223, using a loss function to calculate and process the training result information to obtain a loss function value; S224, determining whether the loss function value is less than a preset loss function threshold, and obtaining a first determination result; When the first judgment result is no, determining that the multi-satellite collaborative observation training model is the multi-satellite collaborative observation model, executing S222; When the first judgment result is yes, it is determined that the multi-satellite collaborative observation training model is a multi-satellite collaborative observation optimization model.

4. The multi-satellite collaborative observation method according to claim 3, characterized in that: The loss function is: 1≤i≤N; 1≤j≤M; Among them, Loss is the loss function value, P(x i,j ′) is the result information of the jth task feature of the i-th sample in the training result information, P(x i,j ) is the true result information corresponding to the jth task feature of the i-th sample, w j is the weight coefficient of the jth task feature, N is the number of samples, M is the number of task features in the samples, and α is the deviation coefficient.

5. The multi-satellite collaborative observation method according to claim 1, characterized in that: The processing of the initial state information, the state information, the action information, the state-action table information, and the reward value to obtain updated state information and updated state-action table information includes: S3361: Using a multi-satellite collaborative observation update model, processing the initial state information, the state information, the action information, the state-action table information, and the reward value to obtain a state update action value and updated state information; The multi-satellite collaborative observation update model is: Where Q'(s,a) is the state update action value, Q(s,a) is the state action value corresponding to the action information corresponding to the state information in the state action table information, s is the state information, s0 is the initial state information, a is the action information, s' is the updated state information, a' is the state information corresponding to the maximum state action value obtained after the updated state information executes all action information in the action space information, max a′ Q(s′, a′) is the maximum state-action value obtained after the updated state information executes all action information in the action space information, ||s-s0|| is the distance value between the initial state information and the state information, r is the reward value, α and β represent the preset learning rate and the preset weight parameter respectively, and γ is the preset discount factor; S3362: Use the state update action value to update the state action table information to obtain updated state action table information.

6. A multi-satellite collaborative observation device, characterized in that: The device comprises: processor; a memory coupled to the processor and storing executable program code; The processor calls the executable program code stored in the memory to execute the multi-satellite collaborative observation method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the multi-satellite collaborative observation method according to any one of claims 1 to 5.

Images