Multi-satellite intelligent collaborative observation method based on model acceleration reinforcement learning

Abstract

The invention discloses a multi-satellite collaborative observation method and device, and the method comprises the steps: obtaining a training data set, task information and time window information; performing fusion processing on the training data set, the task information and the time window information to obtain to-be-processed collaborative observation information; and analyzing and calculating the task information, the time window information and the to-be-processed collaborative observation information to obtain target collaborative observation information. Therefore, according to the method, the decision speed of multi-satellite collaborative observation can be accelerated, so that the searching and optimizing difficulty and time consumption of satellite task planning are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-satellite autonomous collaborative observation in land and resources survey, and in particular to a multi-satellite intelligent collaborative observation method based on model accelerated reinforcement learning. Background Art

[0002] At present, satellite remote sensing has become an important means of land and resources survey and disaster monitoring. With the continuous increase in mission requirements, the reasonable scheduling of satellites has become particularly important. In the existing satellite mission planning research, most of the imaging satellite mission planning problems are modeled as optimization problems. However, the existing satellite collaborative mission modeling mostly adopts combinatorial optimization or machine scheduling models, and the solution algorithms mostly use heuristic search or intelligent optimization algorithms. The satellite mission planning problem is an NP-Hard problem, and the difficulty and time consumption of its search and optimization increase sharply with the increase of the problem scale. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a multi-satellite intelligent collaborative observation method based on model accelerated reinforcement learning, which is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the search and optimization difficulty and time consumption of satellite mission planning.

[0004] In order to solve the above technical problems, a first aspect of an embodiment of the present invention discloses a multi-satellite collaborative observation method, the method comprising:

[0005] S1, obtain training data set, task information and time window information;

[0006] S2, fusing the training data set, the task information and the time window information to obtain collaborative observation information to be processed;

[0007] S3, analyzing and calculating the task information, the time window information and the collaborative observation information to be processed to obtain target collaborative observation information.

[0008] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the fusing processing of the training data set, the task information and the time window information to obtain the collaborative observation information to be processed includes:

[0009] S21, preprocessing the training data set to obtain a preprocessed training data set;

[0010] S22, using the preprocessed training data set, training a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model;

[0011] S23, using the multi-satellite collaborative observation optimization model, calculating and processing the task information and the time window information to obtain collaborative observation information to be processed.

[0012] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the using of the preprocessed training data set to train a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model includes:

[0013] S221, processing the preprocessed training data set to obtain a target training data set;

[0014] S222, using the target training data set, training the multi-satellite collaborative observation model to obtain training result information and a multi-satellite collaborative observation training model;

[0015] S223, using a loss function, calculating and processing the training result information to obtain a loss function value;

[0016] S224, determining whether the loss function value is less than a preset loss function threshold, and obtaining a first determination result;

[0017] When the first judgment result is no, determining that the multi-satellite collaborative observation training model is the multi-satellite collaborative observation model, and executing S222;

[0018] When the first judgment result is yes, it is determined that the multi-satellite collaborative observation training model is a multi-satellite collaborative observation optimization model.

[0019] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the loss function is:

[0020]

[0021] Wherein, Loss is the loss function value, P(x i,j ′) is the result information of the jth task feature of the i-th sample in the training result information, P(x i,j ) is the real result information corresponding to the jth task feature of the i-th sample, w j is the weight coefficient of the jth task feature, N is the number of samples, M is the number of task features in the samples, and α is the deviation coefficient.

[0022] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the analyzing and calculating the task information, the time window information and the collaborative observation information to be processed to obtain the target collaborative observation information includes:

[0023] S31, determining that the collaborative observation information to be processed is initial state information;

[0024] S32, fusing the task information and the time window information to obtain state space information and action space information;

[0025] S33, processing the state space information, the action space information and the initial state information to obtain target collaborative observation information.

[0026] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the state space information, the action space information and the initial state information to obtain the target collaborative observation information includes:

[0027] S331, processing the state space information and the action space information to obtain state action table information;

[0028] S332, determining that the initial state information is state information;

[0029] S333, preset t=1;

[0030] S334, calculating and processing the state information and the state action table information to obtain action information;

[0031] S335, calculating and processing the action information and the state information to obtain a reward value;

[0032] S336, processing the initial state information, the state information, the action information, the state-action table information and the reward value to obtain updated state information and updated state-action table information, and determining the updated state information as the state information, and determining the updated state-action table information as the state-action table information;

[0033] S337, determining whether t is greater than a preset number of iterations, and obtaining a first determination result;

[0034] When the first judgment result is no, increase the t by 1 and execute S334;

[0035] When the first judgment result is yes, it is determined that the state action table information is the target collaborative observation information.

[0036] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the initial state information, the state information, the action information, the state action table information and the reward value to obtain updated state information and updated state action table information includes:

[0037] S3361, using a multi-satellite collaborative observation update model, processing the initial state information, the state information, the action information, the state action table information and the reward value to obtain a state update action value and updated state information;

[0038] The multi-satellite collaborative observation update model is:

[0039]

[0040] Wherein, Q′(s,a) is the state update action value, Q(s,a) is the state action value corresponding to the action information corresponding to the state information in the state action table information, s is the state information, s0 is the initial state information, a is the action information, s′ is the updated state information, a′ is the state information corresponding to the maximum state action value obtained after the updated state information executes all the action information in the action space information, max a′ Q(s′,a′) is the maximum state-action value obtained after the updated state information executes all action information in the action space information, ∥s-s0∥ is the distance value between the initial state information and the state information, r is the reward value, α and β represent the preset learning rate and the preset weight parameter respectively, and γ is the preset discount factor.

[0041] S3362: Use the state update action value to update the state action table information to obtain updated state action table information.

[0042] A second aspect of an embodiment of the present invention discloses a multi-satellite collaborative observation device, characterized in that the device comprises:

[0043] The acquisition module is used to obtain the training data set, task information and time window information;

[0044] A first computing module is used to perform a fusion process on the training data set, the task information and the time window information to obtain collaborative observation information to be processed;

[0045] The second calculation module is used to analyze and calculate the task information, the time window information and the collaborative observation information to be processed to obtain target collaborative observation information.

[0046] A third aspect of an embodiment of the present invention discloses another multi-satellite collaborative observation device, characterized in that the device comprises:

[0047] processor;

[0048] a memory coupled to the processor and storing executable program code;

[0049] The processor calls the executable program code stored in the memory to execute part or all of the steps of the multi-satellite collaborative observation method disclosed in the first aspect of the embodiment of the present invention.

[0050] The fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and when the computer instructions are called, some or all steps of the multi-satellite collaborative observation method disclosed in the first aspect of the embodiment of the present invention are executed.

[0051] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0052] In an embodiment of the present invention, a training data set, task information and time window information are obtained; the training data set, the task information and the time window information are fused to obtain the collaborative observation information to be processed; the task information, the time window information and the collaborative observation information to be processed are analyzed and calculated to obtain the target collaborative observation information. It can be seen that the present application is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing for satellite mission planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 A schematic diagram of a process flow of a multi-satellite collaborative observation method disclosed in an embodiment of the present invention;

[0055] Figure 2 Inputting characteristic schematic information for the training samples of multi-satellite collaborative observation disclosed in the embodiment of the present invention;

[0056] Figure 3 A schematic diagram of the structure of a multi-satellite collaborative observation device disclosed in an embodiment of the present invention;

[0057] Figure 4 The present invention is a schematic diagram of the structure of another multi-satellite collaborative observation device disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0060] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0061] The present invention discloses a multi-satellite collaborative observation method and device, which is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing for satellite mission planning. Detailed descriptions are given below.

[0062] Embodiment 1

[0063] See also Figure 1 , Figure 1 : is a flow chart of a multi-satellite collaborative observation method disclosed in an embodiment of the present invention. Figure 1 The multi-satellite collaborative observation method described is applied to a multi-satellite collaborative observation device, such as a local server or cloud server for optimizing and managing multi-satellite collaborative observation, and the embodiments of the present invention do not limit this. Figure 1 As shown, the multi-satellite collaborative observation method may include the following operations:

[0064] S1, obtain training data set, task information and time window information;

[0065] It should be noted that the training data set is a data set that can perform machine learning on multi-satellite collaborative observations. For example, the training data set is artificial experience historical planning data or planning data obtained using genetic algorithms, backtracking algorithms, reinforcement learning algorithms, etc. Specifically, the embodiments of the present invention do not limit this.

[0066] It should be noted that the time window information is the time window information of the task area corresponding to the scanning task information calculated through the open source satellite library, wherein the time window information includes a plurality of time windows.

[0067] S2, fusing the training data set, task information and time window information to obtain the collaborative observation information to be processed;

[0068] S3, analyzing and calculating the task information, time window information and collaborative observation information to be processed to obtain target collaborative observation information.

[0069] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0070] In an optional embodiment, the training data set, the task information, and the time window information are fused to obtain the collaborative observation information to be processed, including:

[0071] S21, preprocessing the training data set to obtain a preprocessed training data set;

[0072] It should be noted that the above-mentioned preprocessing is to perform operations such as data cleaning, noise removal, normalization, and data enhancement on the training data set. The specific preprocessing method can be set by the user or obtained by processing using a conventional machine learning data set preprocessing method. The embodiment of the present invention does not make specific limitations.

[0073] S22, using the preprocessed training data set, training the preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model;

[0074] S23, using a multi-satellite collaborative observation optimization model, calculates and processes the mission information and the time window information to obtain collaborative observation information to be processed.

[0075] It should be noted that the above-mentioned use of the multi-satellite collaborative observation optimization model to calculate and process the task information and the time window information to obtain the collaborative observation information to be processed is to use the task information and the time window information as the input of the multi-satellite collaborative observation optimization model, and the collaborative observation information to be processed is used as the output of the multi-satellite collaborative observation optimization model.

[0076] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0077] In another optional embodiment, a preset multi-satellite collaborative observation model is trained using a preprocessed training data set to obtain a multi-satellite collaborative observation optimization model, including:

[0078] S221, processing the preprocessed training data set to obtain a target training data set;

[0079] It should be noted that the process of processing the preprocessed training data set to obtain the target training data set is as follows:

[0080] The preprocessed training data set includes task information and window information for training, and the window information in the preprocessed training data set includes a plurality of time windows;

[0081] In order to facilitate the description of the multi-satellite collaborative mission decision-making problem, the following definition is made:

[0082] x i = <f1,f2,…,f n >;

[0083] In the formula, x i represents the time window, f * is the task feature in the task information, n is the total number of task features in the task information, and i is the number of the time window.

[0084] X={x i |i∈[1,N],N∈Z +};

[0085] In the formula, X is the task feature data, N is the number of time windows, and (x1, ..., x i-1 ) is x i The predecessor time window sequence, (x i+1 , …, x N ) is x i The subsequent time window sequence of .

[0086] Y={y i |i∈[1,N],N∈Z +};

[0087] In the formula, Y represents the actual selection result corresponding to X, y i Represents each time window x i The corresponding actual selection result.

[0088] Y'={y i'|i∈[1,N],N∈Z +};

[0089] In the formula, Y' represents the prediction result corresponding to X, y i 'Indicates the prediction result corresponding to each time window.

[0090] Train = {<X i ,Y i >|i∈[1,M],M∈Z +};

[0091] In the formula, Train represents the target training dataset, and M represents the number of sample batches in the target training dataset.

[0092] Test = {X i |i∈[1,m],m∈Z +};

[0093] In the formula, Test represents the sample data information in the target training data set, and m represents the number of sample batches in the target training data set.

[0094] The neural network input feature information in the target training data set is shown in Table 1:

[0095] Table 1 Neural network input feature information

[0096]

[0097]

[0098] See also Figure 2 , combining the task information and time window information as neural network input feature information (excluding the last column, which is the training sample label value). The specific meaning of each field is: train_df = df[["window sequence number","task type","task priority","point observation / regional observation","longitude 1","latitude 1","longitude 2","latitude 2","longitude 3","latitude 3","longitude 4","latitude 4","weather cloud cover","satellite id","payload type","resolution","window start time","window end time","task coverage","window score","whether this window is selected"]].

[0099] S222, training the multi-satellite collaborative observation model using the target training data set to obtain training result information and the multi-satellite collaborative observation training model;

[0100] It should be noted that the multi-satellite collaborative observation model is a 4-layer LSTM. The input is the target training data set designed in the previous step, and the output is the training result information. Each LSTM time step is a time window, and each input is task information and window information.

[0101] S223, using the loss function, calculating and processing the training result information to obtain a loss function value;

[0102] S224, determining whether the loss function value is less than a preset loss function threshold, and obtaining a first determination result;

[0103] When the first judgment result is no, determining that the multi-satellite collaborative observation training model is a multi-satellite collaborative observation model, and executing S222;

[0104] When the first judgment result is yes, the multi-satellite collaborative observation training model is determined to be a multi-satellite collaborative observation optimization model.

[0105] It should be noted that the preset loss function threshold value ranges from [0, 0.5], which is not limited in the specific embodiment of the present invention.

[0106] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0107] In yet another optional embodiment, the loss function is:

[0108]

[0109] Among them, Loss is the loss function value, P(x i,j ′) is the result information of the jth task feature of the i-th sample in the training result information, P(x i,j ) is the real result information corresponding to the jth task feature of the i-th sample, w j is the weight coefficient of the jth task feature, N is the number of samples, M is the number of task features in the sample, and α is the deviation coefficient.

[0110] It should be noted that w j (1≤j≤M) and the deviation coefficient may be set by a user or obtained based on historical data, which is not limited in the embodiment of the present invention.

[0111] It should be noted that P(x i,j )(1≤i≤N,1≤j≤M) is set by the user or obtained through historical data, and is not limited in the embodiment of the present invention.

[0112] It should be noted that the target training data set includes several samples, each of which contains several task features.

[0113] It should be noted that the value of the sample quantity N is at least 20,000, and the embodiment of the present invention does not limit this.

[0114] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0115] In an optional embodiment, the task information, the time window information and the collaborative observation information to be processed are analyzed and calculated to obtain the target collaborative observation information, including:

[0116] S31, determining that the collaborative observation information to be processed is initial state information;

[0117] S32, fusing the task information and the time window information to obtain state space information and action space information;

[0118] It should be noted that the task information and time window information are fused to obtain the state space information and action space information, including:

[0119] In the scenario of multi-satellite collaborative observation, state information refers to the time window selection of task information. For example, selecting the first 5 time windows from all time windows in the time window information is one kind of state information, and selecting the first 10 time windows is another kind of state information. Here, s is used to represent the state information. In the above example, the state information is s = [0, 1, 2, 3, 4], [0, 1, 2, ..., 9]. The set of all state information in the task information is the state space information;

[0120] Action information refers to the actions that the task information can take under each state information. The possible operations for each state information are not necessarily the same. For example, in an embodiment of the present invention, for each state information s, there are (the total number of time windows in the time window information - the time windows included in s) action information. Using a to represent the action information, the set of all actions that the task information can take under all state information becomes the action space information.

[0121] S33, processing the state space information, action space information and initial state information to obtain target collaborative observation information.

[0122] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0123] In an optional embodiment, the state space information, the action space information and the initial state information are processed to obtain the target collaborative observation information, including:

[0124] S331, processing the state space information and the action space information to obtain state action table information;

[0125] It should be noted that all state information in the state space information is used as the row attribute of the state action table information, and all action information in the action space information is used as the column attribute of the state action table information. The value obtained from any column corresponding to any row in the state action table information is used as the state action value of the action information in the column corresponding to the state information of the row. In the initial state, all state action values ​​are set to 0, as shown in the following Table 2, s0-sn is the state information in the state action table information, a1-am is the action information in the state action table information, and the value of the action information a1 corresponding to the state information s0 is the state action value, which is initially 0:

[0126] Table 2 Status action table information initial value

[0127]

[0128] S332, determining that the initial state information is state information;

[0129] S333, preset t=1;

[0130] S334, calculating and processing the state information and the state action table information to obtain action information;

[0131] It should be noted that the state information and the state action table information are calculated and processed to obtain the action information, which is calculated using the collaborative observation probability model;

[0132] Among them, the multi-satellite collaborative observation probability model is:

[0133]

[0134] Wherein, DZ is the action information, s is the state information, argmaxQ(s, DZ) is the action information corresponding to the maximum action state value corresponding to the state information in the state-action table information, DS is any action information except argmaxQ(s, DZ) corresponding to the state information in the state-action table, p is the preset decay rate, st is the preset decay step number, and δ is the preset first weight parameter.

[0135] It should be noted that the preset attenuation rate is between [0.01, 0.1], and the preset attenuation step number is between [10, 50]. Specifically, the embodiment of the present invention does not limit this.

[0136] It should be noted that the above-mentioned collaborative observation probability model indicates that DZ has The possibility of choosing argmaxQ(s,DZ) as action information is The possibility of selecting DS as action information.

[0137] It should be noted that the above-mentioned calculation and processing of the state information and the state-action table information to obtain the action information may also be performed using an exponential decay function or a linear decay function, which is not limited in detail in the embodiment of the present invention.

[0138] S335, calculating and processing the action information and the state information to obtain a reward value;

[0139] It should be noted that the action information and state information are calculated and processed to obtain the reward value. When the state information executes the action information, it is determined whether the executed action information satisfies the constraint information corresponding to the state information. If it satisfies, a preset D is obtained, and D is a number greater than 0 (specifically, the embodiment of the present invention does not limit it). If it does not satisfy, a preset F is obtained, and F is a number less than 0 (specifically, the embodiment of the present invention does not limit it), and the obtained D or F is used as the reward value. The specific constraint information and judgment process are not specifically limited in the embodiment of the present invention.

[0140] It should be noted that the above-mentioned calculation and processing of the action information and the state information to obtain the reward value can be calculated and processed using the OpenAI Gym tool, which is not limited in the embodiment of the present invention.

[0141] It should be noted that the above reward value can also be obtained using the following method.

[0142] Agents usually consume resources when performing tasks, such as time, energy, or computing resources. Reward values ​​can be set based on the savings or consumption of resources.

[0143] Resource Optimization Rewards:

[0144] When an action leads to a reduction in resource consumption, a reward is given, otherwise a penalty is imposed. For example, if the number of time windows selected by the satellite is reduced but the task is still completed, a reward is given because fewer resources are consumed.

[0145] r=Initial_resource-Used_resource;

[0146] Where r is the reward value, Initial_resource is the initial resource, and Used_resource is the used resource.

[0147] Resource overconsumption penalty:

[0148] If the agent consumes too many resources during task execution, a negative reward is given.

[0149] r=-CF·(Used_resource-Expected_resource)

[0150] Among them, CF is a penalty coefficient, representing the severity of overconsumption of resources, and expected_resource is the expected resource;

[0151] It should be noted that the penalty coefficient has a value range of [0.01, 0.05], which is not limited in the embodiment of the present invention.

[0152] It should be noted that the specific values ​​of initial resources, used resources and expected resources may be obtained based on a monitoring tool (such as Prometheus) or based on historical data, which is not limited in the embodiments of the present invention.

[0153] S336, processing the initial state information, the state information, the action information, the state-action table information and the reward value to obtain updated state information and updated state-action table information, and determining the updated state information as the state information, and determining the updated state-action table information as the state-action table information;

[0154] S337, determining whether t is greater than a preset number of iterations, and obtaining a first determination result;

[0155] When the first judgment result is no, t is increased by 1 and S334 is executed;

[0156] When the first judgment result is yes, the state action table information is determined to be target collaborative observation information.

[0157] It should be noted that the preset number of iterations is in a range of more than 1000, and is not limited in the embodiment of the present invention.

[0158] It should be noted that when the first judgment result is yes, the following operations may also be performed to obtain target collaborative observation information;

[0159] Step 1: Obtain a historical state action table information set; the historical state action table information set includes a plurality of historical state action table information;

[0160] Step 2: Using the first calculation model of multi-satellite collaborative observation, the historical state action table information set is calculated and processed to obtain the pre-processed state action table information;

[0161] Among them, the first calculation model of multi-satellite collaborative observation is:

[0162]

[0163] Where QS is the pre-processing state action table information, LS i is the i-th historical state action table information in the historical state action table information set, and G is the number of historical state action table information in the historical state action table information set;

[0164] Step 3, using the second calculation model of multi-satellite collaborative observation, calculate and process the pre-processed state action table information and the state action table information to obtain target collaborative observation information;

[0165] Among them, the second calculation model of multi-satellite collaborative observation is:

[0166] MB=δ2·QS+δ3·QP 0<δ2, δ3<1;

[0167] δ2+δ3=1;

[0168] Where MB is the target collaborative observation information, QS is the preprocessing state action table information, QP is the state action table information, δ2 and δ3 are the second weight parameter and the third weight parameter;

[0169] It should be noted that the second weight parameter and the third weight parameter may be set by a user or may be obtained based on historical data, and the embodiment of the present invention does not specifically limit this.

[0170] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0171] In an optional embodiment, the initial state information, the state information, the action information, the state action table information and the reward value are processed to obtain the updated state information and the updated state action table information, including:

[0172] S3361, using a multi-satellite collaborative observation update model, processing the initial state information, state information, action information, state action table information and reward value to obtain a state update action value and updated state information;

[0173] Among them, the multi-satellite collaborative observation update model is:

[0174]

[0175] In the formula, Q′(s,a) is the state update action value, Q(s,a) is the state action value corresponding to the action information corresponding to the state information in the state action table information, s is the state information, s0 is the initial state information, a is the action information, s′ is the updated state information, a′ is the state information corresponding to the maximum state action value obtained after the updated state information executes all the action information in the action space information, max a′ Q(s′,a′) is the maximum state-action value obtained after the updated state information executes all action information in the action space information, ∥s-s0∥ is the distance value between the initial state information and the state information, r is the reward value, α and β represent the preset learning rate and the preset weight parameter respectively, and γ is the preset discount factor.

[0176] It should be noted that the action information a is added to the state information s to obtain the updated state information.

[0177] It should be noted that |s-s0|| can be calculated using the Euclidean distance calculation formula, and the embodiment of the present invention does not make a specific limitation.

[0178] It should be noted that the above-mentioned multi-satellite collaborative observation update model is used to process the initial state information, state information, action information, state action table information and reward value to obtain the state update action value and the updated state information. It can also be implemented by using the update Q table operation in the Q-learning algorithm. Specifically, the embodiment of the present invention is not limited.

[0179] S3362, using the state update action value, update the state action table information to obtain updated state action table information.

[0180] It should be noted that the above update processing is to set the state update action value to the state action value of the action information corresponding to the state information in the state action table information.

[0181] It can be seen that implementing the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0182] As shown in Table 3, Table 3 is an analysis and comparison table of the embodiment of the present invention and the traditional multi-satellite collaborative observation method.

[0183] Table 3 Analysis and comparison of the embodiment of the present invention and the traditional multi-satellite collaborative observation method

[0184]

[0185]

[0186] It can be seen that, under the same score, the implementation of the embodiment of the present invention is significantly better than the traditional multi-satellite collaborative observation method in terms of time consumption. The implementation of the multi-satellite collaborative observation method described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing for satellite mission planning.

[0187] Embodiment 2

[0188] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of a multi-satellite collaborative observation device disclosed in an embodiment of the present invention. Figure 3 The multi-satellite collaborative observation device described is applied to a multi-satellite collaborative observation optimization system, such as a local server or a cloud server for multi-satellite collaborative observation, etc., which is not limited in the embodiments of the present invention. Figure 3 As shown, the multi-satellite collaborative observation device includes:

[0189] Acquisition module 201, used to acquire training data set, task information and time window information;

[0190] The first computing module 202 is used to perform fusion processing on the training data set, task information and time window information to obtain collaborative observation information to be processed;

[0191] The second calculation module 203 is used to analyze and calculate the task information, the time window information and the collaborative observation information to be processed to obtain the target collaborative observation information.

[0192] It can be seen that the implementation of the multi-satellite collaborative observation device described in the embodiment of the present invention is conducive to accelerating the decision-making speed of multi-satellite collaborative observation, thereby reducing the difficulty and time consumption of searching and optimizing satellite mission planning.

[0193] Embodiment 3

[0194] See also Figure 4 , Figure 4 is a schematic diagram of the structure of another multi-satellite cooperative observation device disclosed in an embodiment of the present invention. Figure 4 The multi-satellite collaborative observation device described is applied to a multi-satellite collaborative observation optimization system, such as a local server or a cloud server for multi-satellite collaborative observation, etc., which is not limited in the embodiments of the present invention. Figure 4 As shown, the multi-satellite collaborative observation device includes:

[0195] Processor 301;

[0196] A memory 302 coupled to the processor 301 and storing executable program codes;

[0197] The processor 301 calls the executable program code stored in the memory 302 to execute part or all of the steps of the multi-satellite collaborative observation method of the first embodiment.

[0198] Embodiment 4

[0199] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, which, when called, are used to execute part or all of the steps of the multi-satellite collaborative observation method of embodiment one.

[0200] Embodiment 5

[0201] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute some or all of the steps in the multi-satellite collaborative observation method described in Example 1.

[0202] The system embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative labor.

[0203] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0204] Finally, it should be noted that the multi-satellite collaborative observation method and device disclosed in the embodiments of the present invention disclose only the preferred embodiments of the present invention, which are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-satellite collaborative observation method, characterized in that: The method comprises: S1, obtain training data set, task information and time window information; S2, fusing the training data set, the task information and the time window information to obtain collaborative observation information to be processed; S3, analyzing and calculating the task information, the time window information and the collaborative observation information to be processed to obtain target collaborative observation information.

2. The multi-satellite collaborative observation method according to claim 1, characterized in that: The fusing the training data set, the task information and the time window information to obtain the collaborative observation information to be processed includes: S21, preprocessing the training data set to obtain a preprocessed training data set; S22, using the preprocessed training data set, training a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model; S23, using the multi-satellite collaborative observation optimization model, calculating and processing the task information and the time window information to obtain collaborative observation information to be processed.

3. The multi-satellite collaborative observation method according to claim 2, characterized in that: The method of using the preprocessed training data set to train a preset multi-satellite collaborative observation model to obtain a multi-satellite collaborative observation optimization model includes: S221, processing the preprocessed training data set to obtain a target training data set; S222, using the target training data set, training the multi-satellite collaborative observation model to obtain training result information and a multi-satellite collaborative observation training model; S223, using a loss function, calculating and processing the training result information to obtain a loss function value; S224, determining whether the loss function value is less than a preset loss function threshold, and obtaining a first determination result; When the first judgment result is no, determining that the multi-satellite collaborative observation training model is the multi-satellite collaborative observation model, and executing S222; When the first judgment result is yes, it is determined that the multi-satellite collaborative observation training model is a multi-satellite collaborative observation optimization model.

4. The multi-satellite collaborative observation method according to claim 3, characterized in that: The loss function is: 1≤i≤N; 1≤j≤M; Among them, Loss is the loss function value, P(x i,j ′) is the result information of the jth task feature of the i-th sample in the training result information, P(x i,j ) is the real result information corresponding to the jth task feature of the i-th sample, w j is the weight coefficient of the jth task feature, N is the number of samples, M is the number of task features in the samples, and α is the deviation coefficient.

5. The multi-satellite collaborative observation method according to claim 1, characterized in that: The analyzing and calculating the task information, the time window information and the collaborative observation information to be processed to obtain the target collaborative observation information includes: S31, determining that the collaborative observation information to be processed is initial state information; S32, fusing the task information and the time window information to obtain state space information and action space information; S33, processing the state space information, the action space information and the initial state information to obtain target collaborative observation information.

6. The multi-satellite collaborative observation method according to claim 5, characterized in that: The processing of the state space information, the action space information and the initial state information to obtain target collaborative observation information includes: S331, processing the state space information and the action space information to obtain state action table information; S332, determining that the initial state information is state information; S333, preset t=1; S334, calculating and processing the state information and the state action table information to obtain action information; S335, calculating and processing the action information and the state information to obtain a reward value; S336, processing the initial state information, the state information, the action information, the state-action table information and the reward value to obtain updated state information and updated state-action table information, and determining the updated state information as the state information, and determining the updated state-action table information as the state-action table information; S337, determining whether t is greater than a preset number of iterations, and obtaining a first determination result; When the first judgment result is no, increase the t by 1 and execute S334; When the first judgment result is yes, it is determined that the state action table information is the target collaborative observation information.

7. The multi-satellite collaborative observation method according to claim 6, characterized in that: The processing of the initial state information, the state information, the action information, the state action table information and the reward value to obtain updated state information and updated state action table information includes: S3361, using a multi-satellite collaborative observation update model, processing the initial state information, the state information, the action information, the state action table information and the reward value to obtain a state update action value and updated state information; The multi-satellite collaborative observation update model is: Wherein, Q′(s,a) is the state update action value, Q(s,a) is the state action value corresponding to the action information corresponding to the state information in the state action table information, s is the state information, s0 is the initial state information, a is the action information, s′ is the updated state information, a′ is the state information corresponding to the maximum state action value obtained after the updated state information executes all the action information in the action space information, max a′ Q(s′, a′) is the maximum state action value obtained after the updated state information executes all action information in the action space information, ||s-s0|| is the distance value between the initial state information and the state information, r is the reward value, α and β represent the preset learning rate and the preset weight parameter respectively, and γ is the preset discount factor; S3362: Use the state update action value to update the state action table information to obtain updated state action table information.

8. A multi-satellite collaborative observation device, characterized in that: The device comprises: The acquisition module is used to obtain the training data set, task information and time window information; A first computing module is used to perform a fusion process on the training data set, the task information and the time window information to obtain collaborative observation information to be processed; The second calculation module is used to analyze and calculate the task information, the time window information and the collaborative observation information to be processed to obtain target collaborative observation information.

9. A multi-satellite collaborative observation device, characterized in that: The device comprises: processor; a memory coupled to the processor and storing executable program code; The processor calls the executable program code stored in the memory to execute the multi-satellite collaborative observation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the multi-satellite collaborative observation method as described in any one of claims 1-7.

Images