Intelligent evolution method and device for remote sensing satellite application system
By constructing a system twin of the remote sensing satellite application system and an improved DQN algorithm, combined with dense reward design, the problems of difficult data acquisition and sparse rewards in the remote sensing satellite application system are solved, and the intelligent evolution of the remote sensing satellite application system and improvement of service quality are achieved.
Patent Information
- Application Number
- CN202511043910.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-28
- Publication Date
- 2025-09-12
- Estimated Expiration
- 2045-07-28
AI Technical Summary
Existing technologies make it difficult to effectively improve the service efficiency and quality of remote sensing satellite application systems, especially in complex systems where data acquisition difficulties and sparse rewards lead to slow learning convergence and low training efficiency.
By combining simulation deduction technology with deep reinforcement learning, a system twin of the remote sensing satellite application system is constructed. Intelligent evolution is carried out through the improved DQN algorithm, a dense reward mechanism is designed, and a dual deep neural network is used for state estimation and action selection to optimize the training process.
It has achieved the intelligent evolution of remote sensing satellite application systems, improved convergence efficiency, significantly accelerated the evolution research process, and enhanced the system's intelligent decision-making capabilities and service quality.
Smart Images

Figure CN120633463A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of simulation and deduction technology, and in particular to a remote sensing satellite application system intelligent evolution method and device. Background Art
[0002] In the current field of remote sensing satellite applications, as remote sensing technology expands across various industries, the demand for intensive use of remote sensing satellite resources, high-quality services, and efficient performance is increasing. To improve the service efficiency and quality of remote sensing satellite applications, artificial intelligence technology is applied to remote sensing satellites and application systems. Deep reinforcement learning algorithms are used to correlate the state space, action space, service quality, and satisfaction of remote sensing satellite application systems, thereby optimizing and improving strategies.
[0003] The DQN (Deep Q-Network) algorithm is a Q-Learning and reinforcement learning algorithm based on deep neural networks. It applies deep learning to reinforcement learning. By leveraging the expressive power of neural networks to replace the traditional Q-value function, it evaluates the advantages of adopting an action strategy under a given state and guides the system to select the optimal strategy. In the intelligent evolution of remote sensing satellite application systems, the satellite system's state, actions, and mission completion can be mapped to the DQN algorithm's state space and action space, respectively. Using mission completion and resource consumption metrics as reward values, this approach addresses the intelligent perception and decision-making challenges of remote sensing satellite application systems.
[0004] Unlike other actual systems, it is particularly difficult to obtain data such as the actual working status, working parameters, and service quality of remote sensing satellite application systems through actual system operation due to factors such as the complex composition, cross-domain deployment, diverse requirements, and long operating time of remote sensing satellite application systems. Therefore, modeling and simulation technology can be used to construct a system twin model of the remote sensing satellite application system. Through the operation of the twin, with the help of a deep reinforcement learning model, the interaction between the remote sensing satellite application system and the environment can be simulated. Iterative learning training is used to obtain the optimal strategy to realize the intelligent evolution of the remote sensing satellite system. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to provide a method and device for the intelligent evolution of a remote sensing satellite application system, which realizes the intelligent evolution of a remote sensing satellite application system by combining simulation and deduction technology with deep reinforcement learning artificial intelligence technology, thereby providing a technical method for improving the service quality and application benefits of the remote sensing satellite application system.
[0006] In order to solve the above technical problems, the first aspect of the embodiments of the present invention discloses a remote sensing satellite application system intelligent evolution method, the method comprising:
[0007] S1, modeling the remote sensing satellite application system to obtain the remote sensing satellite application system twin;
[0008] S2, collecting information on the remote sensing satellite application system twin to obtain twin status information; the twin status information includes satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object;
[0009] S3: Process the twin state information to obtain an intelligent evolution strategy.
[0010] As an optional implementation, in the first aspect of the embodiment of the present invention, modeling the remote sensing satellite application system to obtain a remote sensing satellite application system twin includes:
[0011] S11, modeling the entities and behaviors of the remote sensing satellite application system to obtain a behavior model;
[0012] S12, modeling resource allocation, task allocation, storage compression, detection and recognition, data query, data processing, and data analysis to obtain an abstract model;
[0013] S13, modeling the detection requirements and detection objects of the remote sensing satellite application system to obtain an object model;
[0014] S14, fusing the behavior model, the abstract model, and the object model to obtain a remote sensing satellite application system twin.
[0015] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the twin state information to obtain the intelligent evolution strategy includes:
[0016] S31, improve the DNQ model to obtain the optimized DNQ model;
[0017] S32, using the optimized DNQ model, processing the twin state information to obtain intelligent evolution parameter information;
[0018] S33: Process the intelligent evolution parameter information to obtain an intelligent evolution strategy.
[0019] As an optional implementation, in the first aspect of the embodiment of the present invention, the use of the optimized DNQ model to process the twin state information to obtain intelligent evolution parameter information includes:
[0020] S321, using the optimized DNQ model, processing the remote sensing satellite application system to obtain a state space, an action space, and a neuron output;
[0021] S322, processing the remote sensing satellite application system to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward, and an additional reward;
[0022] S323, processing the remote sensing satellite application system to obtain training hyperparameters; the training hyperparameters include a learning rate, a discount factor, a greedy growth factor, an experience replay pool capacity, and a batch size;
[0023] S234: Integrate the state space, action space, neuron output, reward value, and training hyperparameters to obtain intelligent evolution parameter information.
[0024] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the use of the optimized DNQ model to process the remote sensing satellite application system to obtain the state space, action space and neuron output includes:
[0025] S3211, extracting and encoding features of satellite resources, computing resources, storage resources, and importance levels of the current processing task for executing the current remote sensing satellite application service task to obtain a state space;
[0026] S3212, processing the remote sensing satellite application system to obtain an action space; the action space includes satellite resource adjustment, storage capacity adjustment, and computing resource adjustment;
[0027] S3213, processing the remote sensing satellite application system to obtain neuron output; the neuron output includes the number of observation areas, observation time for each area, and time for providing results.
[0028] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the remote sensing satellite application system is processed to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward, and an additional reward;
[0029] S3221, processes the actions of the remote sensing satellite application system and obtains task completion rewards and task completion timeliness rewards;
[0030] S3222: Process the satellite resource action to obtain a satellite resource reward; the satellite resource reward expression is:
[0031] re=-nsatellite*ke
[0032] If an action is to increase satellite resources, a negative reward value is given; if an action is to reduce satellite resources, a positive reward value is given. re is the reward value, nsatellite is the number of satellites added or reduced, with an increase being a positive value and a decrease being a negative value. ke is the reward coefficient, which is set according to the importance of the satellites added or reduced in the system, and * represents multiplication.
[0033] S3223, processing the storage resource action to obtain a storage resource reward;
[0034] S3224, processing the computing resource action to obtain a computing resource reward;
[0035] S3225, action fault information is processed to obtain system reliability rewards;
[0036] S3226: Integrate the satellite resource reward, the storage resource reward, the computing resource reward, and the system reliability reward to obtain an additional reward.
[0037] As an optional implementation, in the first aspect of the embodiment of the present invention, the processing of the actions of the remote sensing satellite application system to obtain the task completion reward and the task completion timeliness reward includes:
[0038] S32211, processing the actions of the remote sensing satellite application system to obtain a task completion reward; the expression of the task completion reward rtc is:
[0039]
[0040] Among them, Aa i It is the actual observation area of the remote sensing satellite application system based on the research results of intelligent evolution for the i-th area requiring detection, which comes from the actual mission requirements; Ar i is the area of the i-th area to be detected; ta i is the observation time required for the i-th area to be detected; tr i is the actual observation time of the i-th area to be detected, w i is the preset weight coefficient, ktc is the reward coefficient, which is set by the experiment, n is the number of areas to be detected, and * indicates multiplication;
[0041] S32212: Process the actions of the remote sensing satellite application system to obtain a timeliness reward for task completion; the timeliness reward for task completion is expressed as:
[0042]
[0043] where kt i is the importance coefficient of the i-th task, treqi is the completion time of the i-th task, actual i is the required completion time of the i-th task.
[0044] A second aspect of an embodiment of the present invention discloses an intelligent evolution device for a remote sensing satellite application system, the device comprising:
[0045] System modeling module, used to model the remote sensing satellite application system and obtain the remote sensing satellite application system twin;
[0046] A status information acquisition module is used to collect information about the remote sensing satellite application system twin to obtain twin status information; the twin status information includes satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object;
[0047] The intelligent evolution strategy generation module is used to process the twin state information to obtain an intelligent evolution strategy.
[0048] As an optional implementation, in the second aspect of the embodiment of the present invention, modeling the remote sensing satellite application system to obtain a remote sensing satellite application system twin includes:
[0049] S11, modeling the entities and behaviors of the remote sensing satellite application system to obtain a behavior model;
[0050] S12, modeling resource allocation, task allocation, storage compression, detection and recognition, data query, data processing, and data analysis to obtain an abstract model;
[0051] S13, modeling the detection requirements and detection objects of the remote sensing satellite application system to obtain an object model;
[0052] S14, fusing the behavior model, the abstract model, and the object model to obtain a remote sensing satellite application system twin.
[0053] As an optional implementation, in the second aspect of the embodiment of the present invention, the processing of the twin state information to obtain an intelligent evolution strategy includes:
[0054] S31, improve the DNQ model to obtain the optimized DNQ model;
[0055] S32, using the optimized DNQ model, processing the twin state information to obtain intelligent evolution parameter information;
[0056] S33: Process the intelligent evolution parameter information to obtain an intelligent evolution strategy.
[0057] As an optional implementation, in the second aspect of the embodiment of the present invention, the use of the optimized DNQ model to process the twin state information to obtain intelligent evolution parameter information includes:
[0058] S321, using the optimized DNQ model, processing the remote sensing satellite application system to obtain a state space, an action space, and a neuron output;
[0059] S322, processing the remote sensing satellite application system to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward, and an additional reward;
[0060] S323, processing the remote sensing satellite application system to obtain training hyperparameters; the training hyperparameters include a learning rate, a discount factor, a greedy growth factor, an experience replay pool capacity, and a batch size;
[0061] S234: Integrate the state space, action space, neuron output, reward value, and training hyperparameters to obtain intelligent evolution parameter information.
[0062] As an optional implementation, in the second aspect of the embodiment of the present invention, the use of the optimized DNQ model to process the remote sensing satellite application system to obtain the state space, action space and neuron output includes:
[0063] S3211, extracting and encoding features of satellite resources, computing resources, storage resources, and importance levels of the current processing task for executing the current remote sensing satellite application service task to obtain a state space;
[0064] S3212, processing the remote sensing satellite application system to obtain an action space; the action space includes satellite resource adjustment, storage capacity adjustment, and computing resource adjustment;
[0065] S3213, processing the remote sensing satellite application system to obtain neuron output; the neuron output includes the number of observation areas, observation time for each area, and time for providing results.
[0066] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the remote sensing satellite application system is processed to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward, and an additional reward;
[0067] S3221, processes the actions of the remote sensing satellite application system and obtains task completion rewards and task completion timeliness rewards;
[0068] S3222: Process the satellite resource action to obtain a satellite resource reward; the satellite resource reward expression is:
[0069] re=-nsatellite*ke
[0070] If an action is to increase satellite resources, a negative reward value is given; if an action is to reduce satellite resources, a positive reward value is given. re is the reward value, nsatellite is the number of satellites added or reduced, with an increase being a positive value and a decrease being a negative value. ke is the reward coefficient, which is set according to the importance of the satellites added or reduced in the system, and * represents multiplication.
[0071] S3223, processing the storage resource action to obtain a storage resource reward;
[0072] S3224, processing the computing resource action to obtain a computing resource reward;
[0073] S3225, action fault information is processed to obtain system reliability rewards;
[0074] S3226: Integrate the satellite resource reward, the storage resource reward, the computing resource reward, and the system reliability reward to obtain an additional reward.
[0075] As an optional implementation, in the second aspect of the embodiment of the present invention, the processing of the actions of the remote sensing satellite application system to obtain the task completion reward and the task completion timeliness reward includes:
[0076] S32211, processing the actions of the remote sensing satellite application system to obtain a task completion reward; the expression of the task completion reward rtc is:
[0077]
[0078] Among them, Aa i It is the actual observation area of the remote sensing satellite application system based on the research results of intelligent evolution for the i-th area requiring detection, which comes from the actual mission requirements; Ar i is the area of the i-th area to be detected; ta i is the observation time required for the i-th area to be detected; tr i is the actual observation time of the i-th area to be detected, w i is the preset weight coefficient, ktc is the reward coefficient, which is set by the experiment, n is the number of areas to be detected, and * indicates multiplication;
[0079] S32212: Process the actions of the remote sensing satellite application system to obtain a timeliness reward for task completion; the timeliness reward for task completion is expressed as:
[0080]
[0081] where kt i is the importance coefficient of the i-th task, treq i is the completion time of the i-th task, actual i is the required completion time of the i-th task.
[0082] A third aspect of the present invention discloses another remote sensing satellite application system intelligent evolution device, the device comprising:
[0083] a memory storing executable program code;
[0084] a processor coupled to the memory;
[0085] The processor calls the executable program code stored in the memory to execute part or all of the steps in the remote sensing satellite application system intelligent evolution method disclosed in the first aspect of the embodiment of the present invention.
[0086] The fourth aspect of the present invention discloses a computer-storable medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the remote sensing satellite application system intelligent evolution method disclosed in the first aspect of the embodiment of the present invention.
[0087] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0088] 1. This paper proposes a framework for the intelligent evolution of remote sensing satellite application systems that integrates simulation and deduction technology with deep reinforcement learning technology. By constructing a scenario-based twin deduction operation of the remote sensing satellite application system and integrating it with the deep reinforcement learning DQN algorithm, it overcomes the problem of difficulty in obtaining and reusing actual system operation data, and realizes the intelligent evolution of the remote sensing satellite application system.
[0089] 2. A dense reward method using the DQN algorithm for the intelligent evolution of remote sensing satellite application systems is proposed. This method overcomes the problems of traditional deep reinforcement learning caused by sparse rewards and multi-dimensional complex factors, resulting in slow convergence, low training efficiency, high training cost and even difficulty in convergence due to reward and punishment delays. It effectively improves the convergence efficiency of the intelligent evolution of remote sensing satellite application systems and can significantly accelerate the evolution research process.
[0090] 3. A method for intelligent evolution of remote sensing satellite application systems based on the dense reward design DNQ algorithm is proposed. Combined with the actual working conditions of the remote sensing satellite application system, dense reward values, deep reinforcement learning hyperparameters, and experimental parameter design are provided. The dual neural network is used to update the Q value, the algorithm steps are clarified, and the specific method steps for realizing the intelligent evolution of the remote sensing satellite application system are clarified. BRIEF DESCRIPTION OF THE DRAWINGS
[0091] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0092] Figure 1 This is a flow chart of an intelligent evolution method for a remote sensing satellite application system disclosed in an embodiment of the present invention;
[0093] Figure 2 This is a flow chart of another method for intelligent evolution of a remote sensing satellite application system disclosed in an embodiment of the present invention;
[0094] Figure 3 This is a schematic diagram of the structure of an intelligent evolution device for a remote sensing satellite application system disclosed in an embodiment of the present invention;
[0095] Figure 4 It is a structural diagram of another intelligent evolution device for a remote sensing satellite application system disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0096] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0097] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or device.
[0098] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0099] The present invention discloses a method and device for the intelligent evolution of a remote sensing satellite application system, the method comprising: modeling a remote sensing satellite application system to obtain a remote sensing satellite application system twin; collecting information on the remote sensing satellite application system twin to obtain twin status information; the twin status information includes satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object; processing the twin status information to obtain an intelligent evolution strategy. The present invention overcomes the problem of difficulty in obtaining and reusing actual system operation data by constructing a scenario-based remote sensing satellite application system twin deduction operation and integrating it with the deep reinforcement learning DQN algorithm, thereby realizing the intelligent evolution of the remote sensing satellite application system. The following are detailed descriptions.
[0100] Example 1
[0101] See also Figure 1 , Figure 1 This is a flow chart of an intelligent evolution method for a remote sensing satellite application system disclosed in an embodiment of the present invention. Figure 1 The described remote sensing satellite application system intelligent evolution method is applied to the field of simulation and deduction technology, and the embodiment of the present invention does not limit it. Figure 1 As shown, the remote sensing satellite application system intelligent evolution method may include the following operations:
[0102] S1, modeling the remote sensing satellite application system to obtain the remote sensing satellite application system twin;
[0103] S2, collecting information on the remote sensing satellite application system twin to obtain twin status information; the twin status information includes satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object;
[0104] S3: Process the twin state information to obtain an intelligent evolution strategy.
[0105] Optionally, the modeling of the remote sensing satellite application system to obtain a remote sensing satellite application system twin includes:
[0106] S11, modeling the entities and behaviors of the remote sensing satellite application system to obtain a behavior model;
[0107] S12, modeling resource allocation, task allocation, storage compression, detection and recognition, data query, data processing, and data analysis to obtain an abstract model;
[0108] S13, modeling the detection requirements and detection objects of the remote sensing satellite application system to obtain an object model;
[0109] S14, fusing the behavior model, the abstract model, and the object model to obtain a remote sensing satellite application system twin.
[0110] Optionally, the processing of the twin state information to obtain an intelligent evolution strategy includes:
[0111] S31, improve the DNQ model to obtain the optimized DNQ model;
[0112] The Dual DDNQ (Double DNQ) method overcomes the overestimation of Q values in the DNQ method. Two deep neural networks are introduced: an online network for selecting and estimating the current Q value, and a target network for estimating the target Q value based on the next state and action. This dual neural network mechanism avoids the overestimation problem caused by the traditional DQN algorithm using the same network for both action selection and target estimation. This enables remote sensing satellite application systems to more accurately learn optimal strategies during intelligent evolution, improving the efficiency of evolutionary computation.
[0113] S32, using the optimized DNQ model, processing the twin state information to obtain intelligent evolution parameter information;
[0114] S33: Process the intelligent evolution parameter information to obtain an intelligent evolution strategy, including:
[0115] Set the maximum number of training tasks to Nepi. In one task, set the decision interval of the remote sensing satellite application system to tinterval, and the maximum number of decision steps to Nstep. For each decision step, collect and record the state, action, reward, and next state, and provide them to the DQN network for training. The method flow is as follows:
[0116] (1) Initialization settings
[0117] The data that needs to be initially set include: experience replay pool memory, maximum number of training tasks, number of training data collection rounds, current Q value network parameters and target Q value network parameters, and hyperparameters of the DQN method.
[0118] (2) Single-game loop deduction
[0119] A system twin of the remote sensing satellite application system is constructed, including mission scenarios, mission requirements, satellite resources, storage resources, computing resources, behavioral rules, and the entities being detected. The system twin of the remote sensing satellite application system is simulated and run. The intelligent evolution strategy generation module obtains the operating status information of the system twin of the remote sensing satellite application system and selects the output action to be executed under a greedy strategy. The specific method is to compare a randomly generated number with the greedy growth factor. If the random number is greater than the greedy growth factor, the action with the largest Q value is selected as the output; otherwise, an action is randomly selected in the action space.
[0120] Obtain the next state information and reward information. The reward information is the reward value of the dense reward design, which is comprehensively generated by the task completion reward, task completion timeliness reward, satellite resource reward, storage resource reward, computing resource reward, and system reliability reward.
[0121] (3) Collecting Experience Points
[0122] If the set number of experience collection games is reached, a set of experience values (sj, aj, rj, sj+1) will be stored in the replay experience pool. If the number exceeds the experience pool cache, the original experience value will be overwritten.
[0123] (4) Update DQN network parameters
[0124] Extract batch_size number of experience values from the experience pool, use the current Q network to calculate the estimated value Q(sj,aj), use the loss function {[rj+γ*Qtarget(sj+1,aj+1)]-Qcurrent(sj,aj)}2 to perform gradient descent to update the current network parameter θ, and use θ to update the target network parameter θ' when a specific number of intervals are reached.
[0125] When one mission is completed, continue training for the next mission.
[0126] (5) Storage Intelligent Evolution Strategy Generation Model
[0127] During any remote sensing satellite application system twin deduction and operation mission, if the gradient descent reaches the convergence target, the intelligent evolution strategy generation model is saved. When the maximum number of training tasks Nepi is completed, the intelligent evolution strategy generation model is also saved.
[0128] Optionally, the optimized DNQ model is used to process the twin state information to obtain intelligent evolution parameter information, including:
[0129] S321, using the optimized DNQ model, processing the remote sensing satellite application system to obtain a state space, an action space, and a neuron output;
[0130] S322, processing the remote sensing satellite application system to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward, and an additional reward;
[0131] S323, processing the remote sensing satellite application system to obtain training hyperparameters; the training hyperparameters include a learning rate, a discount factor, a greedy growth factor, an experience replay pool capacity, and a batch size;
[0132] S234: Integrate the state space, action space, neuron output, reward value, and training hyperparameters to obtain intelligent evolution parameter information.
[0133] Optionally, the use of the optimized DNQ model to process a remote sensing satellite application system to obtain a state space, an action space, and a neuron output includes:
[0134] S3211, extracting and encoding features of satellite resources, computing resources, storage resources, and importance levels of the current processing task for executing the current remote sensing satellite application service task to obtain a state space;
[0135] S3212, processing the remote sensing satellite application system to obtain an action space; the action space includes satellite resource adjustment, storage capacity adjustment, and computing resource adjustment;
[0136] S3213, processing the remote sensing satellite application system to obtain neuron output; the neuron output includes the number of observation areas, observation time for each area, and time for providing results.
[0137] Optionally, the remote sensing satellite application system is processed to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward, and an additional reward;
[0138] S3221, processes the actions of the remote sensing satellite application system and obtains task completion rewards and task completion timeliness rewards;
[0139] S3222: Process the satellite resource action to obtain a satellite resource reward; the satellite resource reward expression is:
[0140] re=-nsatellite*ke
[0141] If an action is to increase satellite resources, a negative reward value is given; if an action is to reduce satellite resources, a positive reward value is given. re is the reward value, nsatellite is the number of satellites added or reduced, with an increase being a positive value and a decrease being a negative value. ke is the reward coefficient, which is set according to the importance of the satellites added or reduced in the system, and * represents multiplication.
[0142] S3223, processing the storage resource action to obtain a storage resource reward; the storage resource reward rs is expressed as:
[0143] rs=-nstore*ks
[0144] During the mission, if an action increases storage resources, a negative reward value is given; if an action reduces satellite storage resources, a positive reward value is given. nstore is the storage resource adjustment amount, with an increase being a positive value and a decrease being a negative value. ks is the reward coefficient, which is set according to the importance of the increased or decreased storage resources in the system.
[0145] S3224, processing the computing resource action to obtain a computing resource reward; the computing resource reward rc is expressed as:
[0146] rc=-ncalculate*kc,
[0147] During the task, if an action increases computing resources, a negative reward value is given; if an action reduces computing resources, a positive reward value is given. ncalculate is the storage resource adjustment amount, with an increase being a positive value and a decrease being a negative value. kc is the reward coefficient, which is set according to the importance of the increased or decreased computing resources in the system.
[0148] S3225, the action fault information is processed to obtain a system reliability reward; the system reliability reward expression is:
[0149] rs-stable=-kstable
[0150] During the mission, if the remote sensing satellite application system fails after a certain action, such as data transmission interruption, abnormal satellite attitude, etc., a corresponding penalty rs-stable = -kstable will be given according to the severity of the failure kstable, and the reward will be a negative value.
[0151] S3226: Integrate the satellite resource reward, the storage resource reward, the computing resource reward, and the system reliability reward to obtain an additional reward.
[0152] Optionally, the processing of the actions of the remote sensing satellite application system to obtain a task completion reward and a task completion timeliness reward includes:
[0153] S32211, processing the actions of the remote sensing satellite application system to obtain a task completion reward; the expression of the task completion reward rtc is:
[0154]
[0155] Among them, Aai It is the actual observation area of the remote sensing satellite application system based on the research results of intelligent evolution for the i-th area requiring detection, which comes from the actual mission requirements; Ar i is the area of the i-th area to be detected; ta i is the observation time required for the i-th area to be detected; tr i is the actual observation time of the i-th area to be detected, w i is the preset weight coefficient, ktc is the reward coefficient, which is set by the experiment, n is the number of areas to be detected, and * indicates multiplication;
[0156] S32212: Process the actions of the remote sensing satellite application system to obtain a timeliness reward for task completion; the timeliness reward for task completion is expressed as:
[0157]
[0158] where kt i is the importance coefficient of the i-th task, treq i is the completion time of the i-th task, actual i is the required completion time of the i-th task.
[0159] Example 2
[0160] This embodiment provides a method for intelligent evolution of a remote sensing satellite application system, which integrates modeling and simulation technology and deep reinforcement learning technology. First, the system twin of the remote sensing satellite application system is constructed using modeling and simulation technology. The twin interacts with the intelligent evolution strategy generation calculation module to provide state space input for the intelligent evolution strategy module, calculate decision actions, guide the operation of the twin, and obtain the service efficiency of the remote sensing satellite application system, thereby solving the problem of difficulty in setting actual experience values when using the traditional DNQ algorithm. Second, by customizing the twin operation process data, the state space, action space and reward value in the training process of obtaining the intelligent evolution strategy of the remote sensing satellite application system using the DNQ algorithm are reasonably designed in combination with the feasibility of remote sensing satellite application system twin data collection, providing a basis for evolution strategy generation. Third, a dense reward design method is adopted. During the operation of the remote sensing satellite application system twin, rewards are designed for key critical actions, thereby changing the problem of sparse rewards in the traditional DNQ learning algorithm and accelerating the efficiency of the intelligent evolution of the remote sensing satellite application system.
[0161] This paper proposes a learning framework for the intelligent evolution of remote sensing satellite application systems that integrates simulation and deduction techniques with deep reinforcement learning. It also designs parameters for the intelligent evolution of remote sensing satellite application systems based on the dense reward-based DNQ algorithm and clarifies the specific method and steps for implementing this approach. This paper can provide technical support for the efficient implementation of intelligent evolution in remote sensing satellite application systems.
[0162] 1. Intelligent Evolutionary Learning Framework for Remote Sensing Satellite Application Systems Integrating Simulation and Derivative Learning Technology
[0163] Combining the working scenarios of remote sensing satellite application systems, modeling simulation and digital twin technologies are used to construct an intelligent evolutionary learning framework for remote sensing satellite application systems.
[0164] (1) Construction of twin systems of remote sensing satellite application systems
[0165] Model the satellite equipment, sensor payload equipment, ground stations, communication equipment and other entities of the remote sensing satellite application system, as well as the corresponding orbital maneuvers, information communication, ground control, data collection, information distribution and other behaviors; perform abstract modeling for resource allocation, task allocation, storage compression, detection and identification, data query, data processing, data analysis and other links; model the detection requirements and detection objects of the remote sensing satellite application system; integrate and fuse the above models to construct a system twin of the remote sensing satellite application system.
[0166] (2) Twin operation status information collection
[0167] Twin operation status acquisition is responsible for collecting key information on the dynamic operation of the system twin of the remote sensing satellite application system in operation, including satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object.
[0168] (3) Intelligent Evolution Strategy Generation
[0169] Intelligent evolution strategy generation is responsible for calculating and generating specific execution actions based on the state information of the remote sensing satellite application system twin and the established action selection strategy. Intelligent evolution strategy generation can cyclically output the next action based on the interval state specified by the remote sensing satellite application system twin. It can also output the entire action sequence to guide the actual operation of the remote sensing satellite application system.
[0170] The intelligent evolution learning framework of remote sensing satellite application system based on simulation deduction technology is as follows: Figure 2 shown.
[0171] 2. Intelligent Evolution Parameter Design of Remote Sensing Satellite Application System Based on Dense Reward Design DNQ Algorithm
[0172] This paper employs a dual DDNQ (Double DNQ) approach in intelligent evolution strategy calculations to overcome the overestimation of Q values often found in DNQ. By introducing two deep neural networks—an online network for selecting and estimating the current Q value, and a target network for estimating the target Q value based on the next state and action—this dual neural network mechanism avoids the overestimation problem inherent in traditional DQN algorithms, which use the same network for both action selection and target estimation. This enables remote sensing satellite application systems to more accurately learn optimal strategies during intelligent evolution, improving the efficiency of evolutionary computations.
[0173] During the evolutionary learning process, an intelligent evolutionary strategy generates an action a based on the current system state s, selects it through the online network, and executes it. The remote sensing satellite application system twin responds to the action, returning a new state s′ and a reward r. The reward setting is closely tied to the system's optimization objectives. If the corresponding strategy action reduces resource usage, improves resource utilization, enhances task completion quality, or satisfies specific task requirements, a higher reward is given. Conversely, if the action results in increased resources, resource waste, task delays, or errors, a lower reward is given. The online network parameters are updated based on the reward signal and the new state. This process is repeated continuously, gradually learning the optimal strategy that maximizes the long-term cumulative reward.
[0174] (1) State space and action space design
[0175] Aiming at the intelligent evolution of remote sensing satellite application systems, the state space, action space, and neuron output of the DNQ algorithm based on dense reward design are designed as follows:
[0176] State space: The satellite resources, computing resources, and storage resources required to execute the current remote sensing satellite application service mission, as well as the importance of the current processing task, are extracted and encoded into a vector containing multiple eigenvalues as input. If the dimension of the final encoded feature vector is n, then the number of neurons in the input layer is n.
[0177] The design action space is: satellite resource adjustment, storage capacity adjustment, and computing resource adjustment.
[0178] Number of output neurons: number of observation areas, observation time for each area, and time for providing results.
[0179] (2) Reward value design
[0180] During the intelligent evolution of remote sensing satellite application systems, if rewards are directly based on task completion and timeliness after a mission is completed, rewards and penalties will arrive late due to reward sparseness and the influence of multidimensional and complex factors. This makes it difficult to determine the correspondence between actions and the impact of the final reward, resulting in slow convergence, low training efficiency, high training costs, and even difficulty in convergence. This paper adopts a dense reward value approach, setting rewards for each action of the remote sensing satellite application system. In addition to the final rewards for task completion and task completion quality, additional rewards are designed, including satellite resource rewards, storage resource rewards, computing resource rewards, and system reliability rewards.
[0181] Satellite Resource Reward: During a mission, if an action increases satellite resources, a negative reward is given; if an action decreases satellite resources, a positive reward is given. The reward value is re = -nsatellite * ke, where nsatellite is the number of satellites added or removed, with increases representing positive values and decreases representing negative values. Ke is the reward coefficient, set based on the importance of the satellite being added or removed in the system.
[0182] Storage Resource Reward: During a mission, if an action increases storage resources, a negative reward is given; if an action reduces satellite resources, a positive reward is given. The storage resource adjustment amount is nstore, and the reward rs = -nstore * ks, where nstore is the amount of storage resources increased or decreased, with increases being positive and decreases being negative. ks is the reward coefficient, set based on the importance of the storage resource being increased or decreased in the system.
[0183] Computing Resource Reward: During a mission, if an action increases computing resources, a negative reward is given; if an action reduces satellite resources, a positive reward is given. The storage resource adjustment amount is ncalculate, and the reward rc = -ncalculate * kc, where ncalculate is the amount of computing resources increased or decreased, with increases being positive and decreases being negative. kc is the reward coefficient, set based on the importance of the computing resources added or decreased in the system.
[0184] System reliability reward: During the mission, if the remote sensing satellite application system fails after a certain action, such as data transmission interruption, abnormal satellite attitude, etc., a corresponding penalty kstable will be given according to the severity of the failure rs-stable = -kstable, and the reward will be a negative value.
[0185] Mission Completion Reward: After a remote sensing satellite application system mission is completed, a mission completion reward will be given according to the mission completion percentage. The reward value is Among them, Aa_i is the actual observation area of the remote sensing satellite application system based on the research results of intelligent evolution for the i-th area to be detected, which comes from the actual mission requirements; Ar_i is the area of the i-th area to be detected; ta_i is the required observation time of the i-th area to be detected; tr_i is the actual observation time of the i-th area to be detected.
[0186] Timely reward for task completion: If the task result is provided later than the required time, the reward is negative; if the result is provided earlier than the required time, the reward is positive. Where kt is the task importance coefficient, treq is the time when the task is required to be completed, and tactual is the time when the task is required to be completed.
[0187] Combining all the above reward designs, the total reward generated by the intelligent evolution strategy designed according to the dense reward design method is:
[0188] R=re+rs+rc+rs-stable+rtc+rt
[0189] The intelligent evolution strategy module will continuously adjust the strategy based on the total reward signal to achieve intelligent evolution of the system.
[0190] (3) Training hyperparameter design
[0191] Learning rate: During training, a learning rate decay strategy can be used. The initial learning rate can be set to a small value, and then gradually reduced as the number of training steps increases to balance the convergence speed and stability of the algorithm. After every N steps of training, the learning rate is multiplied by the decay factor kr, that is, the learning rate α = kr * α. This allows the network to quickly explore different strategies in the early stages of training, and later adjust the parameters more finely to approach the optimal solution.
[0192] Discount factor: The discount factor γ is used to measure the importance of future rewards. A larger discount factor means the agent focuses more on long-term rewards and tends to choose actions that will bring long-term benefits. A smaller discount factor makes the agent focus more on short-term rewards. In remote sensing satellite application systems, due to the complexity and long-term nature of the mission, a discount factor close to 1 is chosen to tend to meet the final mission objectives and make decisions that are conducive to long-term stable operation and performance optimization of the system.
[0193] Greedy growth factor: The action with the highest current Q value is selected with a probability of 1-ε, and other actions are randomly selected with a probability of ε. In this method, setting a small ε allows the intelligent evolution strategy to use the best known action in most cases, while also allowing a certain amount of opportunity to explore new actions and discover better strategies.
[0194] Experience Replay Pool Capacity: The experience replay pool is used to store experience samples (sj, aj, rj, sj+1) generated by the agent's interaction with the environment, where s is the current state, a is the action taken, r is the reward received, and sj+1 is the next state. Setting the experience replay pool capacity M to a large value ensures the network can learn from a rich set of experiences during training and avoids overfitting caused by high correlation between consecutive samples. When the experience replay pool is full, new experience samples will overwrite the oldest stored samples.
[0195] Batch size: During each training, a batch of samples is randomly drawn from the experience replay pool to update the network parameters. The batch size batch_size can ensure that enough information is included in one update to stabilize the gradient, while not causing excessive consumption of computing resources and slowing down the training speed due to too large a batch.
[0196] Relying on the system twin operation of the remote sensing satellite application system, the intelligent evolution strategy generation module gradually realizes intelligent evolution of the entire remote sensing satellite application system through the twin operation status changes and strategy calculation training, which can better adapt to various complex and changeable task requirements and environmental conditions, and the overall performance is optimized.
[0197] 3. Intelligent Evolution Method for Remote Sensing Satellite Application System Based on Dense Reward Design DNQ Algorithm
[0198] The method proposed in this paper is based on the dense reward design of the DNQ algorithm and combines it with the system twin operation deduction of the remote sensing satellite application system. The maximum number of training tasks is Nepi. In one task, the decision interval of the remote sensing satellite application system is set to tinterval, and the maximum number of decision steps is Nstep. For each decision step, the state, action, reward, and next state are collected and recorded, and provided to the DQN network for training. The method process is as follows:
[0199] (1) Initialization settings
[0200] The data that needs to be initially set include: experience replay pool memory, maximum number of training tasks, number of training data collection rounds, current Q value network parameters and target Q value network parameters, and hyperparameters of the DQN method.
[0201] (2) Single-game loop deduction
[0202] A system twin of the remote sensing satellite application system is constructed, including mission scenarios, mission requirements, satellite resources, storage resources, computing resources, behavioral rules, and the entities being detected. The system twin of the remote sensing satellite application system is simulated and run. The intelligent evolution strategy generation module obtains the operating status information of the system twin of the remote sensing satellite application system and selects the output action to be executed under a greedy strategy. The specific method is to compare a randomly generated number with the greedy growth factor. If the random number is greater than the greedy growth factor, the action with the largest Q value is selected as the output; otherwise, an action is randomly selected in the action space.
[0203] Obtain the next state information and reward information. The reward information is the reward value of the dense reward design, which is comprehensively generated by the task completion reward, task completion timeliness reward, satellite resource reward, storage resource reward, computing resource reward, and system reliability reward.
[0204] (3) Collecting Experience Points
[0205] If the set number of experience collection games is reached, a set of experience values (sj, aj, rj, sj+1) will be stored in the replay experience pool. If the number exceeds the experience pool cache, the original experience value will be overwritten.
[0206] (4) Update DQN network parameters
[0207] Extract batch_size number of experience values from the experience pool, use the current Q network to calculate the estimated value Q(sj,aj), use the loss function {[rj+γ*Qtarget(sj+1,aj+1)]-Qcurrent(sj,aj)}2 to perform gradient descent to update the current network parameter θ, and use θ to update the target network parameter θ' when a specific number of intervals are reached.
[0208] When one mission is completed, continue training for the next mission.
[0209] (5) Storage Intelligent Evolution Strategy Generation Model
[0210] During any remote sensing satellite application system twin deduction and operation mission, if the gradient descent reaches the convergence target, the intelligent evolution strategy generation model is saved. When the maximum number of training tasks Nepi is completed, the intelligent evolution strategy generation model is also saved.
[0211] Example 3
[0212] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an intelligent evolution device for a remote sensing satellite application system disclosed in an embodiment of the present invention. Figure 3 The remote sensing satellite application system intelligent evolution device described is applied to the field of simulation and deduction technology, and the embodiment of the present invention does not limit it. Figure 3As shown, the remote sensing satellite application system intelligent evolution device may include the following operations:
[0213] S301, a system modeling module, is used to model the remote sensing satellite application system and obtain a remote sensing satellite application system twin;
[0214] S302, a status information acquisition module is used to collect information about the remote sensing satellite application system twin to obtain twin status information; the twin status information includes satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object;
[0215] S303, an intelligent evolution strategy generation module, is used to process the twin state information to obtain an intelligent evolution strategy.
[0216] Example 4
[0217] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of another intelligent evolution device for remote sensing satellite application system disclosed in an embodiment of the present invention. Figure 4 The remote sensing satellite application system intelligent evolution device described is applied to the field of simulation and deduction technology, and the embodiment of the present invention does not limit it. Figure 4 As shown, the remote sensing satellite application system intelligent evolution device may include the following operations:
[0218] A memory 401 storing executable program code;
[0219] a processor 402 coupled to the memory 401;
[0220] The processor 402 calls the executable program code stored in the memory 401 to execute the steps of the remote sensing satellite application system intelligent evolution method described in the first and second embodiments.
[0221] Example 5
[0222] An embodiment of the present invention discloses a computer-readable storage medium that stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the remote sensing satellite application system intelligent evolution method described in Embodiment 1 and Embodiment 2.
[0223] The device embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.
[0224] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.
[0225] Finally, it should be noted that the intelligent evolution method and device for a remote sensing satellite application system disclosed in the embodiments of the present invention only disclose a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that it is still possible to modify the technical solutions recorded in the aforementioned embodiments, or to replace some of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A remote sensing satellite application system intelligent evolution method, characterized in that: The method comprises: S1, modeling the remote sensing satellite application system to obtain the remote sensing satellite application system twin; S2, collecting information on the remote sensing satellite application system twin to obtain twin status information; the twin status information includes satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object; S3: Process the twin state information to obtain an intelligent evolution strategy.
2. The remote sensing satellite application system intelligent evolution method according to claim 1, characterized in that: The remote sensing satellite application system is modeled to obtain a remote sensing satellite application system twin, including: S11, modeling the entities and behaviors of the remote sensing satellite application system to obtain a behavior model; S12, modeling resource allocation, task allocation, storage compression, detection and recognition, data query, data processing, and data analysis to obtain an abstract model; S13, modeling the detection requirements and detection objects of the remote sensing satellite application system to obtain an object model; S14, fusing the behavior model, the abstract model, and the object model to obtain a remote sensing satellite application system twin.
3. The remote sensing satellite application system intelligent evolution method according to claim 1, characterized in that: The processing of the twin state information to obtain an intelligent evolution strategy includes: S31, improve the DNQ model to obtain the optimized DNQ model; S32, using the optimized DNQ model, processing the twin state information to obtain intelligent evolution parameter information; S33: Process the intelligent evolution parameter information to obtain an intelligent evolution strategy.
4. The remote sensing satellite application system intelligent evolution method according to claim 3, characterized in that: The optimized DNQ model is used to process the twin state information to obtain intelligent evolution parameter information, and the intelligent evolution parameter information is obtained, including: S321, using the optimized DNQ model, processing the remote sensing satellite application system to obtain a state space, an action space, and a neuron output; S322, processing the remote sensing satellite application system to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward, and an additional reward; S323, processing the remote sensing satellite application system to obtain training hyperparameters; the training hyperparameters include a learning rate, a discount factor, a greedy growth factor, an experience replay pool capacity, and a batch size; S234: Integrate the state space, action space, neuron output, reward value, and training hyperparameters to obtain intelligent evolution parameter information.
5. The remote sensing satellite application system intelligent evolution method according to claim 4, characterized in that: The optimized DNQ model is used to process the remote sensing satellite application system to obtain the state space, action space and neuron output, including: S3211, extracting and encoding features of satellite resources, computing resources, storage resources, and importance levels of the current processing task for executing the current remote sensing satellite application service task to obtain a state space; S3212, processing the remote sensing satellite application system to obtain an action space; the action space includes satellite resource adjustment, storage capacity adjustment, and computing resource adjustment; S3213, processing the remote sensing satellite application system to obtain neuron output; the neuron output includes the number of observation areas, observation time for each area, and time for providing results.
6. The remote sensing satellite application system intelligent evolution method according to claim 4, characterized in that: The remote sensing satellite application system is processed to obtain a reward value; the reward value includes a task completion reward, a task completion timeliness reward and an additional reward; S3221, processes the actions of the remote sensing satellite application system and obtains task completion rewards and task completion timeliness rewards; S3222: Process the satellite resource action to obtain a satellite resource reward; the satellite resource reward expression is: re=-nsatellite*ke If an action is to increase satellite resources, a negative reward value is given; if an action is to reduce satellite resources, a positive reward value is given. re is the reward value, nsatellite is the number of satellites added or reduced, with an increase being a positive value and a decrease being a negative value. ke is the reward coefficient, which is set according to the importance of the satellites added or reduced in the system, and * represents multiplication. S3223, processing the storage resource action to obtain a storage resource reward; r s =-n store *ks Among them, r s For storage resource rewards, n store The amount of storage resources increased or decreased, with an increase being a positive value and a decrease being a negative value; ks is the reward coefficient, which is set according to the importance of the increased or decreased storage resources in the system; S3224, processing the computing resource action to obtain a computing resource reward; rc=-n calculate *kc Among them, rc is the computing resource reward, n calculate is the amount of computing resources added or reduced, with an increase being a positive value and a decrease being a negative value; kc is the reward coefficient, which is set according to the importance of the added or reduced computing resources in the system; S3225, action fault information is processed to obtain system reliability rewards; r s -stable=-k stable Among them, r s-stable The system reliability reward is: if the remote sensing satellite application system fails after a certain action during the mission, the severity of the failure k stable Give corresponding punishment s -stable, the reward is negative; S3226: Integrate the satellite resource reward, the storage resource reward, the computing resource reward, and the system reliability reward to obtain an additional reward.
7. The remote sensing satellite application system intelligent evolution method according to claim 6, characterized in that: The processing of the actions of the remote sensing satellite application system to obtain the task completion reward and the task completion timeliness reward includes: S32211, processing the actions of the remote sensing satellite application system to obtain a task completion reward; the expression of the task completion reward rtc is: Among them, Aa i It is the actual observation area of the remote sensing satellite application system based on the research results of intelligent evolution for the i-th area requiring detection, which comes from the actual mission requirements; Ar i is the area of the i-th area to be detected; ta i is the observation time required for the i-th area to be detected; tr i is the actual observation time of the i-th area to be detected, w i is the preset weight coefficient, ktc is the reward coefficient, which is set by the experiment, n is the number of areas to be detected, and * indicates multiplication; S32212: Process the actions of the remote sensing satellite application system to obtain a timeliness reward for task completion; the timeliness reward for task completion is expressed as: where kt i is the importance coefficient of the i-th task, treq i is the completion time of the i-th task, actual i is the required completion time of the i-th task.
8. An intelligent evolution device for a remote sensing satellite application system, characterized in that: The device comprises: System modeling module, used to model the remote sensing satellite application system and obtain the remote sensing satellite application system twin; A status information acquisition module is used to collect information about the remote sensing satellite application system twin to obtain twin status information; the twin status information includes satellite system status information, ground control status information, satellite data storage and calculation status information, and service task completion information for the detection object; The intelligent evolution strategy generation module is used to process the twin state information to obtain an intelligent evolution strategy.
9. A remote sensing satellite application system intelligent evolution device, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the remote sensing satellite application system intelligent evolution method according to any one of claims 1 to 7.
10. A computer storable medium, characterized in that The computer storable medium stores computer instructions, and when the computer instructions are called, they are used to execute the remote sensing satellite application system intelligent evolution method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Deep reinforcement learning design method of intelligent satellite in digital twin environment
CN115391924A
Simulation deduction system based on digital twinning
CN117150757A
Cluster collaborative target search method based on digital twinning and deep reinforcement learning
CN117930863A
Satellite-ground collaborative edge network resource allocation method based on depth deterministic strategy gradient
CN119109504A