Spacecraft form control method and system based on evolutionary algorithm and reinforcement learning

Abstract

The invention discloses a spacecraft form control method and system based on an evolutionary algorithm and reinforcement learning, and the method specifically comprises the steps: S1, building an initial population, and enabling individuals in the population to be spacecrafts of different forms, which are different functional modules and different module number combinations; s2, performing inner ring initialization learning training on all individuals in the population, and calculating a fitness value of each individual; s3, selecting individuals with relatively high fitness values to form an elite population; s4, performing uniform crossover and single-point mutation on the elite population by using genetic and mutation operations in a genetic algorithm to generate elite offspring; s5, inner ring reinforcement learning: performing learning training on the elite offspring obtained in the step S4; s6, performing form evaluation; and S7, forming an optimal individual. According to the method, the characteristics of a space environment and task requirements are fully combined, an inner and outer ring algorithm architecture based on deep evolution reinforcement learning is adopted, and through continuous alternation of outer ring form evolution and inner ring learning training, form autonomous generation of the modular spacecraft is realized.

Description

Technical Field

[0001] The present invention relates to the field of spacecraft design optimization and autonomous control, and in particular to a spacecraft morphology control method and system based on evolutionary algorithms and reinforcement learning. Background Art

[0002] Traditional spacecraft utilize a highly customized design approach, resulting in relatively fixed topology and mission adaptability. Functionality must be pre-designed based on specific mission requirements. Their structure, functionality, and operational modes remain largely unchanged during in-orbit operation, leading to high R&D costs, limited flexibility, and long development cycles. Modular spacecraft can effectively address these issues.

[0003] By defining existing single machines or subsystems as standardized modules and designing controllable connection mechanisms, modular spacecraft can achieve flexible combinations of different modules and realize economical and efficient mass production by leveraging the advantages of low R&D costs of standardized modules. By upgrading and replacing modules, the service life of the spacecraft can be significantly improved, the entire satellite system design can be simplified, the production and development cycle can be shortened, and the needs of complex space environments and diversified missions can be better adapted.

[0004] However, existing modular spacecraft still face limitations in their reconfiguration methods during actual operation. These methods rely primarily on manual operation by astronauts or with the assistance of orbiting robots, resulting in limited reconfiguration opportunities, poor autonomy, low efficiency, and insufficient flexibility.

[0005] Furthermore, traditional spacecraft have a fixed structural design, requiring pre-designed functionality based on mission requirements. Existing modular spacecraft reconfiguration methods rely primarily on manual operation by astronauts or with the assistance of orbiting robots, making it difficult to achieve true autonomous on-orbit reconfiguration capabilities during operation. Summary of the Invention

[0006] To address the challenges of the existing technology, the present invention aims to provide a spacecraft morphology control method based on an evolutionary algorithm and reinforcement learning. This method achieves autonomous morphology generation for modular spacecraft through the continuous alternation of outer-loop morphology evolution and inner-loop learning training. Another object of the present invention is to provide a spacecraft morphology control system based on an evolutionary algorithm and reinforcement learning that implements the aforementioned method.

[0007] To achieve the above objectives, the present invention provides a spacecraft morphology control method based on evolutionary algorithms and reinforcement learning, specifically: S1. Establish an initial population, where the individuals in the population are spacecraft of different configurations, with different functional modules and different numbers of modules. S2. Perform inner loop initialization learning training on all individuals in the population and calculate the fitness value of each individual; S3. Select individuals with higher fitness values to form an elite population; S4. Use the inheritance and mutation operations in the genetic algorithm to perform uniform crossover and single-point mutation on the elite population to generate elite offspring; S5. Inner-loop reinforcement learning: Training the elite offspring obtained in step S4. S6. Morphological Assessment: Analyze the spacecraft's mission performance in the current mission scenario, obtain comprehensive morphological assessment results, and use these results as the fitness value of the spacecraft's morphology, which will serve as the basis for evaluating the outer loop evolution. S7. Forming the optimal individual: The elite offspring that have completed the inner-loop reinforcement learning training are added to the original population. The individuals in the population are sorted by their fitness values and the individuals with the lowest fitness are eliminated until the number of individuals in the population matches the initial population size. When the outer loop reaches the set evolutionary generation, the optimal individual is output.

[0008] Furthermore, steps S1, S2, S3, S4, and S7 are the outer loop morphological evolution process, and steps S5 and S6 are the inner loop reinforcement learning process.

[0009] Furthermore, the inner loop includes an intelligent agent, a sensor, a controller and an actuator; the intelligent agent is the research object, that is, an individual in the population, and the sensor, the controller and the actuator realize the perception and control behavior of the intelligent agent.

[0010] Furthermore, the reward value obtained from the inner loop learning and training process will be used to calculate the fitness value of the individuals in the population and output to the outer loop.

[0011] Furthermore, the outer loop completes the evolution of the population from generation to generation according to the fitness values of the individuals in the population, thus realizing the evolution of the spacecraft morphology.

[0012] Further, in step S1, define As the initial population set in the genetic algorithm, the total number of Individuals.

[0013] Furthermore, in step S3, randomly select Individuals, hold tournaments, The winner of each tournament, that is, the individual with the highest fitness value, is the parent of the elite population. Individual composition.

[0014] Furthermore, in step S4, based on the elite genetic algorithm, by using the crossover mutation operator in the population iteration, the elite population is obtained through uniform crossover and single point mutation. Elite offspring.

[0015] Furthermore, in step S5, the inner loop reinforcement learning stage uses the nested PPO algorithm to Elite offspring are trained for study.

[0016] The present invention provides a spacecraft morphology control system based on an evolutionary algorithm and reinforcement learning, and the system is used to implement the above-mentioned spacecraft morphology control method based on an evolutionary algorithm and reinforcement learning of the present invention.

[0017] This invention fully combines the characteristics of the space environment and mission requirements, and is based on the inner and outer loop algorithm architecture of deep evolutionary reinforcement learning. Through the continuous alternation of outer loop morphological evolution and inner loop learning training, the autonomous generation of the morphology of modular spacecraft is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a framework diagram of the present invention; Figure 2 It is a flow chart of the present invention; Figure 3 This is the block diagram of the inner loop reinforcement learning algorithm; Figure 4 is the attitude and trajectory control reward function curve; Figure 5 This is the structure diagram of the outer loop evolutionary algorithm; Figure 6 It is a diagram showing the change process of the fitness value of the outer ring population. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0021] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0022] The following combination Figures 1-6 The specific embodiments of the present invention are described in detail. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0023] The present invention is based on the spacecraft morphology control method and system of evolutionary algorithm and reinforcement learning, based on the inner and outer loop algorithm framework of deep evolutionary reinforcement learning, and uses evolutionary algorithm and reinforcement learning to realize modular spacecraft morphology optimization for different mission scenarios. The structural framework of the present invention is shown in the figure below. Figure 1 shown.

[0024] Among them, the definition As the initial population set in the genetic algorithm, 、 、 ∈ , 、 、 is an individual in the population set, each individual represents a different form of spacecraft, that is, a spacecraft with different functional modules and different module quantity combinations. The total number of individuals in the initial population is Individuals.

[0025] It should be clarified that the present invention defines the form of a spacecraft as a configuration, that is, a spacecraft is composed of a combination of different functional modules and different numbers of modules.

[0026] The outer cycle is the process of realizing the intergenerational evolution of the population: First, an initial population is established. Then, all individuals in the population undergo inner-loop initialization learning and training, and the fitness value of each individual is calculated. Here, the fitness value is the result of the fitness function and serves as a quantitative indicator of individual quality, determined by the morphological evaluation rules described later. Individuals with higher fitness values are then selected to form an elite population. Using the inheritance and mutation operations in the genetic algorithm, uniform crossover and single-point mutation are performed on this elite population to generate elite offspring.

[0027] The inner cycle is the process of realizing the lifelong evolution of individuals in the population: The inner loop includes agents, sensors, controllers, and actuators. The agent is the research object, that is, the individual in the population, and the sensors, controllers, and actuators realize the agent's perception and control behavior.

[0028] First, a training environment is set up within the outer loop, given the spacecraft configuration. Through continuous interaction between individuals in the population and the mission scenario, the optimal controller, output configuration, and attitude and orbit control strategy are trained according to the predefined configuration evaluation rules. The rewards earned from completing the mission form the optimal configuration for the current generation. The rewards earned during the inner loop's learning and training process are used to calculate the fitness values of the individuals in the population and output them to the outer loop. Based on the fitness values of the individuals in the population, the outer loop evolves the population generation by generation, achieving the evolution of the spacecraft configuration.

[0029] The present invention is based on the spacecraft morphology control method of evolutionary algorithm and reinforcement learning. The specific method steps are as follows: Figure 2 As shown: ① Generate the initial population, ② Inner loop initialization learning and training, ③ Obtain the elite population, ④ Generate elite offspring, ⑤ Inner loop reinforcement learning, ⑥ Morphological evaluation, ⑦ Form the optimal individual. ①, ③, ④, and ⑦ are the outer loop morphological evolution algorithm, while ②, ⑤, and ⑥ are the inner loop reinforcement learning algorithm.

[0030] ①Generate the initial population: In the initial stage, a modular spacecraft is first modeled based on different functional modules and different numbers of modules. Define the vector The functional modules and the number of functional modules inside the spacecraft are shown below.

[0031] (1) ; in, Indicates the number of functional modules in a modular spacecraft. Specifically, Indicates the number of attitude and orbit control modules, Indicates the number of propulsion modules, Indicates the number of energy modules, Indicates the number of control modules, Indicates the number of computing modules, Indicates the number of communication modules, Indicates the number of optical sensing modules, Indicates the number of radar perception modules, Indicates the number of electronic positioning modules, Represents the number of electromagnetic interference modules. According to the functional modules and the number of functional modules in the spacecraft and the connection constraints between the functional modules, the population individuals are randomly generated to establish the initial population of the modular spacecraft.

[0032] ②Inner loop initialization learning and training: Individuals in the initial population are sent to the inner loop to interact with the task scenario for initial learning and training, and the fitness value of each individual in the population is obtained. During the inner loop training process, a multi-process approach is used to assign a separate process to each individual's learning and training, thereby achieving parallel computing and improving computing efficiency. The individual fitness function is expressed as: (2); in, Indicates the number of typical task scenarios, Indicates the spacecraft in the mission scenario The reward value obtained through inner-loop reinforcement learning, Representing a task scenario The upper limit of the reward value, Represents the standard weight of the reward function, which is used to unify the magnitude of the reward value and fitness value.

[0033] In the inner loop reinforcement learning phase, a nested PPO algorithm is used to train individuals in the entire initial population. The PPO algorithm uses an Actor-Critic architecture to implement spacecraft configuration and attitude and orbit control strategies. The Actor model uses a neural network Fit the control strategy function of the spacecraft, where is the Actor network parameter to be optimized. The Actor network is based on the environmental situation information of the task scene. , output the spacecraft's configuration and maneuvering strategies ; The Critic model also consists of a neural network fitting an evaluation function ,in are network parameters. Output The PPO algorithm needs to sample a large amount of data to form an experience buffer to feed the neural network training and update the strategy. However, a single Actor network cannot train and update the network parameters when performing the sampling task. Therefore, a sampling network is designed. Specifically for experience sampling, where It is the sampling network parameters, while the Actor network is only responsible for continuously updating parameters and strategies using empirical data.

[0034] The PPO algorithm takes the expectation of maximizing the dynamic weight advantage function as the target cost function, as shown below: (3); in, , Respectively represent the current time step in the task process The state of the task scenario environment and the actions taken by the Actor network; Indicates that the current parameters Under the strategy followed, the mission scenario is due to the spacecraft performing actions By status Transition to the next state The probability of It means that in the parameter The goal of the optimization function is to maximize the advantage function In the probability weight Expectations , where the advantage function can be expressed as: (4); in, represents the spacecraft at time step The environment reward obtained through reinforcement learning reward function, represents the reward discount factor, Indicates the slave state Expected cumulative discount rewards from Indicates the slave state The expected cumulative discount rewards to be obtained from

[0035] The reinforcement learning training framework of the double PPO algorithm nested in the present invention is as follows Figure 3 As shown. vectors Input inner loop configuration reinforcement learning. Process vectors sequentially The functional modules corresponding to each element in attitude and orbit control modules, A propulsion module... Based on the module connectivity constraints and the order of installation (attitude and orbit control, propulsion, and electromagnetic interference), each module is randomly located in turn, ultimately forming an initial configuration. This method uses a single attitude and orbit control module as the initial unit and iteratively solves the connection space in which the current configuration satisfies the connectivity constraints. New modules are then randomly added to update the configuration until all functional modules are traversed. This method yields an initial configuration that is connected, reasonable, and random.

[0036] The configuration strategy output by the Actor 1 network of the Configuration PPO algorithm is a sequence of actions, followed by a transformation of the initial configuration. The action sequence specifies the flipping direction for each module. Each action is processed sequentially: the motion space for the corresponding module is calculated; if the flip is feasible (within the motion space) and the flipped configuration satisfies connectivity and constraints, the flip is performed and the configuration is updated. This process is iterated based on the new configuration until the sequence is processed completely, resulting in the training configuration.

[0037] The training configuration is input into the attitude and orbit control reinforcement learning, and the attitude and orbit control strategy for typical mission scenarios is obtained through reinforcement learning. The training configuration completes the state transfer by executing the attitude and orbit control strategy output by the Actor2 network, and obtains the new attitude and orbit control reinforcement learning environment state and the corresponding reward value. It consists of action rewards, boundary penalties, success rewards, and failure penalties. The Critic2 network evaluates the control strategy of Actor2 based on the attitude and orbit control reinforcement learning environment state and reward value, and optimizes the network parameters of Actor2 accordingly to improve the performance of attitude and orbit control. The attitude and orbit control reward function curve under the present invention is shown in the figure below: Figure 4 At the same time, the attitude and orbit control effect of the controller is evaluated through the effectiveness evaluation rule, and the evaluation result is output to the configuration reinforcement learning.

[0038] Then the Critic1 network evaluates the configuration change strategy of Actor1 through the current state information and evaluation information, and completes the parameter adjustment of Actor1 network. Configuration reinforcement learning evaluates the strategy of Actor1 through the configuration reward function (numerical form). The expression is as follows ; in, They represent the spacecraft's mission completion rate reward, mission completion time reward, mission cost penalty, platform cost penalty, and the cumulative reward of reinforcement learning during the interaction of the inner loop typical mission scenario, and The calculation method is shown in "Morphological Evaluation ⑥".

[0039] ③ Obtain elite population: Randomly select from the initial population Individuals, hold tournaments, Group Championship. is the tournament size, which refers to the number of individuals randomly selected for comparison in each competition. The winner of each tournament, that is, the individual with the highest fitness value, serves as the parent to form the elite population. The elite population consists of Individual composition.

[0040] ④ Producing elite offspring: The elite population is obtained through uniform crossover and single-point mutation Elite offspring. Based on the elitist genetic algorithm, the crossover mutation operator is used in population iterations to increase population diversity, expand the algorithm's optimization space, and ensure full evolution of the spacecraft morphology. The crossover operator uses a uniform crossover method. The population is traversed. When an individual triggers the crossover operator with a crossover probability, that individual becomes the father. A uniform crossover is then performed on another individual as the mother. The individual's chromosomes are traversed, i.e., every gene position in its morphological variable matrix is traversed. If a gene position triggers a uniform crossover with a uniform crossover probability, the genes at the corresponding gene position in the father and mother are exchanged, adjusting the number of functional modules in the individual. During this process, the module types corresponding to the gene position are checked to see if they meet the specific module connection constraints in the model. If a specific module connection exists, the genes of the specific connection module at the parental exchange gene position are simultaneously transformed to ensure that the offspring consistently meet the module constraints. If a uniform crossover is not triggered, the gene position remains unchanged and the operation moves on to the next operation position until all gene positions in the morphological variable matrix are traversed, completing a uniform crossover.

[0041] The mutation operator adopts a single-point mutation method. The population is traversed, and when an individual triggers the mutation operator through the mutation probability, the chromosome of this individual is selected for operation, that is, a gene position is randomly selected in the individual's morphological variable matrix for single-point mutation. During the single-point mutation process, the genes on the gene position, that is, the corresponding number of functional modules, are adjusted to ensure random mutation reconstruction within the range allowed by the module quantity constraint. At the same time, check whether the module type corresponding to the gene position meets the specific module connection constraint in the model constraint. If there is a specific module connection, the number of modules on the specific connection module gene position corresponding to the mutated gene position is synchronously transformed to ensure that the offspring always meets the module constraint. The above process completes a morphological variable mutation operation. The complete outer loop evolutionary algorithm structure is as follows Figure 5 shown.

[0042] ⑤ Inner-loop reinforcement learning: The inner loop reinforcement learning stage calls the same nested PPO algorithm as the initialization learning training in step ②. Unlike step ②, this step does not require the input of the entire population of individuals to the inner loop reinforcement learning, but only requires the input of the individuals obtained in step ④ of the outer loop. Elite offspring are trained and the fitness value of the elite offspring is obtained through the fitness function.

[0043] ⑥Morphological assessment: The task completion status of the spacecraft in the current mission scenario is analyzed, and a comprehensive morphological evaluation method is proposed. The reward function value and task execution status obtained by training in the current mission scenario are used as the basis for comprehensive morphological evaluation. The design includes the task completion rate , Task completion time 、Task Reward , cost , platform value Spacecraft morphology evaluation rules including indicators. Mission completion rate Assess spacecraft mission execution progress and mission completion time The evaluation of the number of training steps of attitude and orbit control reinforcement learning is as follows: ; in, The number of time steps for attitude and orbit control mission execution, This is the sign of the completion of attitude and orbit control mission. This scene of reinforcement learning training mission is completed. =1, otherwise =0. Mission Reward Refers to the value of the reward function for attitude and orbit control reinforcement learning training; cost It is used to evaluate the energy consumption of spacecraft mission execution process, and its expression is as follows: ; in, For the The control force output by the spacecraft, The number of time steps executed for the attitude and orbit control task. It is used to evaluate the total number of functional modules required for a spacecraft to complete its mission. The expression is as follows: ; in, is the total number of functional modules.

[0044] The reward function will be constructed It is used as the fitness value of the spacecraft morphology, and as the evaluation basis for the outer loop evolution to support morphological evolution.

[0045] ⑦Optimal individual: After the inner loop reinforcement learning training is completed, the elite offspring with the obtained fitness value will be added to the original population of their father to expand the number of individuals in the population. Then, the individuals in the population will be sorted according to the size of the fitness value and the individuals with the lowest fitness will be eliminated until the number of individuals in the population is consistent with the initial population. When the outer loop reaches a certain evolutionary generation, the optimal individual will be output. The process of the outer loop population fitness value change is as follows: Figure 6 .

[0046] This paper addresses the challenges of traditional fixed-structure spacecraft, such as long R&D cycles, high development costs, rigid functionality, and the inability of microsatellites to flexibly adapt to complex external environments. By implementing modular modeling, the system then employs an inner and outer loop algorithm framework based on deep evolutionary reinforcement learning. The outer loop uses an elite genetic algorithm to topologically reorganize functional modules to achieve spacecraft morphological evolution. The inner loop uses reinforcement learning to learn the spacecraft's configuration strategy and attitude and orbit control strategy, using the expected maximization of a dynamic weighted advantage function as its target cost function. Data generated during the inner loop's training process is used to calculate the fitness values of individuals in the population and output this data to the outer loop, which then evolves the spacecraft's morphology based on these fitness values until the population converges to the optimal individual.

[0047] The present invention realizes the morphology optimization and control of modular spacecraft through the continuous alternation of morphology evolution and training, thereby achieving the goal of improving the spacecraft's environmental adaptability, rapid responsiveness and mission agility.

[0048] Any process or method described in the flowchart of the present invention or in other ways herein can be understood as representing a module, segment or portion of code including one or more executable instructions for implementing specific logical functions or process steps, which can be implemented in any computer-readable medium for use by an instruction execution system, device or apparatus. The computer-readable medium can be any medium that stores, communicates, propagates or transmits a program for use by an execution system, device or apparatus, including read-only memory, magnetic disk or optical disk, etc.

[0049] Throughout this specification, reference to terms such as "embodiment" and "example" indicates that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, those skilled in the art may combine or integrate different embodiments or examples described in this specification, as well as features therein, without creating any inconsistency.

[0050] Although the above content has shown and described the embodiments of the present invention, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can perform update operations such as changes, modifications, replacements and variations on the above embodiments within the scope of the present invention.

Claims

1. A spacecraft morphology control method based on evolutionary algorithms and reinforcement learning, characterized in that: Specifically: S1. Establish an initial population, where the individuals in the population are spacecraft of different configurations, with different functional modules and different numbers of modules. S2. Perform inner loop initialization learning training on all individuals in the population and calculate the fitness value of each individual; S3. Select individuals to form an elite population based on their fitness values; S4. Use the inheritance and mutation operations in the genetic algorithm to perform uniform crossover and single-point mutation on the elite population to generate elite offspring; S5. Inner-loop reinforcement learning: Training the elite offspring obtained in step S4. S6. Morphological Assessment: Analyze the spacecraft's mission performance in the current mission scenario, obtain a comprehensive morphological assessment result, and use the assessment result as the fitness value of the spacecraft morphology, which will serve as the evaluation basis for the outer loop evolution; S7. Forming the optimal individual: The elite offspring that have completed the inner loop reinforcement learning training are added to the original population. The individuals in the population are sorted according to the size of the fitness value and the individuals with the lowest fitness are eliminated until the number of individuals in the population is consistent with the initial population. When the outer loop reaches the set evolutionary generation, the optimal individual is output.

2. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 1 is characterized in that: The steps S1, S2, S3, S4 and S7 are the outer loop morphological evolution process, and the steps S5 and S6 are the inner loop reinforcement learning process.

3. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 2 is characterized in that: The inner loop includes an intelligent agent, a sensor, a controller, and an actuator; the intelligent agent is the research object, that is, an individual in the population, and the sensor, controller, and actuator realize the perception and control behavior of the intelligent agent.

4. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 2 is characterized in that: The reward value obtained from the inner loop learning and training process will be used to calculate the fitness value of the individuals in the population and output to the outer loop.

5. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 4 is characterized in that: The outer loop completes the evolution of the population from generation to generation based on the fitness values of the individuals in the population, thus realizing the evolution of the spacecraft morphology.

6. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 1 is characterized in that: In step S1, define As the initial population set in the genetic algorithm, the total number of Individuals.

7. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 1 is characterized in that: In step S3, randomly select Individuals, hold tournaments, The winner of each tournament, that is, the individual with the highest fitness value, is the parent of the elite population. Individual composition.

8. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 1, characterized in that: In step S4, based on the elite genetic algorithm, by using the crossover mutation operator in the population iteration, the elite population is obtained through uniform crossover and single point mutation. Elite offspring.

9. The spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to claim 1, characterized in that: In step S5, the inner loop reinforcement learning phase uses the nested PPO algorithm to Elite offspring are trained for study.

10. A spacecraft morphology control system based on evolutionary algorithms and reinforcement learning, characterized by: The system is used to implement the spacecraft morphology control method based on evolutionary algorithm and reinforcement learning according to any one of claims 1 to 9.

Images