Robot co-evolution method and system based on sub-isovariant graph neural network

Abstract

The invention discloses a robot co-evolution method and system based on a sub-isovariant graph neural network, and belongs to the technical field of intelligent and reinforcement learning. The method comprises the following steps: constructing a co-evolution framework comprising three stages of structure transformation, attribute adjustment and behavior control; introducing a sub-isovariant graph neural network, dynamically generating a local reference frame (LRF), and projecting a vector state in a 3D environment to a geometric invariant space; a symmetric group constraint network structure is utilized to ensure that the strategy is kept consistent under rotation and translation; morphological evolution and behavior strategies are jointly optimized through a PPO algorithm. According to the method, the sample efficiency, generalization ability and form-task adaptability in 3D complex tasks are remarkably improved, and the method is suitable for reconfigurable robot platforms such as four-footed and human-shaped robot platforms.

Description

TECHNICAL FIELD

[0001] The present application relates to the cross field of artificial intelligence, robotics and reinforcement learning, in particular to a method and system for realizing joint adaptive evolution of robot morphology and behavior in 3D environment by using geometric symmetry prior, which can be applied to the autonomous design and control of reconfigurable robots, bionic robots and embodied agents. BACKGROUND

[0002] The current robot design field generally adopts the traditional paradigm of fixed morphology combined with special controllers. This paradigm can achieve specific functions in structured and static scenes, but when facing dynamic changes and unstructured 3D complex environments (such as field search and rescue, dynamic confrontation, unknown terrain navigation, etc.), it has significant adaptability shortcomings. The fixed morphology is difficult to match the changing physical interaction requirements in the environment, and the special controller cannot flexibly cope with the task differences brought by the changes in spatial orientation and target position, resulting in a significant decline in the task completion rate of robots in complex scenes, and it is difficult to meet the core requirements of environmental adaptation of embodied intelligence.

[0003] In recent years, to break through the limitations of the traditional paradigm, Morphology-Behavior Co-evolution technology has become a research hotspot in the field of embodied intelligence. This technology uses reinforcement learning algorithms to jointly optimize the morphology structure (such as the number of limbs, joint layout) and behavior control strategy (such as motion trajectory planning, action execution logic) of the robot, trying to realize the closed-loop adaptation of morphology adapting to the environment and behavior matching the morphology. Among them, the representative research results include the Transform2Act system and the CompetEvo framework: Transform2Act realizes the dynamic adjustment of morphology structure and the synchronous update of behavior strategy through hierarchical reinforcement learning, and can optimize the morphology of the robot end effector for simple grasping tasks; CompetEvo introduces an adversarial training mechanism to promote the co-evolution of morphology and behavior in multi-robot interaction scenarios. However, these existing technologies still have three major core defects when facing 3D complex tasks, which seriously restrict their practical application value: 1. Lack of 3D geometric symmetry utilization, resulting in weak strategy generalization ability and low sample efficiency In typical 3D tasks such as navigation, confrontation, and obstacle avoidance, the environment itself has natural rotation / translation invariance. For example, in the navigation task, the target point can be located in any direction of the robot, and in the confrontation task, the opponent can initiate interaction from multiple angles. The core task logic of these scenarios, such as moving towards the target and avoiding opponent attacks, is independent of the absolute spatial orientation. However, existing form-behavior co-evolution methods do not incorporate this geometric symmetry into their technical design. Instead, they treat the robot's position, direction, and other state information as absolute coordinates: for example, in strategy training, the robot facing north and the target located south are considered two completely different states, and separate strategies need to be trained to adapt to them. This design leads to two key problems: first, the strategy generalization ability is weak, and the trained strategy can only adapt to specific orientation scenarios, and performance will drop sharply after changing the target direction or initial position; second, the sample efficiency is low, and to cover all possible spatial orientations, a large number of training samples with different orientations need to be collected, significantly increasing training time and computational resource consumption, making it difficult to meet the needs of rapid deployment in practical applications.

[0004] Second, joint optimization search space explosion, easy to fall into local optimal solution The optimization goal of form-behavior co-evolution includes two dimensions: form and behavior. The form dimension includes structure (such as the number of limb nodes, joint connection relationship) and attribute (such as limb length, mass, joint torque limit) two types of parameters. Taking a quadruped robot as an example, the form dimension alone includes dozens of optimization variables such as whether to add additional limbs, length / mass adjustment of each limb, torque upper limit setting of each joint, while the behavior dimension involves continuous action control instructions (such as joint angle, motion speed) optimization. Existing technologies do not effectively constrain or reduce the optimization space, and directly include all variables in a unified optimization framework, resulting in an exponential expansion of the overall search space. In the reinforcement learning process, the algorithm is difficult to traverse all possible optimization combinations and is easily trapped in a local optimal solution. For example, in the confrontation task, a combination of short limbs, high torque form, and close-range collision behavior may be evolved, which performs well in close-range confrontation but cannot adapt to medium- and long-range confrontation scenarios. The algorithm cannot escape this local optimum and further explore more optimal form-behavior combinations such as long limbs and flexible avoidance. This local optimal problem makes it difficult for existing technologies to evolve high-performance form and behavior strategies that adapt to complex 3D tasks.

[0005] Third, lack of geometric consistency guarantee, unable to achieve one-time training, multi-direction reuse Since no geometric symmetry constraint is imposed on the policy, the existing technology-trained morphology-behavior policy does not have geometric consistency, i.e. the same task needs to be repeated in different spatial orientations to adapt to the new orientation. For example, in a 3D navigation task, if the initial training is based on the scenario where the target is in front of the robot, when the target is changed to the side or behind the robot, the original policy cannot be directly reused, and the training samples for this orientation need to be collected again and a complete training cycle needs to be performed. In the confrontation task, if the opponent only attacks from the left during training, when the opponent attacks from the right, the robot cannot quickly adjust the defense strategy, and it is easy to make mistakes in response. This one-time training mode violates the core goal of embodied intelligence generalization and adaptation, not only greatly increasing the time cost of technology landing, but also limiting the real-time response capability of the robot in a dynamic changing environment. In the scene of field search and rescue, dynamic confrontation and other scenarios that require quick response to orientation changes, the lag of repeated training will directly lead to task failure.

[0006] IV. Hard-coded symmetric structure limits morphology diversity, violating the principle of environment-driven evolution To solve the above geometric symmetry-related problems, some studies (such as the SARD method) attempt to introduce symmetry constraints, but use a hard-coded symmetric structure design approach: that is, by artificially presetting the symmetric morphology of the robot, it tries to indirectly ensure the geometric adaptability of the strategy through the symmetry of the morphology. Although this design can alleviate the generalization problem to some extent, it has a more fundamental flaw that limits the diversity of morphology: for example, in tasks that require unilateral limb flexibility and the other side to provide stable support, the hard-coded bilateral symmetric structure forces both sides to have the same morphology, making it impossible to evolve an asymmetric adaptive morphology with a long operating limb and a short supporting limb. In a low-gravity 3D environment, the hard-coded symmetric structure also cannot explore an asymmetric limb layout to improve motion efficiency. More importantly, this artificial preset symmetric structure approach violates the core principle of environment-driven evolution of morphology-behavior co-evolution, and the evolution of morphology should be determined by task requirements and environmental characteristics, rather than being limited by artificial preset structure constraints. Hard-coded symmetric structures make the morphology evolution deviate from the actual needs of the environment, making it difficult to adapt to complex and changing 3D task scenarios.

[0007] In summary, existing morphology-behavior co-evolution technology faces four core problems in 3D complex environments: lack of geometric symmetry utilization, search space explosion, lack of geometric consistency, and limited morphology diversity. A new technical solution is urgently needed: it can automatically perceive and utilize the geometric symmetry of the 3D environment without artificial preset symmetric structures; it can effectively constrain the search space of joint optimization to improve optimization efficiency; it can ensure the geometric consistency of the strategy, enabling one-time training and multi-direction reuse; and it supports any initial morphology, truly realizing environment-driven morphology-behavior co-evolution, and providing technical support for the autonomous design and control of embodied agents such as reconfigurable robots and bionic robots in 3D complex scenarios. SUMMARY

[0008] To solve the above problems, the application provides a robot collaborative evolution method based on a sub-isometry graph neural network, comprising the following steps: S1, representing the morphology of the robot as a topological graph and representing the state of the robot as a sub-isometry graph, the sub-isometry graph containing scalar features, vector features, and a gravity direction vector, and the vector features including at least one of position, velocity, and angular velocity, the position being converted into a translation-invariant representation by subtracting the root node position; S2, sequentially performing the following in each training period: S21, a structure transformation stage: modifying the nodes and edges of the topological graph by a first graph neural network to change the robot morphology structure; S22, an attribute adjustment stage: modifying the node attributes of the topological graph by a second graph neural network to adjust the limb parameters of the robot, the limb parameters including at least one of limb length, mass, and joint torque limit; S23, a behavior control stage: predicting a local reference frame by a sub-isometry graph neural network, projecting the sub-isometry graph into a geometric-invariant graph, and outputting a control action based on the geometric-invariant graph by a third graph neural network; S3, jointly updating the first, second, and third graph neural networks according to the environment reward using a proximal policy optimization (PPO) algorithm, and estimating a value function based on the geometric-invariant graph.

[0009] Further, the sub-isometry graph neural network is configured to satisfy isometry, is a Euclidean transformation subgroup composed of rotation, reflection, and translation operations that keep the gravity direction unchanged, and the message passing mechanism of the sub-isometry graph neural network is configured to: calculate edge messages based on the relative positions and features between nodes; aggregate the edge messages and update the node state in combination with the node's own features; and all message passing operations satisfy isometry.

[0010] Further, the local reference frame is constructed by: outputting two candidate vectors by the sub-isometry graph neural network; orthogonalizing the two candidate vectors with the gravity direction vector as the third axis to construct a rotation matrix; and projecting the vector features into a local coordinate system using the rotation matrix to obtain geometric-invariant representation.

[0011] Further, the structure transformation stage and the attribute adjustment stage are performed in the early stage of the training period, do not interact with the environment, and do not generate environment rewards; the behavior control stage is performed in the later stage of the training period, interacts with the environment, and obtains environment rewards.

[0012] Further, the method is suitable for long-term tasks in three-dimensional space, including at least one of navigation, confrontation, and obstacle crossing.

[0013] In another aspect, the present application also provides a system for implementing the above method, comprising: a morphology representation module for constructing a topological graph and a sub-isovariant graph of a robot; a three-stage strategy execution module including a structure transformation sub-network, an attribute adjustment sub-network, and a behavior control sub-network; a joint optimization module for updating the three sub-networks according to environmental rewards; and an execution interface module for connecting a simulation environment or a real robot platform.

[0014] Further, the behavior control sub-network in the three-stage strategy execution module is configured to receive the geometric invariant graph projected in the local reference frame as input and output motor control instructions.

[0015] Further, the execution interface module supports at least one of the simulation engines MuJoCo, Isaac Gym, and PyBullet, or supports a reconfigurable quadruped robot hardware platform.

[0016] Further, in the behavior control stage, the control action outputted by the third graph neural network after inputting the projected invariant graph corresponds to the target angle of the robot joint, and the environmental reward obtained by the robot after performing the action includes a task success reward, a distance-related reward, a control cost penalty, and a survival reward.

[0017] Further, the total step length of the training cycle can be set according to task requirements, the execution step number of the structure transformation stage and the attribute adjustment stage accounts for a smaller proportion of the total step length of the training cycle, and the execution step number of the behavior control stage accounts for a larger proportion of the total step length of the training cycle, so as to ensure that the robot and the environment fully interact and obtain effective rewards for strategy optimization.

[0018] The present application has the following advantages: (1) The sub-isovariant graph neural network satisfies symmetry group constraints, maintains rotation, reflection, and translation invariance with the direction of gravity unchanged, and projects the vector state to the geometric invariant space combined with the dynamic local reference frame (LRF), so that the strategy is free from dependence on absolute coordinates. In a 3D navigation task, the trained strategy can be directly adapted to scenes with target points in any orientation without the need for repeated training for different orientations. In a test environment rotated by 90°, the performance does not decrease significantly, solving the generalization shortcoming of the traditional method of one thing one training.

[0019] (2) Improve sample efficiency and reduce training resource consumption Since the strategy can automatically utilize the geometric symmetry of the 3D environment, it is not necessary to collect a large number of training samples with different orientations and different initial positions, thereby greatly reducing the data demand and computational cost. Taking the four-legged robot (Ant) navigation task as an example, the method (EquiEvoAnt) in the present application only needs about 1 million environment interactions to achieve a task success rate of 95%, while the sub-isometric baseline (EvoAnt) needs an additional 30% of interactions to approach the same performance, and the hard-coded symmetric method (SARDEvoAnt) is less efficient.

[0020] (3) Realize environment-driven adaptive morphological evolution and improve task matching degree Unlike the traditional hard-coded symmetric structure that limits the diversity of morphological diversity, the present application dynamically optimizes the morphology according to the task demand through three-stage cooperative evolution: in the 3D navigation task, the four-legged robot can evolve from the atomic torso to a radially symmetric long-legged structure, improving the stride and motion stability; in the Sumo confrontation task, a high-torque and long-legged structure is evolved, and the pushing efficiency is 40% higher than that of the baseline method; in the humanoid robot navigation task, the limb proportion is optimized to be tall and coordinated, the motion center of gravity is more stable, and the step frequency is improved by 20%. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is the overall flowchart of the method of the present application. It shows the three-stage evolution process from the initial morphology to the structure transformation, to the attribute adjustment, and to the behavior control, as well as the role of LRF projection and sub-isometric GNN. Figure 2 is a flowchart comparison diagram of whether to use the present application. It shows the detailed calculation process of message passing, LRF prediction, orthogonalization and projection. Figure 3 is a task schematic diagram of an embodiment of the present application. It includes 3D navigation (random target point) and Sumo confrontation (circular arena) scenes. Figure 4 is a comparison diagram of evolution results of the present application. It shows the performance difference of the results evolved by EquiEvo (the present application) and the baseline method (Evo, etc.) under the same task.

[0022] Figure 5 is a comparison diagram of evolved morphologies of the present application. It shows the morphology difference of the morphologies evolved by EquiEvo (the present application) and the baseline method (Evo, etc.) under the same task. DETAILED DESCRIPTION

[0023] With reference to the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0024] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture, and if the certain posture changes, the directional indications also change accordingly.

[0025] In addition, if the embodiments of the present application involve descriptions such as “first”, “second”, etc., the descriptions of “first”, “second”, etc. are only for description purposes, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first” and “second” can explicitly or implicitly include at least one of the features. In addition, the meaning of “and / or” appearing throughout the text includes three parallel solutions. Taking “A and / or B” as an example, it includes A solution, or B solution, or A and B solutions. In addition, “multiple” means more than two. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it. When the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist and is not within the protection scope of the present application.

[0026] The present application provides a morphology-behavior co-evolution method based on a sub-isometric graph neural network, which solves the problems of poor strategy generalization, low sample efficiency, and weak morphology-task mapping in 3D tasks through dynamic local reference system and symmetry constraint.

[0027] First, a three-stage co-evolution process is designed.

[0028] In each training cycle (episode), the agent performs three stages in turn: (1) Structure transformation stage: starting from an initial morphology (such as a single-node atomic torso), the graph neural network is used to decide whether to add or delete limbs and joints, and the topology of the robot is dynamically changed; (2) Attribute adjustment stage: on the basis of a fixed topology, the physical attributes of each limb, such as length, mass, joint torque limit, etc. are further adjusted; (3) Behavior control stage: Based on the evolved form, perform motor control actions, interact with the environment and obtain task rewards.

[0029] It is worth noting that the first two stages do not interact with the environment and therefore do not generate environmental rewards; their policy updates rely entirely on the long-term rewards of the behavior control stage, thereby achieving end-to-end joint optimization of form and behavior.

[0030] Second, we introduce sub-variable graph neural networks to process 3D geometric information.

[0031] Traditional methods treat robot states (such as position and velocity) as ordinary scalars, ignoring their vector nature and spatial directionality. This invention represents robot states as subequivariant graphs, where nodes contain scalar features (such as joint angles) and vector features (such as position and linear velocity), and explicitly introduces the gravity direction vector as a reference. Based on this, a subequivariant graph neural network is designed that can automatically learn the geometric symmetries in the environment (such as rotational invariance about the gravity axis), ensuring that the policy maintains consistent behavior under different orientations or positions.

[0032] Third, construct a dynamic local reference frame (LRF).

[0033] To enhance policy generalization capabilities, this invention introduces a Local Reference Frame (LRF) mechanism in the behavior control phase. Specifically, two candidate vectors are output from a sub-equivariant graph neural network. These vectors, combined with the gravity direction, are orthogonalized to construct a coordinate system that dynamically changes with the robot's posture. Subsequently, all vector states (such as limb position and velocity) are projected from the world coordinate system onto this local coordinate system, yielding a geometrically invariant representation. This process ensures that the policy no longer depends on absolute orientation but only on relative geometric relationships, thus significantly improving generalization performance in tasks with arbitrary orientation.

[0034] Fourth, achieve end-to-end training of morphology-behavior joint strategies.

[0035] This invention employs a proximal policy optimization (PPO) algorithm to uniformly train three sub-policies (structure, attribute, and behavior). The behavior control policy predicts actions based on a projected invariant graph; the value function is also estimated based on the invariant graph, ensuring that the value assessment is invariant to geometric transformations. Since morphological transformations do not offer immediate rewards, their gradients are backpropagated through long-term rewards in the behavior phase, forming a closed-loop optimization. The entire training process does not require a predefined symmetric structure; instead, it automatically evolves a task-adaptive morphology through environmental interaction.

[0036] Fifth, a 3DS-MB simulation environment was constructed to verify the effectiveness of the method.

[0037] To test the performance of the present application in real 3D scenes, we extended the MuJoCo simulation platform and designed two types of tasks with rich spatial geometric information: (1) 3D navigation task: the robot starts from a random starting point and navigates to a random target point within a radius of [3, 4], requiring it to adapt to movement in any direction; (2) Sumo confrontation task: two robots compete in a circular arena, with the goal of pushing the opponent out of the arena, requiring it to be able to respond to multi-angle attack and defense.

[0038] Unlike previous benchmarks, the 3DS-MB task has variable directionality and a 3D gravity environment, making it more similar to real robot application scenarios.

[0039] Sixth, systematic experimental verification was carried out.

[0040] We conducted a large number of experiments on Ant (quadruped) and Humanoid (humanoid) robots. The results show that: • In the Ant navigation task, EquiEvo successfully evolved from the atomic torso to a quadruped radial symmetric structure, with significantly higher navigation success rate than the non-symmetry baseline (Evo) and the hard-coded symmetry method (SARD); • In the Humanoid navigation, EquiEvo evolved a more slender and coordinated body structure, with significantly improved stability and flexibility; • In the Sumo confrontation, the EquiEvo robot has a win rate of up to 60%, far exceeding fixed form or non-symmetry methods; • Even starting from a non-symmetrical initial form (such as a three-legged robot), EquiEvo can still evolve high-performance structures through anisotropy guidance, proving its robustness to initial conditions.

[0041] Seventh, the necessity and universality of anisotropy were verified.

[0042] Through ablation experiments, we found that: • If only anisotropy is introduced in the Actor or Critic, the performance will decrease significantly, indicating that both need to be designed collaboratively; • After removing the LRF projection, the strategy's performance in a rotating environment deteriorates dramatically; • Compared with manually designed LRF (such as based on target direction or heading angle), the learnable LRF of the present application has stronger generalization ability; • When the task symmetry changes (such as adding forward reward), EquiEvo can automatically evolve forward and backward asymmetric structures, proving its ability to flexibly adjust its form according to task requirements, rather than being forced to be symmetric.

[0043] Eighth, the engineering applicability of the method was clarified.

[0044] The present application only relies on a standard graph neural network and a PPO component, without complex evolutionary algorithms or artificially designed symmetry constraints, and is easy to integrate into existing robot learning frameworks. The output form has a clear physical meaning (such as limb length, joint torque limit), which can be directly used for hardware design and control deployment of real reconfigurable robots (such as Unitree Go series).

[0045] The present application realizes the unification of geometric perception, efficient training and task adaptation in the 3D morphology-behavior co-evolution task through sub-equi-variant graph representation, dynamic LRF and three-stage collaborative evolution. Experiments show that this method not only improves sample efficiency and generalization ability, but also evolves a more reasonable and robust robot morphology, providing a practical technical path for the autonomous design of embodied agents.

[0046] Embodiment 1: Quadruped robot morphology-behavior co-evolution method based on sub-equi-variant graph neural network Figure 1 The flowchart of the robot morphology-behavior co-evolution method based on the sub-equi-variant graph neural network provided by the embodiments of the present application is shown.

[0047] As shown in Figure 1 The robot morphology-behavior co-evolution method based on the sub-equi-variant graph neural network provided by the present embodiment includes the following steps: Step 101: Construct a 3DS-MB simulation environment and initialize the robot morphology Specifically, a 3D navigation task environment is constructed in the MuJoCo physical simulation engine, named 3DS-MB (Subequivariant Morphology-Behavior Co-Evolution in 3D Environments). As shown in FIG. 3(b), the environment includes a quadruped robot (Ant) and a random target point, which is located in a circular area with a radius of 3 to 4 meters centered on the initial position of the robot. After the robot reaches the target each time, a new random target is automatically generated, requiring the robot to continuously adapt to the new direction.

[0048] The initial morphology of the robot is atomic morphology, which only includes a central torso node without limb structure. The morphology is represented as a topological graph, where the node set contains only one torso node, the edge set is empty, and the attribute matrix contains information such as torso mass, size, joint torque limit, etc.

[0049] Step 102: Perform a three-stage collaborative evolution process Specifically, in each training episode, three stages of structure transformation, attribute adjustment, and behavior control are sequentially executed, with a total step length of 1000 steps.

[0050] (1) Structure transformation stage (step 1021) The first 5 steps are continued. In this stage, the robot does not interact with the environment, and only decides whether to add or delete limbs and joints through the first graph neural network. The action space is discrete, including adding limbs, deleting limbs, etc. After each step is executed, the nodes and edges of the topological graph are updated. There is no environmental reward in this stage, and it is only used to change the morphological structure.

[0051] (2) Attribute adjustment stage (step 1022) The 6th to 10th steps are continued. In this stage, the physical attributes of each limb are adjusted through the second graph neural network, including limb length (range 0.1 to 0.8 meters), mass (range 0.5 to 2.0 kilograms), and joint torque upper limit (range 0.1 to 1.0 Newton-meters). The action space is continuous. This stage also has no environmental reward.

[0052] (3) Behavior control stage (step 1023) The 11th to 1000th steps are continued. In this stage, the robot interacts with the environment based on the evolved morphology. The state is represented as a sub-isomorphic graph, which contains scalar states (such as joint angles, angular velocities) and vector states (such as positions, linear velocities, angular velocities), and explicitly introduces a gravity direction vector as a reference.

[0053] Step 103: Construct a local reference frame (LRF) and project the state Specifically, at each step of the behavior control stage, the local reference frame (LRF) is predicted through the sub-isomorphic graph neural network, and the vector state is projected from the world coordinate system to the geometric invariant space. This process corresponds to Figure 2 The core logic of the LRF and Canonicalization modules in the middle: (1) LRF prediction: The sub-isomorphic graph neural network contains 5 layers of message passing, as shown in Figure 2 Each layer first calculates the edge features (including the relative position between nodes, the difference in vector features, and the average of scalar features), and then generates edge messages through sub-isomorphic functions (such as rotation-invariant inner product operations, translation-invariant relative distance calculations). Finally, the output vector of the root node (trunk) (dimension 128) is mapped to two candidate vectors (dimension 3) through a learnable weight matrix (dimension 128x6), corresponding to the X-axis and Y-axis candidate directions of the local coordinate system.

[0054] (2) Orthogonalization and projection: The two candidate vectors are orthogonalized with the gravity direction as the third axis to construct a rotation matrix. Then, all vector states (e.g., limb positions, velocities) are projected into the local coordinate system defined by the rotation matrix to obtain a geometric-invariant representation. This process ensures that the policy maintains consistent behavior at any orientation or position.

[0055] Step 104: Perform behavior control and collect environment rewards Specifically, the projected invariant graph is input into a third graph neural network (3-layer graph convolutional network with 300-dimensional hidden layers and ReLU activation), which outputs an 8-dimensional continuous action (corresponding to 8 joint target angles). After the robot performs the action, it interacts with the environment and obtains environment rewards, including: • Success reward: +1000 when reaching the target point; • Distance reward: 10 times the reduction of the distance to the target point per step; • Control cost: penalize large actions, calculated as the sum of the square roots of the absolute values of each action; • Survival reward: +1 for each step of survival.

[0056] At the same time, the termination condition is that the robot's height is not within the range of 0.28 to 0.8 meters, or the state / velocity appears non-finite value (e.g., falling over, flying out).

[0057] Step 105: Joint optimization of three-stage policy Specifically, the Proximal Policy Optimization (PPO) algorithm is used to jointly update the three sub-policies: the optimization logic and Figure 2 The policy update link of the Behavior Control module in the middle is consistent: • Low-level optimization: the behavior control policy is updated based on environment rewards and intrinsic rewards (negative distance to sub-target); • High-level optimization: the structure / attribute policy propagates gradients through the long-term return of the behavior stage; • Value estimation: the independent value network estimates the state value based on the invariant graph, ensuring that the value evaluation is invariant to geometric transformations.

[0058] The training hyperparameters are set as follows: • Batch size: 4096; • PPO update times: 10 times per batch; • Learning rate: 0.0001 for the policy network and 0.001 for the value network; • Discount factor: 0.99, GAE parameter: 0.95; • Total training rounds: 500 rounds (about 1 million environment interaction steps).

[0059] Step 106: Verify the evolution results and performance Specifically, after training, the evolved morphology and strategy are evaluated.

[0060] Specifically, after training, the evolved morphology and strategy are evaluated, and the results are referenced Figure 4 (b) and Figure 5 Comparison of Ant-related: Morphology evolution results: starting from the atomic torso, a radially symmetric quadruped structure is evolved after 500 rounds (such as Figure 5 Right EquiEvoAnt morphology), leg length about 0.65 meters, joint torque upper limit about 0.8 N·m, compact torso (size optimized to 0.35 m x 0.35 m x 0.22 m), compared with Figure 5 Short-legged and thick-trunked morphology of EvoAnt (without sub-isometry method) on the left, more suitable for long-distance navigation movement requirements; Performance comparison: as shown in Figure 4 (b), in the Ant navigation task, the final reward of the present invention (EquiEvoAnt) is about 24000, and the success rate is more than 95%; while the reward of the non-sub-isometric baseline (EvoAnt) is about 18000, and the reward of the hard-coded symmetric method (SARDEvoAnt) is about 15000, it can be seen that the present invention is significantly better than the baseline in reward acquisition and task success rate; Generalization verification: apply the trained strategy to a 90-degree rotated environment (target point fixed on the right side of the initial orientation of the robot), the reward of EquiEvoAnt remains above 23000, and the performance does not decrease significantly, while the reward of EvoAnt decreases to 12000, proving that sub-isometry and LRF projection effectively improve the geometric generalization ability; Robustness verification: train EquiEvoAnt from a three-legged asymmetric initial morphology (artificially remove one hind limb), the final reward reaches 9500, which is 46% higher than EvoAnt (6500), and the evolved quadruped structure can be automatically repaired (such as Figure 5 Symmetric morphology of EquiEvoAnt), proving that the method is robust to initial conditions.

[0061] Embodiment 2: Humanoid robot navigation task This embodiment applies the present invention to a humanoid robot (Humanoid) navigation task, Figure 3 (a) is a schematic diagram of the task of this embodiment, Figure 4 (a) is a comparison diagram of the evolution results of the present invention (including Humanoid navigation task results), Figure 5 is a comparison diagram of the evolved morphology of the present invention (including Humanoid morphology differences).

[0062] • Initial morphology: predefine the humanoid skeleton (21 nodes), skip the structure transformation stage, and only perform attribute adjustment and behavior control; • Task setting: goal points randomly generated, radius 3-4 meters; • Reward function: contains success reward (+1000), forward reward (1.25 times forward displacement), distance reward, control cost, contact cost, survival reward (+5); • Termination condition: height between 1 and 2 meters.

[0063] The experimental results show that EquiEvoHumanoid evolves a slender and coordinated body structure, and the reward is significantly higher than EvoHumanoid and the fixed morphology baseline, proving that isometricity is also effective in evolving and performance results in complex morphology tasks Morphology evolution: as shown in Figure 5 EquiEvoHumanoid (the present application) evolves a slender and coordinated body structure, with arm length optimized to 0.55m (reducing motion inertia), leg length optimized to 0.95m (improving stride), and more uniform trunk mass distribution; compared with Figure 5 the short-legged and thick-armed morphology of the left EvoHumanoid (without isometricity method), the EquiEvoHumanoid has a more stable center of gravity when moving, with a 20% increase in step frequency; Performance comparison: as shown in Figure 4 (a), the final reward of EquiEvoHumanoid is about 30000, while the reward of EvoHumanoid is about 22000, and the reward of the fixed morphology baseline (without attribute adjustment) is about 18000, and the present application has obvious advantages in reward acquisition; in terms of navigation efficiency, EquiEvoHumanoid takes an average of 80 steps to reach the target, which is 27% faster than EvoHumanoid (110 steps); Generalization test: adjust the environmental gravitational acceleration to 0.8g (simulate a low gravity scenario), the reward of EquiEvoHumanoid remains above 28000, while the reward of EvoHumanoid drops to 15000, proving that the present application is more adaptable to changes in environmental physical parameters.

[0064] Example 3: Sumo combat task This embodiment applies the present application to the Ants Sumo combat task.

[0065] • Task setting: as shown in Figure 3 two four-legged robots compete in a circular arena (radius 2.5-4.5 meters), with the goal being to push the other out; • Initial morphology: predefined four-legged structure, only attribute adjustment allowed; • Reward function: win +2000, lose -2000, draw -1000, plus pushing / moving reward; • Training method: Adversarial self-game is adopted, with both sides using the method of this invention. After each round of training, the strategies of both sides are exchanged to simulate the dynamic changes of the opponent's strategy in real confrontation.

[0066] Evolution and performance results Strategy and Form Optimization: EquiEvoAnt evolved a form with long legs (0.7m) and high torque (0.9N·m). Its behavioral strategy is mainly lateral movement and frontal ramming. Compared with the baseline method HIRO (short legs 0.3m, low torque 0.4N·m) frontal collision strategy, EquiEvoAnt's ramming efficiency is improved by 40%. Win rate comparison: such as Figure 4 As shown in (b), in 100 adversarial tests, EquiEvoAnt achieved a win rate of 60%, far exceeding HIRO (41%), EvoAnt (35%) and fixed-form baseline (30%); in terms of adversarial duration, EquiEvoAnt averaged 280 win steps, which was more efficient than HIRO (350 steps); Robustness test: Adding a 0.1m high bump to the arena surface (simulating uneven ground), EquiEvoAnt maintained a 55% win rate, while HIRO's win rate dropped to 32%, demonstrating the superior stability of this invention in complex terrain. The above embodiments only illustrate several implementations of the invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the invention, and these all fall within the protection scope of the invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A robot cooperative evolution method based on sub-equivariant graph neural networks, characterized in that, Includes the following steps: S1, The robot's form is represented as a topological graph, and the robot's state is represented as a sub-isovariable graph. The sub-isovariable graph includes scalar features, vector features, and gravity direction vectors. The vector features include at least one of position, velocity, and angular velocity. The position has been converted into a translation-invariant representation by subtracting the root node position. S2 is executed sequentially within each training cycle: S21, Structural Transformation Stage: Modify the nodes and edges of the topological graph through the first graph neural network to change the robot's morphological structure; S22, Attribute Adjustment Stage: Modify the node attributes of the topology graph through the second graph neural network to adjust the robot limb parameters, including at least one of limb length, mass, and joint torque limit; S23, Behavior control stage: Predict the local reference frame through the sub-equivariant graph neural network, project the sub-equivariant graph into a geometrically invariant graph, and output control actions based on the geometrically invariant graph through the third graph neural network; S3, using the Proximal Policy Optimization (PPO) algorithm, jointly update the first, second, and third graph neural networks based on environmental rewards, and the value function is estimated based on the geometrically invariant graph.

2. The method as described in claim 1, characterized in that, The sub-equivariant graph neural network is configured to satisfy Equivalent to sex change, It consists of a subgroup of Euclidean transformations comprising rotation, reflection, and translation operations that maintain the direction of gravity. The message passing mechanism of the sub-variable graph neural network is configured as follows: calculating edge messages based on the relative positions and features between nodes; aggregating edge messages and updating node states by combining them with the node's own features; all message passing operations satisfy... Isovariability.

3. The method as described in claim 1, characterized in that, The local reference frame is constructed as follows: two candidate vectors are output by the sub-equivariant graph neural network; the two candidate vectors are orthogonalized with the gravity direction vector as the third axis to construct a rotation matrix; the vector features are projected onto the local coordinate system using the rotation matrix to obtain a geometrically invariant representation.

4. The method as described in claim 1, characterized in that, The structure transformation phase and attribute adjustment phase are executed in the early stage of the training cycle, without interacting with the environment or generating environmental rewards; the behavior control phase is executed in the later stage of the training cycle, interacting with the environment and obtaining environmental rewards.

5. The method as described in claim 1, characterized in that, The method is applicable to long-duration tasks in three-dimensional space, including at least one of navigation, adversarial, and obstacle crossing.

6. A system for implementing the method as described in any one of claims 1-5, characterized in that, include: The morphological representation module is used to construct the robot's topology graph and sub-isomorphic graph; the three-stage policy execution module includes a structural transformation sub-network, an attribute adjustment sub-network, and a behavior control sub-network. The joint optimization module is used to update the three sub-networks based on environmental rewards; the execution interface module is used to connect to the simulation environment or the real robot platform.

7. The system as described in claim 6, characterized in that, The behavior control sub-network in the three-stage strategy execution module is configured to receive a geometrically invariant graph projected onto a local reference frame as input and output motor control commands.

8. The system as described in claim 6, characterized in that, The execution interface module supports at least one simulation engine among MuJoCo, Isaac Gym, and PyBullet, or supports a reconfigurable quadruped robot hardware platform.

9. The method as described in claim 1, characterized in that, In the behavior control phase, the control action output by the third graph neural network after the invariant graph is input into the projected graph corresponds to the target angle of the robot joint. The environmental rewards obtained by the robot after executing the action include task success reward, distance-related reward, control cost penalty and survival reward.

10. The method as described in claim 1, characterized in that, The total training cycle step length can be set according to task requirements. The number of execution steps in the structure transformation stage and attribute adjustment stage accounts for a smaller proportion of the total training cycle step length, while the number of execution steps in the behavior control stage accounts for a larger proportion of the total training cycle step length, so as to ensure that the robot fully interacts with the environment and obtains effective rewards for strategy optimization.

Images