A Neural Network-Based Redirection Trajectory Optimization Method for Robots with Dexterous Hands

Abstract

This invention discloses a method for retargeting trajectory optimization of a robot with a dexterous hand based on a neural network, involving the interdisciplinary fields of robotics, motion control, and artificial intelligence. The method includes the following steps: Step 1: Acquire human demonstration action data, including motion information of the upper limbs and the dexterous hand; Step 2: Use a deep neural network to nonlinearly map the human demonstration action data to the robot's motion space, wherein the robot is equipped with a dexterous hand to achieve retargeting of high-degree-of-freedom movements; Step 3: Optimize the mapped motion trajectory by constructing a loss function and combining it with the robot's dynamic constraints, joint constraints, and motion coupling characteristics to generate a smooth, coherent, and feasible robot motion trajectory; Step 4: Output the optimized trajectory for controlling the robot to perform high-degree-of-freedom tasks.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of robotics, motion control and artificial intelligence, and in particular to a method for reorientation trajectory optimization of a robot with a dexterous hand based on neural networks. Background Technology

[0002] Learning human operational skills is a core direction of artificial intelligence and robotics research. While traditional motion mapping or remote control methods can reproduce basic movements, they struggle to capture the complex movement characteristics of the human hand and fingers. This is especially true in high-degree-of-freedom tasks (such as music conducting, gesture communication, grasping and manipulation), where the coordination and fine control of dexterous hands are crucial for achieving natural and fluid robot behavior.

[0003] Existing technologies mainly rely on geometric mapping or inverse kinematics (IK) solutions, which can only achieve the mapping of human actions to robots in low-dimensional degrees of freedom, and have limited expressive power for the relocalization of high-dimensional finger joint movements of dexterous hands. In addition, although optimization methods for existing technologies can improve trajectory feasibility by constructing cost functions, they have high computational costs, poor real-time performance, and usually do not fully consider the motion coupling and joint constraints of dexterous hands.

[0004] Therefore, those skilled in the art are dedicated to developing a new method for optimizing robot redirection trajectories, enabling robots to achieve natural, coordinated movements that satisfy dynamic constraints in high-degree-of-freedom tasks, thereby overcoming the aforementioned deficiencies in existing technologies. Summary of the Invention

[0005] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to overcome the defects of existing motion relocation and trajectory processing methods, such as stiff motion, lack of continuity, insufficient adaptability, and insufficient consideration of robot dynamics constraints.

[0006] To achieve the above objectives, this invention provides a repositioning trajectory optimization method for robots with dexterous hands based on neural networks. This method can utilize the nonlinear mapping capability of deep neural networks to achieve joint repositioning of human upper limb and dexterous hand movements. Combined with trajectory optimization technology, it enables robots to achieve natural, coordinated movements that satisfy dynamic constraints in high-degree-of-freedom tasks.

[0007] Specifically, this invention provides a method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network, comprising the following steps: Step 1: Obtain human demonstration movement data, including movement information of the upper limbs and dexterous hands; Step 2: Use a deep neural network to nonlinearly map the human demonstration action data to the robot's action space, wherein the robot is equipped with a dexterous hand to achieve the redirection of high-degree-of-freedom actions; Step 3: Optimize the mapped motion trajectory by constructing a loss function and combining it with the robot's dynamic constraints, joint constraints and motion coupling characteristics to generate a smooth, coherent and feasible robot motion trajectory. Step 4: Output the optimized trajectory, which is used to control the robot to perform high-degree-of-freedom tasks.

[0008] Furthermore, the human demonstration motion data in step 1 includes joint angles, posture, and position information, which are acquired through a motion capture system.

[0009] Furthermore, step 2 includes the following sub-steps: Step 2.1: Preprocess the human demonstration action data into a format that the deep neural network can process; Step 2.2: Extract human motion features using an encoder, and compress the low-dimensional human demonstration motion data into a high-dimensional, representative latent feature vector using a multi-layer graph neural network. Step 2.3: Perform feature transformation, nonlinearly mapping the potential feature vectors into the robot's action space; Step 2.4: Use the decoder to generate a series of normalized joint angle sequences for the robot, which constitute the complete motion trajectory of the robot performing the task; Step 2.5: Using the robot's kinematic model, calculate the position and orientation of each joint of the robot's end effector in three-dimensional space based on the joint angle sequence.

[0010] Further, step 2.1, which involves extracting and constructing graph structure data containing topological and geometric information from the original motion capture data, includes the following sub-steps: Step 2.1.1: Load the human demonstration action data and use forward kinematics to calculate the specific positions of all joints of the human body in three-dimensional space in each frame; Step 2.1.2: Using human joints as graph nodes and human skeletal connections as graph edges, construct a kinetic chain topology representing the human skeleton. Step 2.1.3: Represent the human pose of each frame as a graph data object to form the graph structure data, which is then used as the input to the deep neural network.

[0011] Furthermore, the redirection in step 2 jointly optimizes the motion coordination of the arms and hands.

[0012] Furthermore, the dynamic constraints in step 3 include torque limits, acceleration limits, and energy consumption constraints for the robot, and the joint constraints include the range of motion of dexterous finger joints and decoupling coordination mechanisms.

[0013] Furthermore, the loss function in step 3 is used to measure the difference between the robot action generated by the model and the real target, and to guide the optimization of the deep neural network; The loss function is a multinomial loss function, including end-effector position loss, end-effector orientation loss, similarity loss, self-collision loss, joint constraint loss, and finger joint loss. By minimizing the weighted sum of the multinomial loss functions, the mapping between human action features and robot actions is achieved.

[0014] Furthermore, the end-effector position loss encourages the robot to match the end-effector position demonstrated by a human and uses mean squared error to compare the differences in normalized end-effector positions, where the normalization coefficient refers to the actual length from shoulder to wrist; let and For end effector Position and normalization coefficient, and For the corresponding variable demonstrated to humans, then the terminal position loss... Defined as: The end-effector orientation loss measures the deviation between a specific orientation vector of the human end effector and the orientation vector of the corresponding part of the robot. It is calculated by comparing the differences in the end effector orientation and also uses mean square loss. and Let the rotation matrices of the robot and human end effectors be respectively, then the end-effector orientation loss is... for: .

[0015] Furthermore, the similarity loss measures the deviation between the orientation and posture of key human limb segments and the orientation and posture of corresponding robot limb segments. It is calculated by comparing the difference in normalized vectors from the elbow to the wrist, where the normalization coefficient is the actual length from the elbow to the wrist; assuming the wrist... elbow , These are the wrist position, elbow position, and robotic arm. The normalized coefficient, , , Let each be a corresponding variable in the demonstration, then the similarity loss... The calculation is as follows: The self-collision loss is designed to ensure that the robot does not physically interfere with or contact its own components when performing actions. The links of the robotic arm are modeled as capsules, and the distance between capsule pairs is calculated. If the distance is less than the minimum distance for no collision, a loss is calculated. For capsules With capsules The distance between them If the non-collision threshold is met, then the self-collision loss... for: The joint constraint loss ensures that the generated robot joint angles do not exceed the preset physical motion range; The finger joint loss method constructs two direction vectors using the three joint nodes of the finger, normalizes the two direction vectors, and calculates the mean square error between the two direction vectors and the demonstration vector using the input data.

[0016] Furthermore, in step 3, a combination of spline interpolation and Gaussian filtering is used to optimize the continuity and smoothness of the mapped motion trajectory.

[0017] The present invention provides a neural network-based method for optimizing the redirection trajectory of a robot with a dexterous hand, which has at least the following technical advantages: 1. The technical solution provided by this invention improves the accuracy and safety of motion trajectories. By introducing multiple loss functions (including end effector position loss, orientation vector loss, joint constraint loss, collision penalty term, and finger coordination loss) during neural network training, the generated robot trajectory is comprehensively optimized in terms of position accuracy, posture consistency, physical realizability, and operational safety. Compared with traditional methods, the technical solution provided by this invention can significantly reduce self-collision phenomena and joint over-limit problems in the trajectory, enabling the robot to safely and accurately reproduce the rhythmic and artistic movements of human operators. 2. The technical solution provided by this invention enhances the generalization ability for different robots and tasks. It utilizes graph neural networks to model human motion data, effectively capturing the topological structure and geometric relationships of the skeletal system, rather than relying solely on absolute coordinate information. This relative representation method gives the model stronger transferability, enabling seamless generalization among robots of different sizes and kinematic structures. For example, the trained model can transfer the movements of a single-arm robot (such as a robotic arm) to a dual-arm collaborative robot without requiring large-scale retraining, thus significantly improving the system's versatility and flexibility. 3. The technical solution provided by this invention simplifies the data processing flow and improves training efficiency. By directly converting the raw motion capture data into a graph structure and replacing the traditional inverse kinematics solution process with forward kinematics calculation, the data processing steps are greatly simplified. This design not only reduces computational complexity but also automates and simplifies the training process by unifying and integrating all constraints in an end-to-end training framework, providing a solid foundation for large-scale deployment and application. 4. The technical solution provided by this invention improves the robustness and environmental adaptability of motion. By learning the overall motion patterns of human movements, rather than relying on the coordinates of individual joints, the generated trajectory can maintain its core motion intent even when faced with external disturbances or minor deviations in initial position. Furthermore, through forward kinematics inference and graph-based representation, the model can more fully understand the geometric and topological changes in different task scenarios, thus enabling it to stably and reliably complete motion execution tasks even in unstructured or dynamic environments.

[0018] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0019] Figure 1 This is an overall flowchart of a preferred embodiment of the method of the present invention; Figure 2 This is a block diagram of a deep neural network training method according to a preferred embodiment of the present invention. Detailed Implementation

[0020] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0021] This invention provides a method for efficiently and accurately redirecting complex human upper limb and hand movements (such as conducting music, saluting, grasping, etc.) to a robot with two arms and a dexterous hand using a deep neural network, and jointly optimizing the motion trajectories of the robot's arms and dexterous hand. The purpose of this invention is to overcome the shortcomings of existing motion relocation and trajectory processing methods, such as stiff movements, lack of coherence, insufficient adaptability, and inadequate consideration of robot dynamics constraints. It provides a neural network-based robot motion redirection and trajectory generation method that achieves accurate and smooth execution of complex, high-degree-of-freedom trajectories on the robot (including two arms and a dexterous hand), for example, enabling the robot to perform expressive command movements in sync with music.

[0022] First, the embodiments of this invention aim to solve the problems of stiff and inconsistent movements produced by existing methods based on geometric mapping or inverse kinematics. By learning the nonlinear mapping relationship between human and robot movements through a neural network model, the method provided by the embodiments of this invention can generate more natural, fluid, and artistic command movements.

[0023] Secondly, the embodiments of the present invention aim to address the problem of insufficient adaptability of existing methods to different individual differences. By introducing motion data from different commanders for training and normalization, the method provided by the embodiments of the present invention can adapt to different body types, motion ranges, and motion styles, achieving personalized motion mapping.

[0024] Furthermore, this invention aims to address the problem of insufficient consideration of robot dynamics constraints during trajectory processing. This invention optimizes and smooths joint positions, velocities, and accelerations during the trajectory generation stage, ensuring that the robot meets torque and velocity constraints during execution. This improves the executability and stability of the trajectory and guarantees that the generated trajectory remains executable under real-world robot hardware conditions, thereby enhancing the ability to transfer simulation to reality.

[0025] Furthermore, the embodiments of this invention aim to address the problem of insufficient generalization ability in existing methods. By training a neural network model on a large-scale dataset containing a variety of actions, the method provided by the embodiments of this invention significantly improves the generalization ability to previously unseen command actions, enabling robots to adapt to a wide range of different command action inputs.

[0026] Example 1 The specific technical solutions adopted in the embodiments of the present invention are as follows: First, the Xsens high-precision inertial motion capture system is used to collect motion data of the human commander. IMU sensors deployed on key parts of the body and the hands acquire joint angles, postures, and position information in real time, providing accurate raw data for subsequent modeling. In particular, for fine hand operations, this embodiment of the invention further incorporates dexterous hand motion capture data to describe high-dimensional motion features such as finger joint flexion, thumb opposition, and grasping posture.

[0027] Secondly, the collected full-body and dexterous hand motion data are input into a neural network model based on an encoder-decoder structure to map human movements to robot movements, generating executable joint trajectories for the robot, including hand operations. Subsequently, the generated trajectories are validated in a physical simulation environment to detect collisions, assess the smoothness and naturalness of the trajectories, and evaluate the coordination and stability of the robot limbs and dexterous hand during execution. Based on this, post-processing is performed on the trajectories, including smoothing and optimization of velocity and acceleration constraints, to ensure the trajectories are safe and feasible on actual hardware.

[0028] Finally, the optimized trajectory is uploaded to the robot controller containing the dexterous hand system for real-world verification. By comparing the robot's overall movements with human demonstration data, the system's performance in terms of posture fidelity, precision of hand manipulation, and robustness of overall movements is evaluated, thereby achieving a high-precision natural mapping of human whole-body and hand movements to robot movements.

[0029] Specifically, embodiments of the present invention provide a method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network, including the following steps, such as... Figure 1 As shown: Step 1: Obtain human demonstration movement data, including movement information of the upper limbs and dexterous hands; Step 2: Use a deep neural network to nonlinearly map human demonstration action data to the robot's action space. The robot is equipped with a dexterous hand to achieve the redirection of high-degree-of-freedom actions. Step 3: Optimize the mapped motion trajectory by constructing a loss function and combining it with the robot's dynamic constraints, joint constraints and motion coupling characteristics to generate a smooth, coherent and feasible robot motion trajectory. Step 4: Output the optimized trajectory, which is used to control the robot to perform high-degree-of-freedom tasks.

[0030] Specifically, the human demonstration motion data in step 1, including joint angles, posture, and position information, is acquired through a motion capture system.

[0031] Example 2 Based on Example 1, step 2 includes the following sub-steps, such as... Figure 2 As shown: Step 2.1: Preprocess the human demonstration action data into a format that can be processed by deep neural networks; Step 2.2: Use an encoder to extract human motion features, and compress low-dimensional human demonstration motion data into a high-dimensional, representative latent feature vector through a multi-layer graph neural network. Step 2.3: Perform feature transformation, nonlinearly mapping the latent feature vectors into the robot's action space; Step 2.4: Use the decoder to generate a series of normalized joint angle sequences for the robot, which constitute the complete motion trajectory of the robot performing the task; Step 2.5: Using the robot's kinematic model, calculate the position and orientation of each joint of the robot's end effector in three-dimensional space based on the joint angle sequence.

[0032] Specifically, the redirection in step 2 jointly optimizes the motor coordination of the arms and hands.

[0033] Specifically, the dynamic constraints in step 3 include torque limits, acceleration limits, and energy consumption constraints for the robot, while the joint constraints include the range of motion of dexterous finger joints and decoupling coordination mechanisms.

[0034] Specifically, in step 3, motion capture equipment is used to collect data, which is then used to infer the trajectory. To ensure that the generated trajectory can be executed smoothly and safely on actual robot hardware, this embodiment of the invention further performs post-processing on the trajectory output by the neural network. This post-processing stage uses a combination of spline interpolation and Gaussian filtering to optimize the continuity and smoothness of the mapped motion trajectory, reducing abrupt changes in the velocity and acceleration dimensions to meet the dynamic and torque constraints required during robot execution. Through this optimization process, noise and jitter can be effectively suppressed, improving the executability and physical consistency of the trajectory. Finally, the optimized trajectory is applied to robot simulation and actual execution, achieving a high-precision mapping from human motion data to a robot-executable trajectory, completing a stable transition from simulation to reality.

[0035] Example 3 Building upon Example 2, a dedicated data preprocessing workflow was designed to convert human motion data into a format processable by neural networks. This workflow aims to extract and construct a graph structure data containing rich topological and geometric information from the raw motion capture data. Specifically, step 2.1, extracting and constructing graph structure data containing topological and geometric information from the raw motion capture data, includes the following sub-steps: Step 2.1.1: Load human demonstration motion data and use forward kinematics to calculate the specific positions of all human joints in three-dimensional space in each frame; where the human demonstration motion data is the original AMASS dataset, specifically, the SMPLX model parameters are transformed into the specific positions of all human joints in three-dimensional space in each frame. Step 2.1.2: Using human joints as graph nodes and human skeletal connections as graph edges, construct a kinetic chain topology representing the human skeleton, i.e., a skeletal graph; based on this, further calculate and add various node and edge attributes, such as the relative position of the node with respect to the root joint of the pelvis, the length and direction vector of the bone, and the topological distance of each joint to the root joint and key joints (such as the shoulder and elbow). Step 2.1.3: Represent the human pose of each frame as a graph data object with rich features, forming graph structure data, which is then used as input to the deep neural network.

[0036] Example 4 Based on Example 2, the Encoder in step 2.2 extracts human motion features, is responsible for extracting high-level, abstract features from the preprocessed human motion data, and takes the joint angle and position information that changes over time as input, and processes it through a multi-layer graph neural network to compress the low-dimensional human motion data into a high-dimensional, representative latent feature vector.

[0037] In step 2.3, Latent Transform acts as a bridge connecting human and robot actions, further processing and transforming the latent features output by the encoder. Its core function is to map the human action feature vectors into the robot's action space.

[0038] In step 2.4, the decoder is responsible for generating robot motion. It receives the feature vector after Latent Transform processing and uses it as input to generate a series of normalized joint angle sequences for the robot. Then, it obtains the true joint angle sequence through inverse normalization. These sequences constitute the complete motion trajectory of the robot performing the task.

[0039] In step 2.5, Forward Kinematics (joint position generation) is a traditional robotics module. Based on the robot joint angle sequence generated by the decoder, it uses the robot's kinematic model to calculate the position and orientation of each joint of the robot's end effector in three-dimensional space.

[0040] Example 5 Based on Examples 2-4, step 3 constructs a loss function using the positions of joints in three-dimensional space from the input data and the positions of joints in three-dimensional space from the output data. The loss function is used to measure the difference between the robot action generated by the model and the real target, and to guide the optimization of the deep neural network. The loss function is a multinomial loss function, including end-effector position loss, end-effector orientation loss, similarity loss, self-collision loss, joint constraint loss, and finger joint loss. By minimizing the weighted sum of the multinomial loss functions, the mapping between human action features and robot actions is achieved.

[0041] Specifically, the end-effector position loss encourages the robot to match the end-effector position demonstrated by a human and uses mean squared error to compare differences in normalized end-effector positions, where the normalization coefficient refers to the actual length from shoulder to wrist; let and For end effector Position and normalization coefficient, and For the corresponding variable demonstrated to humans, then the terminal position loss... Defined as: End-effector orientation loss measures the deviation between a specific orientation vector of the human end effector and the orientation vector of the corresponding part of the robot. It is calculated by comparing the differences in the end effector orientation and also uses mean square loss. and Let the rotation matrices of the robot and human end effectors be respectively, then the end-effector orientation loss is... for: .

[0042] Specifically, the similarity loss measures the deviation between the orientation and pose of key human limb segments and the orientation and pose of corresponding robot limb segments. It is calculated by comparing the difference in normalized vectors from the elbow to the wrist, where the normalization coefficient is the actual length from the elbow to the wrist. elbow , These are the wrist position, elbow position, and robotic arm. The normalized coefficient, , , Let each be a corresponding variable in the demonstration, then the similarity loss... The calculation is as follows: Specifically, the self-collision loss aims to ensure that the robot does not physically interfere with or contact its own parts when performing actions. The links of the robotic arm are modeled as capsules, and the distance between capsule pairs is calculated. If the distance is less than the minimum distance for no collision, a loss is calculated. For capsules With capsules The distance between them If the non-collision threshold is met, then the self-collision loss is... for: Joint constraint loss ensures that the generated robot joint angles do not exceed the preset physical motion range; The finger joint loss method constructs two direction vectors using the three joint nodes of the finger, normalizes the two direction vectors, and uses the input data to solve the mean square error between the two direction vectors and the demonstration vector.

[0043] After defining the model structure and loss function, this embodiment of the invention enters the network training and validation phase. Specifically, during the training of the neural network, preprocessed human motion map data is used as input, and robot motion data is used as the supervision target. A gradient descent-based optimization algorithm is employed to iteratively update the network parameters. The model achieves a high-precision mapping between human motion features and robot motion by minimizing the weighted sum of multiple loss functions.

[0044] Example 6 This invention also provides a neural network-based system for optimizing the redirection trajectory of a robot with a dexterous hand, comprising: a motion capture data acquisition module, a motion mapping module, a simulation verification trajectory module, and a trajectory post-processing module.

[0045] The motion capture data acquisition module aims to acquire raw motion data from human operations, and can be implemented in two ways: 1) Optical motion capture system: This system uses markers installed on key parts of the human body and multiple high-speed cameras to accurately capture three-dimensional coordinate data, generating joint angle, position, and posture information. This method offers high precision but requires specialized equipment. 2) Inertial motion capture technology: This technology uses inertial measurement units (IMUs) worn on key parts of the human body to capture the three-dimensional motion posture of the human body in real time by measuring angular velocity, acceleration, and geomagnetic field data. This technology does not require cameras or external markers and can be used in any environment, but it may have cumulative errors and is susceptible to magnetic field interference.

[0046] In this embodiment of the invention, the Xsens high-precision inertial motion capture system is used to acquire the commander's motion data. Advantages: accurate data, convenient operation, and less affected by terrain.

[0047] The motion mapping module is the core of this invention, tasked with converting human commands into executable robot action instructions. This process is implemented using a neural network model based on an encoder-decoder architecture, including key steps such as training data preprocessing, feature extraction, feature transformation, and robot motion generation. The specific implementation of the motion mapping module is as described in the neural network-based reorientation trajectory optimization method for robots with dexterous hands included in any of the preceding embodiments.

[0048] Among these, the simulation verification trajectory module is a crucial step, as it verifies the generated trajectory in a physical simulation environment before applying it to a real robot. This verification can: 1) Collision check: Simulate whether the robot collides with itself or objects in the environment while executing its trajectory; 2) Assess smoothness: Visually observe whether the trajectory is smooth and natural.

[0049] The trajectory post-processing module, in this embodiment of the invention, further optimizes and refines the trajectory after obtaining the robot's motion trajectory generated by the neural network, ensuring its smooth and safe execution on actual robot hardware. Specific processing includes trajectory smoothing, velocity and acceleration constraints, etc.

[0050] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A neural network-based robot re-orientation trajectory optimization method with a dexterous hand, characterized by, The method includes the following steps: Step 1: Obtain human demonstration movement data, including movement information of the upper limbs and dexterous hands; Step 2: Use a deep neural network to nonlinearly map the human demonstration action data to the robot's action space, wherein the robot is equipped with a dexterous hand to achieve the redirection of high-degree-of-freedom actions; Step 3: Optimize the mapped motion trajectory by constructing a loss function and combining it with the robot's dynamic constraints, joint constraints and motion coupling characteristics to generate a smooth, coherent and feasible robot motion trajectory. Step 4: Output the optimized trajectory to control the robot to perform high-degree-of-freedom tasks; The loss function in step 3 is used to measure the difference between the robot action generated by the model and the real target, and to guide the optimization of the deep neural network. The loss function is a multinomial loss function, including end-position loss, end-direction loss, similarity loss, self-collision loss, joint constraint loss, and finger joint loss. By minimizing the weighted sum of the multinomial loss functions, the mapping between human action features and robot actions is achieved. The similarity loss measures the deviation between the orientation and posture of key human limb segments and the orientation and posture of corresponding robot limb segments. It is calculated by comparing the difference in normalized vectors from the elbow to the wrist, where the normalization coefficient is the actual length from the elbow to the wrist. Let the wrist be... elbow , These are the wrist position, elbow position, and robotic arm. The normalized coefficient, , , Let each be a corresponding variable in the demonstration, then the similarity loss... The calculation is as follows: The self-collision loss is designed to ensure that the robot does not physically interfere with or contact its own components when performing actions. The links of the robotic arm are modeled as capsules, and the distance between capsule pairs is calculated. If the distance is less than the minimum distance for no collision, a loss is calculated. For capsules With capsules The distance between them If the non-collision threshold is met, then the self-collision loss... for: The joint constraint loss ensures that the generated robot joint angles do not exceed the preset physical motion range; The finger joint loss method constructs two direction vectors using the three joint nodes of the finger, normalizes the two direction vectors, and calculates the mean square error between the two direction vectors and the demonstration vector using the input data.

2. The method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network as described in claim 1, characterized in that, The human demonstration motion data in step 1 includes joint angles, posture, and position information, which are acquired through a motion capture system.

3. The method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network as described in claim 1, characterized in that, Step 2 includes the following sub-steps: Step 2.1: Preprocess the human demonstration action data into a format that the deep neural network can process; Step 2.2: Extract human motion features using an encoder, and compress the low-dimensional human demonstration motion data into a high-dimensional, representative latent feature vector using a multi-layer graph neural network. Step 2.3: Perform feature transformation, nonlinearly mapping the potential feature vectors into the robot's action space; Step 2.4: Use the decoder to generate a series of normalized joint angle sequences for the robot, which constitute the complete motion trajectory of the robot performing the task; Step 2.5: Using the robot's kinematic model, calculate the position and orientation of each joint of the robot's end effector in three-dimensional space based on the joint angle sequence.

4. The method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network as described in claim 3, characterized in that, Step 2.1 extracts and constructs graph structure data containing topological and geometric information from the raw motion capture data, including the following sub-steps: Step 2.1.1: Load the human demonstration action data and use forward kinematics to calculate the specific positions of all joints of the human body in three-dimensional space in each frame; Step 2.1.2: Using human joints as graph nodes and human skeletal connections as graph edges, construct a kinetic chain topology representing the human skeleton. Step 2.1.3: Represent the human pose of each frame as a graph data object to form the graph structure data, which is then used as the input to the deep neural network.

5. The method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network as described in claim 1, characterized in that, The redirection in step 2 optimizes the motor coordination of the arms and hands.

6. The method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network as described in claim 1, characterized in that, The dynamic constraints in step 3 include the robot's torque limit, acceleration limit, and energy consumption constraints, while the joint constraints include the range of motion of dexterous finger joints and decoupling coordination mechanisms.

7. The method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network as described in claim 1, characterized in that, The end-effector position loss encourages the robot to match the end-effector position demonstrated by a human and uses mean squared error to compare differences in normalized end-effector positions, where the normalization coefficient refers to the actual length from shoulder to wrist; let and For end effector Position and normalization coefficient, and For the corresponding variable demonstrated to humans, then the terminal position loss... Defined as: The end-effector orientation loss measures the deviation between a specific orientation vector of the human end effector and the orientation vector of the corresponding part of the robot. It is calculated by comparing the differences in the end effector orientation and also uses mean square loss. and Let the rotation matrices of the robot and human end effectors be respectively, then the end-effector orientation loss is... for: 。 8. The method for reorientation trajectory optimization of a robot with a dexterous hand based on a neural network as described in claim 6, characterized in that, In step 3, a combination of spline interpolation and Gaussian filtering is used to optimize the continuity and smoothness of the mapped motion trajectory.

Images