A method for generating motion trajectories of coordinated bi-arm hand operation

By generating functional grasping and dual-arm collaborative operation motion trajectories, the problem of scarce data for dual-arm functional grasping and reduced distance during obstacle avoidance is solved, achieving efficient trajectory planning and obstacle avoidance capabilities, and adapting to changes in the shape and size of similar objects.

CN123103682APending Publication Date: 2026-11-17SHANGHAI JIAOTONG UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611560370.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-09
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies suffer from problems such as scarce data for dual-hand grasping, disconnect between grasping and subsequent operations, difficulty in balancing global planning and local control, and reduced distance due to rotation during obstacle avoidance.

Method used

By acquiring the target operation task, target object model, and environmental information, the operation stages are divided, the active hand and functional roles are determined, and the grasping function is generated using contact tags and relative pose priors. The trajectory of the robotic arm is optimized by combining global reference and local constraints, and self-collision avoidance and environmental obstacle avoidance constraints are added to generate the collaborative operation trajectory of the two arms.

Benefits of technology

It achieves high success rate in functional grasping, strong object generalization ability, high obstacle avoidance accuracy, and short trajectory planning generation time. It can generate grasping configurations that conform to the functional area of ​​the object and the division of labor between the two hands, adapt to changes in the shape and size of similar objects, and ensure obstacle avoidance safety in narrow spaces.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123103682A_ABST
    Figure CN123103682A_ABST
Patent Text Reader

Abstract

This invention discloses a method for generating motion trajectories for dual-arm collaborative operations, relating to the field of robot motion planning. The method divides the operation stages based on changes in contact relationships or collaborative modes; acquires contact tags based on human demonstrations and prior knowledge of the relative poses of the two wrists, establishing a correspondence between hand contact areas and robot anchor points; generates functional grasping based on this; converts the object target into a wrist target, generating a sparse global reference; establishes local dual-arm joint velocity optimization, introducing relative Jacobian, joint constraints, self-collision, and environmental obstacle avoidance constraints; synchronizes finger trajectories, and outputs the results after verification. This invention reduces invalid grasping combinations, balances global guidance and local collaborative accuracy, and improves obstacle avoidance safety and trajectory generation success rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of robot motion planning and operation technology, specifically relating to a method for generating motion trajectories for coordinated operation of two arms. Background Technology

[0002] Dual-arm hand systems can perform tasks such as capping, opening, tilting, and transferring objects that are difficult for single-arm systems. However, they have a large number of finger joints, overlapping workspaces between the two arms, and functional divisions, contact coordination, and relative motion constraints between the two hands. Solving for the grasping postures of the two hands separately cannot guarantee that the resulting grasping combination is suitable for subsequent operations. For example, a capping task requires one hand to contact the bottle body for stable support, while the other hand contacts the cap to apply rotation; if both hands grasp the bottle body, even with a stable static grasp, the cap cannot be completed.

[0003] Existing general-purpose grasping planning methods mainly rely on geometric proximity, grasping stability, or force closure indices to generate grasps, typically ignoring the object's manipulative functions and the interdependence between the two hands. While existing single-handed grasping methods can express manipulative intentions using contact areas, when directly extended to both hands, it remains difficult to simultaneously handle contact distribution, relative pose, and collisions between the two hands. Furthermore, acquiring two-handed grasping data individually for different objects, hand shapes, and tasks via teleoperation is costly, and collaborative grasping data is scarce, limiting the batch generation of executable dual-arm manipulative trajectories.

[0004] On the other hand, meeting the functional requirements of the grasping posture does not mean that the robotic arm can perform the corresponding operation. During the processes of capping, opening, and handing over, the two arms need to maintain a specified relative position, relative posture, or relative motion trajectory. Global planning in high-dimensional joint space involves a large amount of computation, and pre-generated trajectories are difficult to respond to environmental changes in a timely manner; relying solely on local reaction control is prone to getting stuck in local stagnation in confined spaces. Obstacle avoidance methods that only limit the approach speed of the nearest point may still result in the actual distance between geometric objects continuing to decrease when the linkage rotates and the nearest point changes accordingly.

[0005] Therefore, there is a need for a trajectory generation method that is operation-oriented, can expand the grasping data of both hands from a small number of demonstrations, and connects the generated grasping data with the staged dual-arm collaborative motion constraints, so that the generation process takes into account the feasible guidance of global reference, local collaborative accuracy, self-collision avoidance and environmental obstacle avoidance. Summary of the Invention

[0006] The purpose of this invention is to provide a method for generating motion trajectories for coordinated bi-arm hand operation, in order to solve problems in the prior art such as scarce data for bi-arm grasping, disconnect between grasping and subsequent operations, difficulty in balancing global planning and local control, and distance reduction caused by rotation during obstacle avoidance.

[0007] The technical solution adopted in this invention is: a method for generating motion trajectories for collaborative operation of two arms, applied to a robot system including a left robotic arm, a right robotic arm, and multi-finger dexterous hands respectively disposed at the ends of the left and right robotic arms, characterized by comprising the following steps: The target operation task, target object model, environmental information and robot state are acquired. Based on the changes in the contact relationship between the hand and the object or the coordination relationship between the two hands, the target operation task is divided into ordered operation stages, and the active hand, functional role and relative motion constraints of each operation stage are determined. Obtain object surface contact tags pre-established based on human function grasping demonstration and relative pose priors of the two wrists in a unified coordinate system. The contact tags at least represent the left and right hand identities and corresponding hand contact areas; establish the correspondence between the hand contact areas and the contact anchor points of the robot's dexterous hand; For the aforementioned operation phase, based on the contact tags and the corresponding relationships, optimization is performed using contact imitation, hand-object penetration, hand-inter-hand penetration, and joint limitation as the basis. In the bimanual collaborative operation phase, a coordination term based on the relative pose prior is added to generate a functional grasp that includes the wrist pose and finger joint configuration in the object coordinate system. If the functional grasp does not meet the functional contact and relative operation requirements of the corresponding phase, the functional grasp is regenerated or the constraints of the corresponding operation phase are adjusted. Based on the pose transformation between the object and the wrist during the grasping process, the stage object target or task target is converted into a wrist target, generating a sparse task space global reference that satisfies the stage boundary and environmental geometric constraints. Based on the global reference and the robot's current state, a local dual-arm joint velocity optimization problem is established. According to the functional roles, single-arm tracking constraints, dual-arm independent tracking constraints, or dual-arm relative motion constraints based on relative Jacobian are selected. Joint constraints, self-collision avoidance, and environmental obstacle avoidance constraints are added to solve the joint velocities and generate the robot arm trajectory. The robotic arm trajectory is synchronously synthesized with the finger transition or hold trajectory corresponding to the stage. The stage contact conditions, relative motion conditions, joint restrictions, collision spacing and stage connection conditions are checked. The final state of the previous stage is used as the initial state of the next stage, and the verified dual-arm hand cooperative operation motion trajectory is output.

[0008] Furthermore, the operation phase is divided into single-handed operation, independent two-handed operation, and coordinated two-handed operation; the functional roles of the coordinated two-handed operation include joint transport that simultaneously constrains absolute task motion and relative motion, a fixed reference that maintains the posture of one hand and constrains the relative motion of the other hand, and a floating relative that only constrains the relative motion of both hands.

[0009] Furthermore, the contact area of ​​each human hand is divided into the palm area, the palmar side area of ​​the knuckles, and the lateral side area of ​​the fingers. Contact candidates are determined based on the signed distance between the vertex of the object surface and the human hand grid, and contact labels are assigned according to the distance between the contact candidates and the reference point set of each hand area. The robot contact anchor point is fixed in the local coordinate system of the corresponding link, and its position is calculated based on the wrist pose, finger joint angle, and link forward kinematics.

[0010] Furthermore, a correspondence is established between the template surface and the target object surface for similar objects with consistent functional structures. The contact labels on the template are transferred to the target object, and it is checked whether the transferred labels are in the corresponding functional area. Based on the transferred labels, multiple robot function grasping is generated and verified to expand the function grasping data.

[0011] Furthermore, the contact imitation is calculated based on the average distance from the robot's contact anchor point to the set of object vertices with corresponding contact tags; the coordination term is calculated based on the translational and rotational differences between the candidate grasped hand relative poses and the reference relative poses obtained from the prior of the relative poses; when the two hands correspond to different objects, the poses of the two wrists are first transformed to a unified coordinate system before the relative poses of the two hands are calculated.

[0012] Furthermore, a friction regularization term is added to the function grabbing optimization to penalize the portion of the tangential force required for the task that exceeds the product of the contact friction coefficient and the non-negative normal support force, and to determine the tangential force requirement according to the stable support or rotation operation role undertaken by both hands.

[0013] Furthermore, a validated object point cloud and function grasping training condition generation model is used, which includes a point cloud feature extractor, a latent variable encoder, and a decoder; multiple function grasping candidates are generated online based on the target object point cloud features and sampled latent variables, and the candidates are screened through coordination, penetration, joint limitation, and stage motion feasibility checks.

[0014] Furthermore, the global reference is optimized on sparse reference points in the coordinate system of the object, end point, or task. The objective function includes a smoothing term and a geometric collision penalty term for adjacent reference points, and the constraints include the start and end points of the stage and the step size limit for adjacent reference points. The joint space executability of the global reference is checked through the local dual-arm joint velocity optimization and collision verification. If it is not satisfied, the reference is replanned or the grasping function is changed.

[0015] Furthermore, the local bi-arm joint velocity optimization problem uses bi-arm joint velocity and tracking relaxation variables as variables, constructs a quadratic programming objective with joint velocity regularization term, tracking relaxation penalty term and operability improvement term, sets a relative tracking relaxation upper limit or a selected relative motion component without relaxation for the coordination stage, and converts joint position constraints into upper and lower bounds of joint velocity according to the control cycle.

[0016] Furthermore, the self-collision avoidance and environmental obstacle avoidance constraints are improved through a velocity damping mechanism: when the minimum distance between the detected geometric pair is between the safe distance and the activation distance, the approach velocity of the nearest point along the normal is limited, and the components of the relative angular velocity of the two geometric bodies in the orthogonal plane of the normal are also limited. The translational and rotational velocity boundaries decrease as the minimum distance approaches the safe distance, and the system switches to separation mode when the minimum distance is not greater than the safe distance. The system maintains the allowed contact set according to the current operation stage, checks the contact tolerance for predetermined hand-object contact, and applies collision avoidance constraints to the remaining geometric pairs. The point velocity of the robot link or the object being held is represented as a linear mapping of the joint velocity. The measured velocity of the dynamic obstacle is taken as a known term. The velocities of the robot geometry on both sides are simultaneously included in the self-collision geometry of the two arms. After integrating the local solution results, the geometric distance, joint constraints and relative pose errors are checked. Collision checks are performed between adjacent trajectory points, either by densification or continuous collision. When the local solution is infeasible, times out or fails the check, the stage progress is paused, feasible deceleration or retreat processing is performed, and the current state is replanned or the grab candidate is switched. No motion trajectory points that have not been checked are output.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention uses the functional area of ​​the object and the roles of both hands as a common basis, connecting the prior knowledge of human contact, the robot's functional grasping, and the motion constraints of the two arms. The functional grasping result not only provides the joint angle of the dexterous hand, but also the wrist pose in the object coordinate system, for subsequent planning and calculation of the end target; the relative motion requirements in the stage description simultaneously constrain the grasping screening and trajectory generation, reducing the situation of "grasping but not being able to operate" and "the hand pose is feasible but the robotic arm cannot perform".

[0018] 2. For objects of the same type and with consistent function and structure, this invention transfers contact tags through the correspondence of template surfaces, optimizes and generates new grasping pairs on the robot hand model, and verifies them, thus expanding the demonstration scope to a small number of objects. Furthermore, by using the generated data to train a conditional generation model, multiple grasping candidates can be quickly proposed after inputting the target object's point cloud; the final candidates are still subject to physical constraints and subsequent motion feasibility checks.

[0019] 3. This invention reduces the number of variables in long-term optimization by planning on sparse reference points in the coordinate system of the object, end effector, or task using a global module; local modules handle relative motion constraints, mechanical limits, and collision avoidance in the unified optimization of the joint speeds of both arms, and adjust the arm configuration using redundant degrees of freedom. Collisions between the two arms are also considered uniformly during the independent operation phase of both arms.

[0020] 4. By improving velocity damping, this invention simultaneously limits the nearest point normal approach velocity and related relative rotation components when the geometry approaches, suppressing the nearest point migration and distance reduction caused by rotation, thereby improving obstacle avoidance safety in narrow spaces.

[0021] 5. The embodiments of the present invention also underwent the following performance verifications: 1) Functional grasping success rate: In the four types of tasks, namely opening the lid, tilting, hand-to-hand transfer and screwing the lid, the average success rate of functional grasping reaches 83.0%, which can generate grasping configurations that conform to the functional areas of the object and the division of labor of the hands.

[0022] 2) Object generalization ability: Among 300 unseen bottles, 273 objects were successfully captured, with an object generalization coverage rate of 91.0%, which can adapt to changes in the shape and size of similar objects.

[0023] 3) Obstacle avoidance accuracy: In the static confined space experiment, a 5mm safety distance setting was used to achieve collision-free movement, demonstrating the obstacle avoidance capability at millimeter-level spacing; this value is the safety distance setting, not the distance control error.

[0024] 4) Success rate of double-arm hand operation trajectory: Among the four types of tasks, opening the lid, tilting, hand ...

[0025] 5) Trajectory planning and generation time: In existing experiments, the global reference trajectory generation time was 0.67s, the local control was running at 100Hz, and the function capture network inference time was 0.008s. The total time for generating the complete arm and hand trajectory needs to be measured separately in a unified system. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the overall process logic of the present invention.

[0027] Figure 2 This is a flowchart illustrating the functional prior establishment, data expansion, and data generation process of the present invention, addressing the main problems.

[0028] Figure 3 This is a flowchart illustrating the global reference planning, local collaborative control, and collision avoidance verification process for secondary issues in this invention.

[0029] Figure 4 This is a schematic diagram illustrating the task phase division and constraint mode selection based on contact relationships and collaborative constraints according to the present invention.

[0030] Figure 5 This is a schematic diagram showing the correspondence between the functional contact areas of the palm and fingers of the human hand and the robot anchor points according to the present invention.

[0031] Figure 6 This is a schematic diagram illustrating the migration of the template contact label to similar objects and the inspection of functional areas according to the present invention.

[0032] Figure 7 This is a schematic diagram illustrating the present invention of training a conditional generation model based on verified crawling data and generating crawling candidates online.

[0033] Figure 8 This is a schematic diagram illustrating the conversion from grasping pose to wrist reference and the absolute and relative motion constraints of the two arms in this invention. Figure 8 (a) A schematic diagram of the conversion of the grasping posture to the wrist reference and the absolute and relative motion constraints of the two arms during the handover preparation of the present invention; Figure 8 (b) Schematic diagram of the conversion of the grasping pose to the wrist reference and the absolute and relative motion constraints of the two arms during cooperative approach.

[0034] Figure 9 This is a schematic diagram of the improved velocity damping that combines the closest point translation restriction and the relative rotation restriction of the present invention.

[0035] Figure 10 This is a schematic diagram of the operation at each stage when the present invention is used for the task of separating the cap.

[0036] In the diagram: 201 is the palm-side contact area; 202 is the finger-side contact area; 203 is the robot contact anchor point; 301 is the left robotic arm; 302 is the right robotic arm; 303 is the left dexterous hand; 304 is the right dexterous hand; 305 is the environmental obstacle; 401 is the bottle body; 402 is the bottle cap; 403 is the operation rotation axis; A and B represent the geometrically closest point, n represents the normal of the closest point, and d represents the geometrically minimum distance. Detailed Implementation

[0037] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0038] This invention comprises two core implementation aspects: functional grasping gesture planning and generation (Example 1) and dual-arm cooperative motion planning (Example 2). Examples 1 and 2 can be implemented independently or in combination. Example 3 provides a system-level implementation that integrates the two core implementation aspects, including Examples 1 and 2. The robot configuration, object type, and parameter settings in the following examples can be adjusted according to specific applications.

[0039] Example 1

[0040] like Figure 2 As shown, this embodiment sequentially completes the establishment of functional contact priors, contact transfer with similar objects, grasping optimization and verification, and can use the generated data to train a fast grasping generation model. The grasping gestures include the wrist pose relative to the manipulated object and the joint configuration of the dexterous hand.

[0041] 1) Collect human body demonstrations and establish functional contact representations.

[0042] Select a representative object corresponding to the target task, collect demonstrations of human hands completing pre-grasping, grasping and collaborative operation, record the left and right wrist poses, finger configurations and object poses, and reconstruct the human hand mesh.

[0043] like Figure 5 As shown, the human hand is divided into areas such as the palm, the palmar side of the knuckles, and the lateral side of the fingers. The palmar contact area 201 and the lateral contact area 202 are used to distinguish different contact methods. This embodiment uses 31 contact areas per hand to express functional requirements such as support, gripping, and lateral force application.

[0044] The contact position is identified by the distance between the object's surface and the human hand's grid, eliminating obviously abnormal penetrations, and a label is assigned based on the distance between the contact position and reference points for each hand area. The label simultaneously records the left and right hand identities and the contact area number, thus describing which hand and which part of the object contacted which area. For example, in a capping task, the operator hand contacts the cap, while the stabilizing hand contacts the bottle body.

[0045] 2) Establish robot contact correspondence and bimanual coordination priors.

[0046] Contact anchor points 203 are set on the corresponding link surfaces of the robot's dexterous hand, corresponding to the palmar and fingertip contact areas of the human hand, respectively. The anchor points update their positions as the robot's wrist and fingers move, thereby converting human contact preferences into a solvable geometric objective for the robot.

[0047] The left and right wrist poses in the same demonstration are transformed to a unified coordinate system, and the relative positions and postures of the hands are extracted. A statistical prior is then established for multiple demonstrations. This prior is used to constrain the spatial coordination of the two hands, reducing grasping combinations where the contact position meets the requirements but subsequent relative operations are difficult to complete.

[0048] 3) Transfer functional touches to similar objects.

[0049] like Figure 6 As shown, for similar objects with identical functional structures but different shapes or sizes, a surface correspondence between the representative object and the target object is established, and the existing contact tags are migrated to the target surface. After migration, it is checked whether the contact tags of the operator's hand and the stabilizing hand are still in the corresponding functional areas; for objects with misaligned contact areas or mismatched functional structures, the correspondence is re-established or a supplementary demonstration is provided. Figure 6 In the diagram, L represents the operator's hand, R represents the stabilizer's hand, and the circle on the bottle indicates the contact label position.

[0050] 4) Jointly optimize the two-handed grasping gesture.

[0051] Using wrist pose and finger joint configuration as optimization variables, the robot's contact anchor point is made close to the target surface with the corresponding label. Unreasonable penetration between the hand and object, between fingers, and between the left and right hands is limited, while considering the range of joint motion. For grasping tasks requiring coordination, the deviation between the relative poses of the two hands and prior coordination is further constrained.

[0052] Multiple initial configurations are used to obtain different grasping candidates. For tasks requiring anti-slip or rotational action, friction regularities can be added based on contact friction conditions and the functional roles of both hands to reduce candidates with insufficient contact support.

[0053] The simulation examines the contact, stability, joint limitations, and relative maneuverability of candidate functional areas, and filters them based on the reachability of both arms. The examined grasps are used as targets for subsequent motion planning and are saved as grasp data.

[0054] 5) Train and use the fast grab generation model.

[0055] like Figure 7 As shown, a conditional generative model is trained using validated object point clouds and grasping data. The model extracts object point cloud features and combines them with latent variables to reconstruct or generate wrist poses and finger configurations, enabling it to propose multiple candidates for the same object.

[0056] In application, the target object point cloud is input into the model, and grasping candidates are generated through different latent variables. Then, penetration, joint limitation, hand coordination, and stage movement feasibility checks are performed. When changing the robot's hand shape, the kinematic model, contact anchor points, and corresponding training data are reconfigured.

[0057] Example 2

[0058] like Figure 3 As shown, the planning process in this embodiment includes global reference generation, local cooperative control, and trajectory verification.

[0059] 1) Determine the phased motion objectives and constraint patterns.

[0060] according to Figure 4 The stage classification shown assigns motion constraints to single-handed operation, independent two-handed operation, and coordinated two-handed operation respectively. The single-handed stage mainly tracks the target of the moving hand; the independent two-handed stage tracks the targets of both hands separately and checks for collisions between the two arms in a unified manner; the coordinated two-handed stage constrains the relative motion of the two hands according to their functional roles.

[0061] When carrying objects together, the overall motion and the relative relationship between the two hands are specified. When operating with a fixed reference, one hand is required to maintain the position of the specified object while the other hand performs the relative operation. When operating with a floating relative position, the relative motion is mainly specified, and the arms are allowed to use the overall position adjustment to meet the requirements of accessibility and obstacle avoidance.

[0062] 2) Convert the motion of an object into a wrist target.

[0063] like Figure 8 As shown, the pose correspondence between the object and the wrist is determined using the grasping results, and the target motion of the object or component is converted into left and right wrist targets. This correspondence remains unchanged during the holding phase, when the grasp has been established and no slippage has occurred.

[0064] When grasping different objects with both hands, first unify the poses of both sides to the robot's base coordinate system, and then determine the relative motion requirements. For operations such as screwing on caps, the motion relationship between the cap and the bottle body should be converted into motion constraints between the wrists, taking into account the grasping positions on both sides.

[0065] 3) Generate a sparse global reference.

[0066] Based on the starting point, ending point, and environmental geometry of the stage, sparse reference points are set in the object or end-task space. By optimizing the position of the reference points and the necessary attitude, the smoothness of the path and obstacle avoidance are taken into account, providing forward guidance for local motion.

[0067] After the global reference is generated, its executability is checked by combining the robot's kinematics and collision model. If the corresponding robotic arm configuration is unreachable or fails to meet the coordination requirements, the reference path is adjusted or the grasping candidate is replaced.

[0068] 4) Solve the arm motion using local quadratic programming.

[0069] In each control cycle, the desired motion speed is determined based on the deviation between the current state and the global reference, with the joint velocities of the left and right robotic arms used as unified optimization variables. The relative Jacobian is used to describe the combined effect of the joint motions of the two arms on the relative motion of the hands, thereby jointly solving for the motion that satisfies the coordination requirements.

[0070] The optimization takes into account joint position and velocity limitations, reference tracking errors, and maneuverability, enabling the robot to adjust its arm configuration using redundant degrees of freedom. For relative motion components that need to be precisely maintained, their permissible deviations are limited; for absolute references that can be appropriately adjusted, deviations within a specified range are allowed to meet obstacle avoidance requirements.

[0071] 5) Implement self-collision and environmental obstacle avoidance.

[0072] like Figure 9 As shown, distance detection is performed on the relevant geometric pairs between the left robotic arm 301, right robotic arm 302, left dexterous hand 303, right dexterous hand 304, and environmental obstacles 305. Pre-defined hand-object contact is checked according to contact tolerance; collision avoidance constraints are applied to the remaining non-contact geometric pairs; the held object is also included in collision detection.

[0073] When two geometric objects enter the set approach range, the approach speed is limited based on the nearest points A and B and their normal n; as the minimum distance d decreases, the speed limit is gradually tightened. At the same time, the relative rotational component that may cause the nearest point to migrate is limited, reducing the risk of continued approach due to link rotation when only constraining translation.

[0074] For collision constraints between the two arms, the motion of the joints on both sides should be taken into account; for dynamic obstacles, their observed velocities should be included in the relative velocity calculation. When the fingers open and close, the effect of finger motion on the velocities of relevant geometric points should also be taken into account.

[0075] 6) Review and update the movement trajectory.

[0076] The obtained joint velocities are integrated over the control cycle, and the updated joint limits, geometric spacing, and relative motion errors are checked. The motion between adjacent trajectory points is then densified or continuously checked for collisions.

[0077] If a local solution is infeasible, times out, or fails verification, pause the phase, slow down or back off within feasible limits, and replan from the current state; replace the grab candidate if necessary. The motion points that pass the inspection are used for execution or trajectory storage.

[0078] Example 3

[0079] In this embodiment, the function grasping generation stage can be implemented in the manner described in Embodiment 1, and the dual-arm collaborative motion planning stage can be implemented in the manner described in Embodiment 2. The two are connected through a unified coordinate definition, robot model, and object identifier.

[0080] 1) Establish a unified system and data interface.

[0081] like Figure 1 As shown, the system includes a task phased module 101, a function contact prior module 102, a function capture and generation module 103, a global reference planning module 104, a local cooperative control module 105, and a trajectory verification and output module 106.

[0082] The task phase module determines the contact relationships, active hands, and functional roles for each phase; the functional contact prior module provides contact tags and relative hand pose priors; the functional grasping generation module outputs wrist pose and finger configuration; the global reference planning module provides motion guidance; the local collaborative control module generates arm movements; and the trajectory verification and output module completes phase transitions, validity checks, and data saving.

[0083] Each module uses a unified coordinate definition, robot model, and object identifier to ensure consistency between the grasping results, stage goals, and motion constraints. Among them, the functional contact prior module 102 and the functional grasping generation module 103 can be established and run as described in Embodiment 1; the global reference planning module 104 and the local cooperative control module 105 can be established and run as described in Embodiment 2.

[0084] 2) Establish a unified stage description.

[0085] like Figure 4 As shown, the complete task is divided into ordered stages based on changes in hand-object contact relationships or bi-hand coordination patterns. Each stage records the operation type, active hand, related objects or components, functional roles of both hands, target contact, motion constraints, and start and end states.

[0086] Within the same phase, the contact relationship and cooperative mode remain consistent, but relative poses are allowed to change as the operation progresses. Approach, closing, holding, releasing, and gripping are each assigned corresponding finger actions and switching conditions. The final state actually reached in the previous phase serves as the starting point for the next phase, avoiding discontinuous states caused by independent planning.

[0087] 3) Generate the complete trajectory of the capping task.

[0088] like Figure 10 As shown, taking the right hand stabilizing the bottle body 401 and the left hand operating the bottle cap 402 as an example, the capping and separation requirements are determined according to the operating rotation axis 403 and the object structure, and Example 1 is called to generate grasping candidates, and Example 2 is called to generate the double arm movements at each stage.

[0089] In the first stage, the right arm approaches the bottle, maintaining a pre-grasping configuration while the left hand waits. In the second stage, the right hand closes to the target grasping configuration; after confirming the bottle grasp is established, the pose relationship between the bottle and the right wrist is recorded. In the third stage, the left arm approaches the bottle cap, the right hand maintains the bottle grasp, and collisions between the arms, hands, and the environment are checked. In the fourth stage, the left hand closes to grasp the bottle cap; after confirming functional contact and initial relative pose meet the requirements, collaborative operation begins. In the fifth stage, the right hand serves as a fixed reference, while the left hand rotates relative to it around a specified axis, with both hands maintaining the corresponding grasping configuration. The required rotation angle and axial movement are determined based on the bottle cap structure; when the thread lead is known, the corresponding axial displacement is included in the motion reference. In the sixth stage, after confirming the unlocking conditions are met, the interaction relationship between the bottle cap and the bottle is updated, generating a cap removal and separation trajectory. The left hand holds the bottle cap, and the right hand holds the bottle; after achieving the separation target, the operation ends or proceeds to the subsequent placement stage.

[0090] 4) Synthesize and save the hand trajectory data of both arms.

[0091] The trajectories of the arm joints are synthesized with the closure, hold, and release trajectories of the fingers along a unified time axis. The finger transitions use smooth interpolation that meets speed limits, and the grasping and holding phases maintain the corresponding configuration; if the transition speed exceeds the limit, the phase duration is adjusted.

[0092] The system checks the start and end states, phase transitions, joint range of motion, collision distances, functional contact, and coordination errors of the complete trajectory. It saves the timestamps of the checks passed, the states of both arm joints, the states of both hand joints, and the phase markers, and can simultaneously save the object pose, contact status, and verification results.

[0093] By changing object instances, initial states, and grasping candidates, the above process is repeated to generate diverse operational data. For tasks such as opening the lid, tilting, and handing over the handle, the lid's motion relationship, the container's relative pose requirements, and the receiving and releasing sequence are configured separately, using the same system and data generation process. The generated results can be used for robot execution verification and subsequent imitation learning training.

[0094] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating motion trajectories for collaborative operation of two robotic arms, applied to a robot system including a left robotic arm, a right robotic arm, and multi-finger dexterous hands respectively disposed at the ends of the left and right robotic arms, characterized in that, Includes the following steps: The target operation task, target object model, environmental information and robot state are acquired. Based on the changes in the contact relationship between the hand and the object or the coordination relationship between the two hands, the target operation task is divided into ordered operation stages, and the active hand, functional role and relative motion constraints of each operation stage are determined. Obtain object surface contact tags pre-established based on human function grasping demonstration and relative pose priors of the two wrists in a unified coordinate system. The contact tags at least represent the left and right hand identities and corresponding hand contact areas; establish the correspondence between the hand contact areas and the contact anchor points of the robot's dexterous hand; For the aforementioned operation phase, based on the contact tags and the corresponding relationships, optimization is performed using contact imitation, hand-object penetration, hand-inter-hand penetration, and joint limitation as the basis. In the bimanual collaborative operation phase, a coordination term based on the relative pose prior is added to generate a functional grasp that includes the wrist pose and finger joint configuration in the object coordinate system. If the functional grasp does not meet the functional contact and relative operation requirements of the corresponding phase, the functional grasp is regenerated or the constraints of the corresponding operation phase are adjusted. Based on the pose transformation between the object and the wrist during the grasping process, the stage object target or task target is converted into a wrist target, generating a sparse task space global reference that satisfies the stage boundary and environmental geometric constraints. Based on the global reference and the robot's current state, a local dual-arm joint velocity optimization problem is established. According to the functional roles, single-arm tracking constraints, dual-arm independent tracking constraints, or dual-arm relative motion constraints based on relative Jacobian are selected. Joint constraints, self-collision avoidance, and environmental obstacle avoidance constraints are added to solve the joint velocities and generate the robot arm trajectory. The robotic arm trajectory is synchronously synthesized with the finger transition or hold trajectory corresponding to the stage. The stage contact conditions, relative motion conditions, joint restrictions, collision spacing and stage connection conditions are checked. The final state of the previous stage is used as the initial state of the next stage, and the verified dual-arm hand cooperative operation motion trajectory is output.

2. The method for generating motion trajectory of dual-arm hand cooperative operation according to claim 1, wherein the operation stage is divided into single-hand operation, independent operation of both hands and cooperative operation of both hands; the functional roles of the cooperative operation of both hands include joint transport that simultaneously constrains absolute task motion and relative motion, a fixed reference that maintains the posture of one hand and constrains the relative motion of the other hand, and a floating relative that only constrains the relative motion of both hands.

3. The method for generating motion trajectories for dual-arm hand cooperative operation according to claim 1, divides the contact area of ​​each hand into a palm region, a knuckle palmar side region, and a finger lateral side region, determines contact candidates based on the signed distance between the vertex of the object surface and the human hand mesh, and assigns contact labels based on the distance between the contact candidates and the reference point set of each hand region; the robot contact anchor point is fixed in the local coordinate system of the corresponding link, and its position is calculated based on the wrist pose, finger joint angle, and link forward kinematics.

4. The method for generating motion trajectories for dual-arm collaborative operation according to claim 3, establishes a correspondence between the template surface and the target object surface for similar objects with consistent functional structures, migrates the contact labels on the template to the target object, and checks whether the migrated labels are in the corresponding functional area; generates and verifies multiple robot function grasping based on the migrated labels to expand the function grasping data.

5. The method for generating motion trajectory of dual-arm hand cooperative operation according to claim 1, wherein the contact imitation is calculated based on the average distance from the robot contact anchor point to the set of object vertices with corresponding contact labels; the coordination term is calculated based on the translation difference and rotation difference between the candidate grasped hand relative pose and the reference relative pose obtained from the prior of the relative pose; when the two hands correspond to different objects, the poses of the two wrists are first transformed to a unified coordinate system before the relative pose of the two hands is calculated.

6. The method for generating motion trajectory of dual-arm hand cooperative operation according to claim 1 further incorporates a friction regularization term in the functional grasping optimization, applies a penalty to the portion of the tangential force required for the task that exceeds the product of the contact friction coefficient and the non-negative normal support force, and determines the tangential force requirement according to the stable support or rotation operation roles undertaken by the two hands respectively.

7. The method for generating motion trajectory of dual-arm hand cooperative operation according to claim 4, utilizes a verified object point cloud and function grasping training condition generation model, the model including a point cloud feature extractor, a latent variable encoder and a decoder; generates multiple function grasping candidates online based on the target object point cloud features and sampled latent variables, and filters the candidates through coordination, penetration, joint limitation and stage motion feasibility checks.

8. The method for generating motion trajectory of dual-arm collaborative operation according to claim 1, wherein the global reference is optimized on sparse reference points in the coordinate system of the object, end point, or task; the objective function includes a smoothing term and a geometric collision penalty term for adjacent reference points; and the constraints include the start and end points of the stage and the step size limit of adjacent reference points. The joint space executability of the global reference is checked through the local dual-arm joint velocity optimization and collision verification. If the requirements are not met, the reference is replanned or the grasping function is changed.

9. The method for generating motion trajectory of dual-arm hand cooperative operation according to claim 1, wherein the local dual-arm joint velocity optimization problem uses the dual-arm joint velocity and tracking relaxation variable as variables, constructs a quadratic programming objective with joint velocity regularization term, tracking relaxation penalty term and operability improvement term, sets a relative tracking relaxation upper limit or a selected relative motion component without relaxation for the cooperative stage, and converts the joint position limit into the upper and lower bounds of joint velocity according to the control cycle.

10. The method for generating motion trajectory of dual-arm hand cooperative operation according to claim 9, wherein the self-collision avoidance and environmental obstacle avoidance constraints are constructed by improving the velocity damping structure: when the minimum distance between the detected geometric pair is between the safe distance and the activation distance, the approach velocity of the nearest point along the normal is limited, and the component of the relative angular velocity of the two geometric bodies in the orthogonal plane of the normal is limited; the translational and rotational velocity boundaries decrease as the minimum distance approaches the safe distance; and when the minimum distance is not greater than the safe distance, the system switches to a separation mode; the allowed contact set is maintained according to the current operation stage, and the contact tolerance is checked for predetermined hand-object contact. Collision avoidance constraints are applied to the remaining geometric pairs. The point velocity of the robot link or the object being held is expressed as a linear mapping of the joint velocity. The measured velocity of the dynamic obstacle is taken as a known term. The velocities of the robot geometry on both sides are simultaneously included in the self-collision geometric pairs of the two arms. After integrating the local solution results, the geometric distance, joint constraints and relative pose errors are checked. Collision checks are performed between adjacent trajectory points, either by densification or continuous collision. When the local solution is infeasible, times out or fails the check, the stage progress is paused, feasible deceleration or retreat processing is performed, and the current state is replanned or the grab candidate is switched. No motion trajectory points that have not been checked are output.