Motion control transfer methods, systems, and computer-readable storage media for heterogeneous robots
By constructing kinematic mapping relationships and introducing adaptive parameters, the problem of inefficient transfer of motion control strategies between heterogeneous robots was solved. This enabled the transfer of existing strategies from the source robot to target robots with different structures and degrees of freedom, reducing the investment of training resources while ensuring control performance.
Patent Information
- Application Number
- CN202611629909.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-21
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to achieve low-cost, high-quality transfer of motion control strategies between heterogeneous robots, especially when the input and output spaces of the source and target robots are not aligned. Direct parameter updates can easily lead to errors in variable correspondence, and large-scale parameter updates require significant investment in training resources.
By acquiring the pre-trained motion control strategy of the source robot and the ontology description information of the target robot, a kinematic mapping relationship between the joint space and the unified semantic joint space of the motion control strategy is constructed. While keeping the main network parameters unchanged, trainable adaptive parameters are introduced to adaptively compensate for dynamic differences, acquire the state information of the target robot and perform mapping transformation, and finally update the adaptive parameters according to the execution results to obtain a motion control strategy suitable for the target robot.
It enables the transfer of existing strategies from the source robot to target robots with different structures and degrees of freedom, reduces the investment of transfer training resources, ensures the control performance of the transferred strategy, and realizes low-cost, high-quality motion control strategy transfer between heterogeneous robots.
Smart Images

Figure CN123113409A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of intelligent nursing and artificial intelligence technologies, and in particular to motion control transfer methods, systems, and computer-readable storage media for heterogeneous robots. Background Technology
[0002] As humanoid robots are increasingly used in scenarios such as full-body motion imitation, teleoperation, walking, turning, running and jumping, and human-robot collaboration, full-body motion control strategies based on reinforcement learning are gradually becoming an important component of robot control systems. These strategies typically output joint positions, joint velocities, or equivalent control commands based on the robot's own state and reference motion information to achieve full-body coordination, balance maintenance, and reference motion tracking.
[0003] Existing control strategies are typically trained for specific robot bodies. Different robots vary in the number of joint degrees of freedom, joint arrangement, joint axis orientation, hip structure, link dimensions, mass distribution, inertia, actuator response characteristics, and contact characteristics, making it difficult to directly apply motion control strategies trained on the source robot to the target robot. For the reuse of control strategies between different robot bodies, existing technologies typically involve directly loading the existing strategy onto the target robot and then fine-tuning all parameters.
[0004] However, the above approach still has shortcomings. When the input-output spaces of the source robot and the target robot are not aligned, directly updating parameters can easily lead to errors in variable correspondence, making it difficult to guarantee the control quality of the transferred policy. At the same time, updating the parameters of the existing policy on a large scale requires a high investment of training resources, making it difficult to balance transfer cost and transfer quality. Summary of the Invention
[0005] This application proposes a motion control transfer method for heterogeneous robots, aiming to solve the problem that motion control strategies are difficult to transfer between heterogeneous robots in a low-cost and high-quality manner in the prior art.
[0006] This application provides a motion control transfer method for heterogeneous robots, the method including: Obtain the pre-trained motion control strategy corresponding to the source robot, as well as the ontological description information of the target robot; Based on ontology description information, a kinematic mapping relationship is constructed between the joint space of the target robot and the unified semantic joint space of the motion control strategy. While keeping the main network parameters of the motion control strategy unchanged, trainable adaptive parameters are introduced into the motion control strategy. The adaptive parameters are only used to compensate for the dynamic differences between the target robot and the source robot. The state information of the target robot is obtained, and the state information is transformed into a unified semantic joint space according to the kinematic mapping relationship. The motion control strategy with adaptive parameters is then input into the motion control strategy to obtain the target action output by the motion control strategy. According to the inverse transformation relationship corresponding to the kinematic mapping relationship, the target motion is transformed into the joint space of the target robot to obtain the joint control command of the target robot; Based on the results of the target robot executing joint control commands, only the adaptive parameters in the motion control strategy are updated to obtain a target motion control strategy suitable for the target robot.
[0007] Accordingly, this application also provides a motion control transfer system for heterogeneous robots, including: The source strategy acquisition module is used to acquire the pre-trained motion control strategy on the source robot. The target robot ontology description module is used to obtain ontology description information of the target robot. The observation layout alignment module is used to align the state information of the target robot according to the observation layout definition of the motion control strategy. The joint-level kinematic alignment module is used to construct the kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space of the motion control strategy based on ontology description information. The dynamics adaptive module is used to introduce trainable adaptive parameters into the motion control strategy while keeping the main network parameters of the motion control strategy unchanged. The adaptive parameters are only used to compensate for the dynamic differences between the target robot and the source robot. The reinforcement learning optimization module is used to update only the adaptive parameters in the motion control strategy based on the result of the target robot executing joint control commands, so as to obtain a target motion control strategy suitable for the target robot. The motion execution and safety control module is used to convert the target motion output by the motion control strategy into the joint space of the target robot according to the inverse transformation relationship corresponding to the kinematic mapping relationship, so as to obtain the joint control command of the target robot.
[0008] Accordingly, this application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the above-described motion control migration method for heterogeneous robots.
[0009] This application obtains a pre-trained motion control strategy for the source robot and ontology description information of the target robot. Based on the ontology description information, it constructs a kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space of the motion control strategy. While keeping the main network parameters of the motion control strategy unchanged, it introduces trainable adaptive parameters into the motion control strategy, which are only used to compensate for the dynamic differences between the target robot and the source robot. It obtains the state information of the target robot, transforms the state information to the unified semantic joint space according to the kinematic mapping relationship, and inputs it into the motion control strategy with introduced adaptive parameters to obtain the target action output by the motion control strategy. According to the inverse transformation relationship corresponding to the kinematic mapping relationship, it transforms the target action to the joint space of the target robot to obtain the joint control commands of the target robot. Based on the result of the target robot executing the joint control commands, it only updates the adaptive parameters in the motion control strategy to obtain a target motion control strategy suitable for the target robot. This scheme can achieve migration from the existing strategy of the source robot to target robots with different structures and degrees of freedom, reduce the investment of migration training resources, ensure the control performance of the migrated strategy, and achieve low-cost, high-quality migration of motion control strategies between heterogeneous robots. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] in: Figure 1 This is a schematic diagram of a motion control migration system for heterogeneous robots provided in an embodiment of this application.
[0012] Figure 2 This is a flowchart illustrating a motion control transfer method for heterogeneous robots, provided as an embodiment of this application.
[0013] Figure 3 This is a schematic diagram of joint-level kinematic alignment and motion inverse transformation provided in an embodiment of this application.
[0014] Figure 4 This is a schematic diagram of dynamic adaptation provided for an embodiment of this application.
[0015] Figure 5 This is a schematic diagram illustrating the inverse transformation of actions provided in an embodiment of this application.
[0016] Figure 6This is a schematic diagram of the overall motion control migration process for heterogeneous robots provided in an embodiment of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0018] This application provides a motion control migration method, system, and computer-readable storage medium for heterogeneous robots. Specifically, the motion control migration method for heterogeneous robots can be executed by an electronic device, which can be a terminal or a server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms.
[0019] To address the aforementioned issues, this application provides a motion control transfer method, system, and computer-readable storage medium for heterogeneous robots. This method enables the transfer of motion control strategies from the source robot to target robots with different structures and degrees of freedom, based on the existing strategies of the source robot. It reduces the investment of transfer training resources, ensures the control performance of the transferred strategies, and achieves low-cost, high-quality transfer of motion control strategies between heterogeneous robots.
[0020] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.
[0021] This application provides a motion control migration system for heterogeneous robots. Please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of a motion control migration system for heterogeneous robots provided in an embodiment of this application.
[0022] like Figure 1 As shown, the system includes a source policy acquisition module, a target robot ontology description module, an observation layout alignment module, a joint-level kinematic alignment module, a dynamics adaptation module, a reinforcement learning optimization module, and a motion execution and safety control module. The functions of each module are as follows: The source strategy acquisition module is used to acquire the pre-trained motion control strategy on the source robot.
[0023] Specifically, the source policy acquisition module can read the motion control policy that has been trained on the source robot and save information such as policy network parameters, input and output definitions, state normalization parameters, and motion scaling parameters. The motion control policy can be a multilayer perceptron, a temporal network, a Transformer, or other neural networks that can generate control actions based on the robot's state and reference motion information.
[0024] The target robot ontology description module is used to obtain ontology description information of the target robot.
[0025] The ontology description information may include at least one of the following: the number of joints of the target robot, joint names and sequence, joint axis directions, coordinate system conventions, link dimensions, mass, moment of inertia, actuator parameters, and constraint information of parallel or closed-loop mechanisms. The target robot ontology description module can generate target robot ontology description data based on the acquired information.
[0026] The observation layout alignment module is used to align the state information of the target robot according to the observation layout definition of the motion control strategy.
[0027] The target robot's state information may include at least one of the following: base state, posture-related quantities, joint positions, joint velocities, historical actions, and reference motion information. The observation layout definition includes the order of input variables and input semantic conventions used by the motion control strategy during the training phase. The observation layout alignment module can rearrange the state information according to the order of input variables, ensuring that the inputs received by the motion control strategy are semantically consistent with those received during the training phase of the motion control strategy.
[0028] The joint-level kinematic alignment module is used to construct the kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space of the motion control strategy based on ontology description information.
[0029] Specifically, the joint-level kinematic alignment module can be used to handle at least one of the following differences between the target robot and the source robot: differences in the number of degrees of freedom, joint arrangement, hip axis system, and parallel or closed-loop mechanisms. The joint-level kinematic alignment module can establish joint correspondences between the joints of the target robot and unified joints in a unified semantic joint space, determine the axis decoupling relationship between the joint axis system of the target robot and the unified semantic joint space, and determine the mechanism correction relationship between the mechanism constraints of the target robot and the unified semantic joint space. Based on the joint correspondences, axis decoupling relationships, and mechanism correction relationships, it constructs kinematic mapping relationships.
[0030] The dynamics adaptive module is used to introduce trainable adaptive parameters into the motion control strategy while keeping the main network parameters of the motion control strategy unchanged. The adaptive parameters are only used to compensate for the dynamic differences between the target robot and the source robot.
[0031] Specifically, the dynamics adaptation module can determine dynamics-related network layers within the motion control strategy. These dynamics-related network layers include at least one of the following: an ontology state input projection layer, a dynamics-sensitive layer in the strategy backbone, an action output-related layer, and a value network. While maintaining the original weight matrix of the motion control strategy, the dynamics adaptation module can introduce low-rank trainable parameters into the selected dynamics-related network layers. The rank of these low-rank trainable parameters is less than the input and output dimensions of the introduced network layers. The reference motion encoding module related to the motion task objective within the motion control strategy remains frozen.
[0032] The reinforcement learning optimization module is used to update only the adaptive parameters in the motion control strategy based on the result of the target robot executing joint control commands, so as to obtain a target motion control strategy suitable for the target robot.
[0033] Specifically, the reinforcement learning optimization module can calculate evaluation metrics in the simulation environment and / or the closed-loop environment of the physical robot corresponding to the target robot, and update the adaptive parameters in the dynamics adaptation module only based on the evaluation metrics. Evaluation metrics may include at least one of tracking evaluation metrics, balance evaluation metrics, posture stability evaluation metrics, and control smoothness evaluation metrics. The reinforcement learning optimization module can execute joint control commands and collect the target robot's operating state in the simulation environment and / or the closed-loop environment of the physical robot, construct an optimization objective based on the operating state, and update the adaptive parameters in the motion control strategy only based on the optimization objective while keeping the main network parameters of the motion control strategy unchanged.
[0034] The motion execution and safety control module is used to convert the target motion output by the motion control strategy into the joint space of the target robot according to the inverse transformation relationship corresponding to the kinematic mapping relationship, so as to obtain the joint control command of the target robot.
[0035] The motion execution and safety control module can map the target motion to the joint space of the target robot according to the joint correspondence; perform the inverse transformation of the axis system on the mapped target motion according to the inverse transformation relationship corresponding to the axis system decoupling relationship; and perform the inverse correction of the mechanism on the target motion after the inverse transformation of the axis system according to the inverse correction relationship corresponding to the mechanism correction relationship, so as to obtain the joint control commands of the target robot.
[0036] The motion execution and safety control module can also be used to convert the adapted motion into the actual joint control quantity of the target robot, and constrain the output according to at least one of joint limit, speed limit, torque limit and safety threshold.
[0037] The above system can operate as follows: The source policy acquisition module acquires the pre-trained motion control policy on the source robot; the target robot ontology description module acquires the ontology description information of the target robot; the joint-level kinematic alignment module constructs a kinematic mapping relationship based on the ontology description information; the dynamics adaptation module introduces trainable adaptive parameters into the motion control policy while keeping the main network parameters of the motion control policy unchanged; the observation layout alignment module aligns the state information of the target robot; the joint-level kinematic alignment module transforms the state information into a unified semantic joint space and inputs the motion control policy with introduced adaptive parameters to obtain the target action; the action execution and safety control module transforms the target action into the joint space of the target robot according to the inverse transformation relationship to obtain the joint control command; the reinforcement learning optimization module updates only the adaptive parameters in the motion control policy based on the result of the target robot executing the joint control command to obtain the target motion control policy suitable for the target robot.
[0038] For further details, please refer to Figure 2 , Figure 2 This is a flowchart illustrating a motion control transfer method for heterogeneous robots, provided as an embodiment of this application. The specific flow of this motion control transfer method for heterogeneous robots is as follows: 101. Obtain the pre-trained motion control strategy corresponding to the source robot, as well as the ontological description information of the target robot.
[0039] In this application, the pre-trained motion control strategy corresponding to the source robot refers to a motion control strategy that has been trained on the source robot and is capable of generating control actions based on the state of the source robot and reference motion information. This motion control strategy has been trained on the source robot and possesses capabilities such as whole-body coordination, balance maintenance, and reference motion tracking, and is used as the source strategy to be transferred during the transfer process.
[0040] When acquiring a motion control strategy, the definitions of its input and output variables, joint sequence, and range of motion can be determined. Input variables may include the robot's own state and reference motion information, while output variables may include joint positions, joint velocities, or equivalent control commands. The joint sequence is used to determine the order in which the motion control strategy is arranged during input and output. The range of motion is used to determine the range of values for the output motion, so that it can be scaled or limited when executed on the target robot.
[0041] In some embodiments, the motion control strategy may be a multilayer perceptron, a temporal network, a Transformer, or other neural networks that can generate control actions based on the robot's state and reference motion information. Information such as the strategy network parameters, input and output definitions, state normalization parameters, and action scaling parameters can be acquired and saved for subsequent migration.
[0042] The ontological description information of the target robot refers to the information used to describe the ontological structure and hardware characteristics of the target robot. It is used to determine the ontological differences between the target robot and the source robot, and to provide a basis for subsequent construction of kinematic mapping relationships and introduction of adaptive parameters.
[0043] In some embodiments, the ontology description information may include at least one of the following: the number of degrees of freedom of the target robot, joint mapping relationships, joint axis orientation, mechanism type, mass and inertia, and actuator parameters.
[0044] Specifically, the number of degrees of freedom is used to determine the difference in the number of degrees of freedom between the target robot and the source robot; the joint mapping relationship is used to determine the position of each joint of the target robot in the unified semantic joint space; the joint axis direction is used to determine the difference in the axis system between the target robot and the source robot; the mechanism form is used to determine whether the target robot adopts a parallel or closed-loop mechanism, and whether mechanism modification is required; mass and inertia are used to determine the dynamic difference between the target robot and the source robot; and the actuator parameters are used to determine the dynamic difference of the actuators.
[0045] 102. Based on ontology description information, construct the kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space of the motion control strategy.
[0046] The unified semantic joint space refers to the joint representation space used by the motion control strategy during the training phase. When the motion control strategy is trained on the source robot, its inputs and outputs are defined according to the unified semantic joint space. Therefore, the unified semantic joint space has unified joint semantics corresponding to the input and output dimensions of the motion control strategy.
[0047] The unified semantic joint space has the following characteristics: semantic uniformity, joints with the same function on different robots have the same position and meaning in the unified semantic joint space, ensuring that the input and output actions received by the motion control strategy are semantically consistent; fixed dimension, the dimension of the unified semantic joint space corresponds to the input and output dimension of the motion control strategy and does not change due to the different number of degrees of freedom of the target robot; decoupled from the specific robot body, the unified semantic joint space is not a direct representation of the actual joint space of a certain robot, but a representation space formed after semantic unification of the joints of the source robot and the target robot.
[0048] For example, a unified semantic joint space can be constructed as follows: Obtain the joint definitions used by the source robot during the training phase, including the number of joints, joint names, joint functional semantics, and joint arrangement order; uniformly name and sort the joints of the source robot according to their functional semantics, forming a unified joint sequence of dimension T; for each joint possessed by the source robot, set the corresponding unified joint position in the unified semantic joint space; for joints possessed by the target robot but not by the source robot, do not set corresponding positions in the unified semantic joint space or discard them during mapping; for joints missing by the target robot but possessed by the source robot, pad with zeros at the corresponding positions during mapping. The unified semantic joint space constructed in this way is consistent with the input-output dimensions of the source robot during the training phase and serves as the benchmark for the kinematic mapping of the target robot. In another implementation, a standard semantic joint system can be predefined manually (e.g., unified joints defined according to the semantics of the left and right hips, knees, and ankles of the human body), and then the joints of the source robot during the training phase can be aligned according to this standard system, so that the input and output of the motion control strategy are represented according to this standard system.
[0049] In some embodiments, the step "constructing a kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space of the motion control strategy based on ontology description information" may include the following operations: Based on the ontology description information, establish the joint correspondence between the joints of the target robot and the unified joints in the unified semantic joint space; Based on the target robot's body structure, determine the axis decoupling relationship between the target robot's joint axis system and the unified semantic joint space, as well as the mechanism correction relationship between the target robot's mechanism constraints and the unified semantic joint space. The axis decoupling relationship is used to compensate for at least one of the following: differences in joint axis direction, joint axis tilt angle, and mirror coordinate convention between the target robot and the source robot. The mechanism correction relationship is used to compensate for inconsistencies between execution variables and kinematic joint variables in the target robot's parallel or closed-loop mechanisms. Based on joint correspondence, axis decoupling, and mechanism correction, a kinematic mapping relationship is constructed between the joint space of the target robot and the unified semantic joint space.
[0050] In some embodiments, the step "establishing a joint correspondence between the joints of the target robot and the unified joints in the unified semantic joint space based on the ontology description information" may include the following operations: The joints of the target robot are mapped to the unified joints in the unified semantic joint space to determine the position of each joint of the target robot in the unified semantic joint space. Based on the differences in joint arrangement between the target robot and the source robot, the joint order of the target robot is adjusted so that the adjusted joint order is consistent with the unified joint order in the unified semantic joint space. For a common joint that is missing in the target robot but present in the source robot, zeros are padded at the corresponding position in the common semantic joint space. For redundant degrees of freedom that the target robot has but the source robot does not, these redundant degrees of freedom are discarded from the unified semantic joint space.
[0051] The target robot has multiple degrees of freedom, and its joints may differ from those of the source robot in name, function, or arrangement. A unified semantic joint space defines multiple unified joints, each corresponding to a joint function or semantic meaning. When establishing joint correspondences, the joints of the target robot can be matched with the unified joints in the unified semantic joint space according to their function or semantic meaning, thereby determining the position of each joint of the target robot within the unified semantic joint space.
[0052] For example, the left hip pitch joint of the target robot can correspond to the left hip pitch unified joint in the unified semantic joint space, and the right knee flexion-extension joint of the target robot can correspond to the right knee flexion-extension unified joint in the unified semantic joint space. Through this correspondence, the joint states of the target robot can be arranged according to their positions in the unified semantic joint space.
[0053] In one implementation, the joint correspondence described above can be achieved using a sparse joint mapping matrix.
[0054] For example, suppose the target robot has N control degrees of freedom, and its joint state vector is denoted as... The motion control strategy uses a unified semantic joint space, denoted by dimension T, and the corresponding state vector is denoted as... When establishing joint correspondences, the positions of each joint of the target robot in the unified semantic joint space can be determined. Based on the differences in joint arrangement between the target robot and the source robot, the joint order of the target robot can be adjusted so that the adjusted joint order is consistent with the unified joint order in the unified semantic joint space.
[0055] The sparse joint mapping matrix is used to represent the joint correspondence between the target joint space and the unified semantic joint space. Its number of rows and columns corresponds to the dimensions of the two joint spaces, respectively. Non-zero elements represent joints with corresponding relationships, and zero elements represent joints without corresponding relationships. It can be used to implement at least one of the following: joint correspondence, order adjustment, zero-padding for missing joints, and discarding redundant joints. It is a component responsible for joint-level mapping in kinematic mapping relationships.
[0056] Different robots may have different joint arrangements. If the arrangement order is different, directly inputting motion control strategies will cause semantic misalignment of variables. By adjusting the joint order, the joint states of the target robot can be arranged according to the unified joint order in the unified semantic joint space after transformation. This joint order adjustment can be achieved through the arrangement relationship in the sparse joint mapping matrix.
[0057] Since the target robot and the source robot may differ in the number of degrees of freedom, the target robot may lack some joints present in the source robot. To enable the joint states of the target robot to be transformed into a unified semantic joint space, zeros can be padded at the positions of the missing joints, ensuring that the dimension of the transformed state vector matches the input dimension of the motion control strategy. This zero-padding operation can be achieved using the zero elements corresponding to the missing joints in the sparse joint mapping matrix.
[0058] The target robot may possess additional degrees of freedom that the source robot does not have. If the motion control strategy has not received corresponding inputs during the training phase, it cannot be directly processed. Therefore, these redundant degrees of freedom can be discarded or isolated from the unified semantic joint space, ensuring that the dimension of the transformed state vector matches the input dimension of the motion control strategy. This discarding or isolation operation can be achieved using the zero elements corresponding to the redundant degrees of freedom in the sparse joint mapping matrix.
[0059] Specifically, determining the axis decoupling relationship between the joint axis system of the target robot and the unified semantic joint space based on the target robot's body structure can include the following: Different robots may differ in joint axis orientation, joint axis tilt angle, and mirror coordinate conventions. The axis decoupling relationship is used to compensate for at least one of the differences in joint axis orientation, joint axis tilt angle, and mirror coordinate conventions between the target robot and the source robot, so that the motion quantities of each joint of the target robot can be transformed into a unified semantic joint space according to a unified coordinate convention.
[0060] In one implementation, for a robot that includes a tilted hip axis, a two-dimensional decoupling sub-relationship corresponding to the tilted hip axis can be set in the axis decoupling relationship, so that the motion of the hip in different directions is converted into a unified coordinate convention in a unified semantic joint space.
[0061] In one implementation, the aforementioned axis decoupling relationship can be represented by an axis decoupling matrix, which is used to compensate for at least one of the differences in joint axis direction, tilt angle, and mirror coordinate conventions. An axis decoupling matrix is a matrix representation used to describe the axis differences between two joint spaces. In robot kinematics, different robots may differ in joint axis direction, joint axis tilt angle, and coordinate system conventions. Even if two robots have joints with the same function, their joint axis directions, tilt angles, or sign conventions may differ. The axis decoupling matrix is used to represent and compensate for these axis differences, enabling the joint motion quantities of the target robot to be transformed to a unified semantic joint space according to a unified coordinate convention.
[0062] Specifically, determining the mechanism correction relationship between the mechanism constraints of the target robot and the unified semantic joint space based on the target robot's body structure can include the following: When a target robot employs parallel, closed-loop, or other non-serial mechanisms, there may be a non-one-to-one correspondence between its executed variables and kinematic joint variables. Mechanism correction relationships are used to compensate for the inconsistency between executed variables and kinematic joint variables in the target robot's parallel or closed-loop mechanisms, ensuring that the target robot's actuation quantities are consistent with the equivalent joint quantities in the unified semantic joint space.
[0063] In one implementation, the aforementioned mechanism correction relationship can be represented by a closed-chain or parallel mechanism correction matrix to compensate for the inconsistency between executed variables and kinematic joint variables. A closed-chain or parallel mechanism correction matrix is a matrix representation used to describe the correspondence between the target robot's mechanism constraints and the unified semantic joint space. In robot kinematics, there are differences in the relationship between joint variables and executed variables between serial, parallel, and closed-chain mechanisms. In serial mechanisms, the joints are usually arranged in a serial order, and there is generally a one-to-one correspondence between joint variables and executed variables; however, in parallel or closed-chain mechanisms, multiple driven joints jointly determine the motion of the end effector or a certain part through mechanism constraints, and there may not be a one-to-one correspondence between driven variables and kinematic joint variables. The closed-chain or parallel mechanism correction matrix is used to represent and compensate for this inconsistency, ensuring that the target robot's driven quantities are consistent with the equivalent joint quantities in the unified semantic joint space.
[0064] Specifically, based on joint correspondence, axis decoupling, and mechanism correction relationships, the kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space is constructed, which may include the following: The joint state vector q_T of the target robot and the state vector in the unified semantic joint space The positive mapping relationship between them can be expressed as: , ; in, This represents the original joint vector of the target robot; This represents the joint vector after transformation to the unified semantic joint space; Represents the overall mapping matrix from the target joint space to the unified semantic joint space; This represents a sparse joint mapping matrix, used to complete joint mapping, order adjustment, padding missing joints with zeros, and discarding redundant joints. This represents the axis decoupling matrix, used to compensate for differences in the conventions of joint axis direction, tilt angle, and mirror coordinates; This represents the correction matrix for closed-loop or parallel mechanisms, used to compensate for inconsistencies between execution variables and kinematic joint variables.
[0065] Sparse joint mapping matrix Construction: Let be a T×N matrix (T is the dimension of the unified semantic joint space, and N is the number of degrees of freedom of the target robot). For the j-th joint of the target robot, if it corresponds to the i-th unified joint in the unified semantic joint space, then let (i,j)=1; if the target robot lacks a common joint possessed by the source robot, then the row corresponding to that common joint is all zeros (equivalent to zero padding); if the target robot has redundant degrees of freedom, then the column corresponding to that degree of freedom is all zeros (equivalent to discarding). When the joint order is inconsistent, it is determined by... The positions of non-zero elements are rearranged.
[0066] Construction of the axis decoupling matrix D: D is an N×N block diagonal matrix (or a block matrix grouped by joints). For each joint or joint group, based on the joint axis direction, tilt angle, and mirror coordinate convention in the ontology description information, the transformation relationship between the target robot coordinate system and the unified semantic joint space coordinate system is determined. For the tilted hip axis system, a 2×2 decoupling sub-matrix is set to convert the motion quantities in the two directions into a unified coordinate convention. For differences in mirror coordinate conventions, a sign factor (such as -1) is set in the corresponding elements. Each element of D can be determined by the projection relationship between the target robot's joint axis unit vector and the coordinate basis of the unified semantic joint space.
[0067] Construction of the correction matrix J for closed-loop or parallel mechanisms: J is an N×N matrix (or an N×M matrix, where M is the dimension of the executed variables). For serial mechanisms, J can be an identity matrix; for parallel or closed-loop mechanisms, a Jacobian relationship between executed variables and kinematic joint variables is established based on the mechanism constraint equations (such as closed-loop vector equations), mapping the executed variables to equivalent joint variables. The elements of J are determined by the partial derivatives of this Jacobian relationship with respect to the joint variables. For different types of robots: for serial robots, J = I; for parallel robots, J is constructed according to the Jacobian matrix of closed-loop constraints; for closed-loop mechanisms, J is constructed according to the Jacobian matrix of closed-loop constraints.
[0068] The S, D, and J parameters mentioned above can be automatically generated in advance based on the target robot's robot description file (such as a URDF file) after extracting the number of joints, joint sequence, joint axis direction, link size, and mechanism constraint information. Alternatively, they can be manually configured based on the body description information.
[0069] For example, please see Figure 3 , Figure 3 This is a schematic diagram illustrating joint-level kinematic alignment and inverse motion transformation provided in an embodiment of this application. Figure 3 As shown, the target joint vector After sequentially undergoing sparse mapping, axis decoupling, and closed-chain correction, the model is transformed into a unified semantic joint space, resulting in unified joint quantities. .
[0070] Among them, sparse mapping is used to complete joint correspondence, sequence adjustment, zero padding and discarding; axis decoupling is used to compensate for differences in the conventions of joint axis direction, tilt angle and mirror coordinates; closed chain correction is used to compensate for inconsistencies between the execution variables and kinematic joint variables in parallel or closed chain mechanisms.
[0071] Through the above mapping, the joint states of the target robot can be represented according to a unified semantic joint space, providing a basis for subsequent input motion control strategies.
[0072] In some embodiments, the kinematic mapping relationship is not limited to a fixed matrix form; it can also be implemented using lookup tables, piecewise functions, analytical geometric transformations, differentiable mapping networks, or mapping rules automatically generated based on the robot description file. For example, a correspondence table between the target robot's joint states and unified semantic joint states can be pre-established, and mapping can be completed by looking up the table; alternatively, intervals can be divided according to the joint range of motion or mechanism constraints, and different mapping functions can be used in different intervals; mapping can also be established through analytical geometric relationships based on the joint axis direction, link size, and mechanism constraints of the target robot; or the mapping relationship between the target robot's joint space and unified semantic joint space can be learned through a differentiable mapping network; or information such as the number of joints, joint sequence, joint axis direction, and mechanism constraints can be automatically extracted from the target robot's robot description file to generate corresponding mapping rules. The above implementation methods can be used individually or in combination, as long as they can establish the correspondence between the target robot's joint space and the unified semantic joint space.
[0073] 103. While keeping the main network parameters of the motion control strategy unchanged, introduce trainable adaptive parameters into the motion control strategy.
[0074] The main network parameters refer to the original network parameters that the motion control strategy possesses after pre-training on the source robot. These parameters may include the weights and biases of the input, intermediate, and output layers of the motion control strategy, and are used to generate control actions based on the robot's state and reference motion information.
[0075] Among them, adaptive parameters refer to trainable parameters added in addition to the main network parameters. Adaptive parameters are only used to compensate for the dynamic differences between the target robot and the source robot.
[0076] In some embodiments, the step "introducing trainable adaptive parameters into the motion control policy while keeping the main network parameters of the motion control policy unchanged" may include the following operations: In the motion control strategy, a dynamic-related network layer is identified, which includes at least one of the following: ontology state input projection layer, dynamic sensitivity layer in the strategy backbone, action output related layer, and value network. Keeping the original weight matrix of the motion control strategy unchanged, low-rank trainable parameters are introduced into the selected dynamics-related network layers; Among them, the rank of the low-rank trainable parameters is smaller than the input dimension and output dimension introduced into the network layer.
[0077] Motion control strategies typically comprise multiple network layers, each with varying sensitivities to differences in robot dynamics. To avoid extensive modifications to the source policy parameters, adaptive parameters can be introduced only in those layers that significantly impact the target robot's dynamic characteristics. Specifically, the ontology state input projection layer receives the target robot's ontology state input, and its output features are directly related to the target robot's dynamic characteristics; the dynamics-sensitive layer in the policy backbone generates control features; the action output correlation layer generates action outputs; and the value network estimates values during reinforcement learning. At least one of these network layers can serve as a dynamics-related layer.
[0078] In the motion control strategy, the reference motion encoding module related to the motion task objective is kept frozen to keep the features related to the expression of the motion objective stable and avoid changes to the expression of the motion task itself due to adaptation.
[0079] Let W be the original weight matrix of a certain network layer in the motion control strategy. This original weight matrix is the weight matrix that the motion control strategy possesses after pre-training on the source robot. During the transfer process, the original weight matrix W remains frozen and does not participate in subsequent parameter updates. This is to preserve the motion generation and control capabilities that the motion control strategy has learned on the source robot, and to avoid the degradation of original balance control, whole-body coordination, and action timing capabilities due to large-scale modifications to the source strategy parameters.
[0080] In one implementation, while keeping the original weight matrix W unchanged, a low-rank trainable matrix can be added to the selected network layer to obtain the adapted network weights: in, The original weight matrix representing the motion control strategy is kept frozen during the transition; and This is a newly added low-rank trainable matrix; This represents the adapted network weights; the low-rank dimension k is smaller than the original weight matrix. The input and output dimensions.
[0081] As can be seen from the above formula, the adapted network weights From the original weight matrix With low-rank terms The original weight matrix is obtained by adding them together. Keep frozen, only low-rank trainable matrices and It participates in subsequent parameter updates. Since the low-rank dimension k is smaller than the input and output dimensions of the original weight matrix, the number of newly trainable parameters is much smaller than the number of parameters in the complete network, thus achieving dynamic adaptation of the target robot with a smaller training cost.
[0082] For example, please see Figure 4 , Figure 4 This is a schematic diagram of the dynamic adaptive module provided in an embodiment of this application. Figure 4 As shown, the dynamics adaptive module includes two pathways. In one pathway, dynamics-related inputs such as the body state and historical actions (i.e., the target robot's state information) are processed by the body input projection (i.e., the body state input projection layer) and the strategy backbone / action head (i.e., the dynamics-sensitive layer and action output-related layer in the strategy backbone) to output the target action; in the other pathway, motion target information such as reference motion is processed by the reference motion encoding module and then input into the value network.
[0083] In the dynamic adaptive process, the original parameters of the motion control strategy While keeping the system frozen, insert low-rank adaptation parameters at at least one of the ontology input projection, policy backbone / action head, and value network. The adapted weights can be represented as follows: The reference motion coding module is kept frozen to keep features related to the representation of the motion target stable.
[0084] Using the above method, the dynamic differences between the target robot and the source robot can be compensated with a small number of low-rank adaptation parameters without modifying the main parameters of the motion control strategy.
[0085] In one implementation, low-rank trainable parameters can be set in at least one of the ontology state input projection layer, the dynamics-sensitive layer in the policy backbone, the action output related layer, and the value network, so that the ontology-related control path can be adjusted according to the dynamic characteristics of the target robot.
[0086] In some embodiments, the dynamic differences compensated by the adaptive parameters include at least one of the following: The difference in the mass matrix between the target robot and the source robot; The difference in Coriolis force and centrifugal force terms between the target robot and the source robot; Difference in gravity terms between the target robot and the source robot; Joint frictional resistance term of the target robot; The equivalent error caused by the actuator dynamics, gain, or response delay of the target robot; The equivalent error caused by the difference between the target robot's contact model and the ground action.
[0087] Among them, the mass matrix difference refers to the difference between the target robot and the source robot in terms of mass, inertia and link size, which leads to different inertial forces that need to be overcome at the same joint position.
[0088] The difference between Coriolis force and centrifugal force refers to the fact that the target robot and the source robot are different in terms of mass distribution, link length and joint arrangement, resulting in different Coriolis force and centrifugal force under the same joint speed and joint position.
[0089] The difference in gravity terms refers to the fact that the target robot and the source robot differ in terms of link mass, center of mass position, and joint arrangement, resulting in different gravity forces that need to be overcome at the same joint position.
[0090] Joint friction resistance refers to unmodeled resistance terms such as joint friction of the target robot. The joint friction characteristics of different robots may vary and are difficult to model accurately.
[0091] Equivalent error caused by actuator dynamics, gain, or response delay refers to the difference between the target robot and the source robot in terms of actuator dynamic response, gain, or response delay, resulting in different actual execution effects under the same control commands.
[0092] The equivalent error caused by the difference between the contact model and the ground action refers to the difference between the target robot and the source robot in terms of foot structure, contact stiffness, friction coefficient and ground characteristics, which leads to different ground action forces under the same motion state.
[0093] The aforementioned dynamic differences can be compensated individually or in combination. The adaptive parameters are used only to compensate for the dynamic differences between the target robot and the source robot, and not to relearn the motion task itself, so that the motion control strategy on the target robot maintains similar motion tracking, balance maintenance, and whole-body coordination capabilities as the source robot.
[0094] After kinematic alignment, the main difference between the target robot and the source robot translates into dynamic differences. The dynamics of the target robot in the unified semantic joint space can be expressed as: in, This represents the mass matrix of the target robot. Represents the joint position in the unified semantic joint space; and These represent joint velocity and joint acceleration, respectively. This indicates the terms related to Coriolis force and centrifugal force; Represents the gravity term; Indicates the driving torque of the target robot; It refers to a generalized force generated by external contact or external force.
[0095] Because different robots have different mass distributions, inertia, gravitational loads, actuator responses, and joint friction characteristics, the control quantities required for the target robot and the source robot to move under the same motion state differ. This difference can be expressed as a dynamic residual: in, This represents the dynamic residual that needs to be compensated by adaptive parameters; , , These represent the differences in the mass matrix, Coriolis force and centrifugal force terms, and gravity terms between the target robot and the source robot, respectively. This represents unmodeled resistance terms such as joint friction; This represents the equivalent error caused by actuator dynamics, gain, or response delay. This represents the equivalent error caused by the difference between the contact model and the ground action.
[0096] The aforementioned dynamic residuals are compensated by adaptive parameters. Since the core network parameters of the motion control strategy remain unchanged, the adaptive parameters are only used to learn the dynamic differences between the target robot and the source robot, and not to relearn the motion task itself. By compensating for at least one of the aforementioned dynamic residuals, the motion control strategy can maintain similar motion tracking, balance maintenance, and whole-body coordination capabilities on the target robot as it does on the source robot.
[0097] In some embodiments, dynamic adaptation is not limited to low-rank adaptation; it can also be implemented using other low-parameter forms, such as residual layers, bottleneck adaptation layers, incremental parameter layers, or combinations thereof. Any implementation that can learn the dynamic differences between the target robot and the source robot using a small number of trainable parameters while keeping the parameters of the main network of the motion control strategy unchanged falls within the scope of this application's coverage of dynamic adaptation.
[0098] In some embodiments, the dynamic differences can be obtained offline, for example, by pre-calculating based on robot parameters of the target robot and the source robot; or they can be estimated online or semi-online, for example, based on at least one of the target robot's state during operation, actuator feedback, contact information, and control errors. The specific method for obtaining the dynamic differences can be determined according to the actual application scenario.
[0099] 104. Obtain the state information of the target robot, transform the state information into a unified semantic joint space according to the kinematic mapping relationship, and input the motion control strategy with introduced adaptive parameters to obtain the target action output by the motion control strategy.
[0100] Among them, the target robot's state information refers to the input state data collected or acquired by the target robot during operation, which is used to input motion control strategies. It can be used to reflect the target robot's current motion state, posture state, and historical actions.
[0101] In some embodiments, the state information may include at least one of base state, joint position, joint velocity, historical actions, and reference motion information. Base state refers to at least one of the state quantities of the target robot's base, such as position, orientation, linear velocity, and angular velocity; joint position refers to the current angle or displacement of each joint of the target robot; joint velocity refers to the current angular velocity or linear velocity of each joint of the target robot; historical actions refer to the actions performed by the target robot in one or more control cycles prior to the current moment; and reference motion information refers to the reference motion data of the target robot to be tracked.
[0102] In one implementation, the state information can be aligned with the observation layout definition used by the motion control strategy during the training phase, so that the inputs received by the motion control strategy are semantically consistent with those during its training phase.
[0103] In some embodiments, the step "acquiring the state information of the target robot, transforming the state information to a unified semantic joint space according to the kinematic mapping relationship and inputting a motion control strategy with introduced adaptive parameters to obtain the target action output by the motion control strategy" may include the following operations: Obtain the state information of the target robot, which includes at least one of the following: base state, joint position, joint velocity, historical actions, and reference motion information; According to the kinematic mapping relationship, the target robot joint state in the state information is transformed into a unified semantic joint space to obtain a unified semantic state; According to the observation layout definition of the motion control strategy, the variable order of the unified semantic state is aligned. The observation layout definition includes the input variable order and input semantic conventions adopted by the motion control strategy during the training phase. The unified semantic state input after variable order alignment has been incorporated into the motion control strategy with adaptive parameters. Obtain the target action output by the motion control strategy in a unified semantic joint space.
[0104] Among them, the base state reflects the position, attitude, linear velocity and angular velocity of the target robot base; the joint position reflects the current angle or displacement of each joint; the joint velocity reflects the current angular velocity or linear velocity of each joint; the historical action reflects the actions executed in one or more control cycles before the current moment, which is used to maintain the temporal continuity of the action; and the reference motion information reflects the reference motion data to be tracked.
[0105] The joint states of the target robot are represented according to its own joint space. The number, order and axis orientation of its joints may be inconsistent with the unified semantic joint space used in the training phase of the motion control strategy. Through kinematic mapping, it can be transformed into the unified semantic joint space so that the joint states received by the motion control strategy are semantically consistent with its training phase.
[0106] Among them, the variable order alignment of the unified semantic state can be carried out by rearranging the variables in the unified semantic state according to the observation layout definition of the motion control strategy, so that the rearranged variable order is consistent with the input variable order used in the training phase of the motion control strategy, and its semantics are consistent with the input semantic conventions used in the training phase.
[0107] A motion control strategy with introduced adaptive parameters refers to a motion control strategy that incorporates trainable adaptive parameters while keeping the main network parameters unchanged. The adaptive parameters are used solely to compensate for the dynamic differences between the target robot and the source robot. After inputting a unified semantic state (with variables aligned in order) into this motion control strategy, the strategy can generate corresponding control actions based on the unified semantic state, and compensate for the dynamic differences of the target robot due to the presence of the adaptive parameters.
[0108] Upon receiving a unified semantic state, the motion control strategy outputs the target action in the unified semantic joint space. The target action can be represented as joint position, joint velocity, or equivalent control commands. This target action is represented in the unified semantic joint space, and can subsequently be transformed into the joint space of the target robot according to the inverse transformation relationship corresponding to the kinematic mapping relationship, thus obtaining the joint control commands of the target robot.
[0109] In some embodiments, the target action output by the motion control strategy can be a joint position increment, a joint position target, a joint velocity target, a torque target, or other equivalent control quantities. The specific form of the target action can be determined according to the underlying control interface and control requirements of the target robot, as long as it can be transformed to the joint space of the target robot through an inverse transformation relationship and obtain the corresponding joint control commands.
[0110] 105. According to the inverse transformation relationship corresponding to the kinematic mapping relationship, the target motion is transformed into the joint space of the target robot to obtain the joint control command of the target robot.
[0111] Inverse transformation is a mapping relationship that is opposite in direction to the kinematic mapping relationship. The kinematic mapping relationship is used to transform the joint space of the target robot to a unified semantic joint space, while the inverse transformation relationship is used to transform the actions in the unified semantic joint space back to the joint space of the target robot. They are opposite in direction and correspond to each other.
[0112] In some embodiments, the step "transforming the target motion into the joint space of the target robot according to the inverse transformation relationship corresponding to the kinematic mapping relationship, and obtaining the joint control commands of the target robot" may include the following operations: Based on the joint correspondence, the target motion is mapped to the joint space of the target robot; According to the inverse axis transformation relationship corresponding to the axis decoupling relationship, perform inverse axis transformation on the mapped target motion; Based on the mechanism inverse correction relationship corresponding to the mechanism correction relationship, the target motion after the axis system inverse transformation is corrected by mechanism inverse correction to obtain the joint control commands of the target robot.
[0113] Specifically, the motion control strategy outputs the target action in a unified semantic joint space. Each action dimension in the unified semantic joint space has a corresponding relationship with the joints of the target robot through a joint correspondence. The target action can be backfilled or mapped to the joint space of the target robot according to this correspondence. Action dimensions in the unified semantic joint space that are not actually executed by the target robot can be discarded during the mapping process.
[0114] Among them, the axis decoupling relationship is used to compensate for the differences in joint axis direction, tilt angle and mirror coordinate conventions in the forward mapping. In the inverse transformation, the target motion after mapping needs to be transformed in reverse according to the corresponding axis inverse transformation relationship so that the target motion is represented according to the axis convention of the target robot itself.
[0115] Among them, the mechanism correction relationship is used to compensate for the inconsistency between the execution variables and kinematic joint variables in parallel or closed mechanisms in the forward mapping. During the inverse transformation, the target motion after the axis system is inversely transformed needs to be corrected in reverse according to the corresponding mechanism inverse correction relationship, so that the target motion is converted into executable joint control commands according to the mechanism constraints of the target robot.
[0116] In one implementation, the target action Joint control commands for the target robot The inverse transformation relationship between them can be expressed as: in, This represents the target action generated by the motion control strategy in the unified semantic joint space; This represents the converted joint control commands that the target robot can execute. This represents the inverse motion transformation matrix from the unified semantic joint space to the target robot's joint space; This means that the target motion will be filled into the joint positions of the target robot according to the joint correspondence. Indicates the inverse transformation of the axis system; It indicates the reverse motion relationship of a closed chain or parallel mechanism.
[0117] For example, please see Figure 5 , Figure 5 This is a schematic diagram illustrating the inverse transformation of actions provided in an embodiment of this application. For example... Figure 5 As shown, source policy action After sequentially undergoing closed-chain inverse transformation, axis inverse transformation, and joint backfilling, the transformation is completed in the joint space of the target robot, thus obtaining the target robot's motion. .
[0118] Among them, closed-chain inverse transformation is used to compensate for the inconsistency between the execution variables and kinematic joint variables in parallel or closed-chain mechanisms; axis inverse transformation is used to compensate for the differences in the conventions of joint axis direction, tilt angle and mirror coordinates; joint backfilling is used to backfill the motion to the joint position of the target robot according to the joint correspondence.
[0119] Through the inverse transformation described above, the actions output by the motion control strategy in the unified semantic joint space can be converted into joint control commands that the target robot can execute.
[0120] 106. Based on the results of the target robot executing joint control commands, only update the adaptive parameters in the motion control strategy to obtain the target motion control strategy suitable for the target robot.
[0121] In some embodiments, the step "updating only the adaptive parameters in the motion control strategy based on the result of the target robot executing joint control commands, to obtain a target motion control strategy suitable for the target robot" may include the following operations: Execute joint control commands in the simulation environment of the target robot and / or the closed-loop environment of the physical robot, and collect the operating status of the target robot; Based on the operating status, determine at least one of the following: motion tracking error, posture balance state, and motion smoothness of the target robot; Optimization objectives are constructed based on at least one of motion tracking error, posture balance, and motion smoothness. While keeping the main network parameters of the motion control strategy unchanged, only the adaptive parameters in the motion control strategy are updated according to the optimization objective. When the optimization objective satisfies the preset convergence condition, the parameters of the kinematic mapping relationship and the updated adaptive parameters are fixed to obtain the target motion control strategy suitable for the target robot.
[0122] The simulation environment or closed-loop environment of the target robot can be used to simulate or actually execute joint control commands. In the simulation environment, joint control commands can be executed through the simulator, and the operating status of the target robot can be acquired. In the closed-loop environment of the physical robot, joint control commands can be sent to the target robot body for execution, and the operating status of the target robot can be acquired in real time. The operating status can be used to reflect the movement of the target robot after executing the joint control commands.
[0123] Among them, motion tracking error can be used to reflect the deviation between the actual motion of the target robot and the reference motion, such as joint tracking error, base position error or base posture error; posture balance state can be used to reflect the balance of the target robot during execution; motion smoothness can be used to reflect the smoothness of the target robot's motion output, avoiding sudden changes or jitter in motion.
[0124] The optimization objective can be used to evaluate the motion control performance of the target robot under current adaptive parameters. In one implementation, the optimization objective can be obtained by a weighted combination of at least one of motion tracking error, posture balance state, and motion smoothness.
[0125] The optimization objective can be expressed as: L = w1·L_track + w2·L_balance + w3·L_smooth, where L_track is the motion tracking error (such as the mean square error of joint position tracking error or the base posture error), L_balance is the posture balance state (such as the base posture angle deviation, zero torque point deviation, or foot contact force deviation), and L_smooth is the smoothness of the motion (such as the sum of squares of the motion difference between adjacent control cycles or the penalty term for the rate of change of motion); w1, w2, and w3 are weighting coefficients, which can be set according to task requirements, for example, w1 is 1.0, w2 is 0.5, and w3 is 0.1, or determined through grid search / empirical adjustment.
[0126] During the optimization process, the main network parameters remain unchanged, and only the adaptive parameters in the motion control strategy are updated. The adaptive parameters are only used to compensate for the dynamic differences between the target robot and the source robot. Therefore, updating only the adaptive parameters is sufficient to adapt the motion control strategy to the dynamic characteristics of the target robot while maintaining the original control capability of the source strategy.
[0127] Updating adaptive parameters can be done using gradient descent or its variants (such as Adam, RMSProp): Joint control commands are executed in a simulation or physical closed-loop environment, the running status is collected, the gradient of the optimization objective L with respect to the adaptive parameters is calculated, and only the adaptive parameters (such as low-rank matrices A and B) are updated, while the main network parameters W remain frozen. The learning rate can be set to the order of 1e-4 to 1e-3, and the low-rank dimension k can be an integer smaller than the input and output dimensions of the network layer, such as 4, 8, or 16. The number of training iterations or the convergence condition can be set to the optimization objective decreasing by less than a threshold (such as 1e-4) within a certain number of consecutive iterations, or reaching a preset maximum number of iterations (such as 1000).
[0128] The preset convergence condition can be set according to the changes in the optimization target, such as when the optimization target no longer decreases significantly or reaches the preset number of iterations. When the optimization target meets the preset convergence condition, the parameters of the kinematic mapping relationship and the updated adaptive parameters can be fixed to form a target robot-specific strategy configuration, resulting in a target motion control strategy suitable for the target robot.
[0129] Through the above operations, an optimization objective can be constructed in the closed-loop environment of the target robot based on at least one of motion tracking error, posture balance state, and motion smoothness. While keeping the main network parameters unchanged, only the adaptive parameters are updated. After the convergence condition is met, the kinematic mapping parameters and adaptive parameters are solidified, thereby obtaining a target motion control strategy suitable for the target robot.
[0130] When deploying the application, the main network parameters of the motion control strategy remain unchanged. The motion control strategy can be transferred to the target robot simply by loading the parameters of the corresponding kinematic mapping relationship and the adaptive parameters.
[0131] The migration configuration consists of two parts: first, kinematic mapping parameters, used for the mutual conversion between the target robot's state information and the unified semantic joint space; and second, adaptive parameters, used to compensate for the dynamic differences between the target robot and the source robot. During deployment, the main network parameters of the motion control strategy, the kinematic mapping parameters, and the adaptive parameters are loaded into the target robot's control system, enabling the target robot to operate according to the motion control strategy.
[0132] The same source strategy entity can store the kinematic mapping parameters and adaptive parameters for different target robots. When migrating to another target robot, there is no need to modify the source strategy entity; simply construct and save the kinematic mapping parameters and adaptive parameters corresponding to that target robot to complete the migration and deployment.
[0133] In some embodiments, the motion control migration method for heterogeneous robots can be executed by a processor running on a training server or a robot master control computing platform.
[0134] In one implementation, kinematic mapping, policy inference, dynamic adaptation, and low-level execution control can be deployed in different computing modules. For example, kinematic mapping and policy inference can be deployed on a training server or the robot's main control computing platform, dynamic adaptation can be deployed on the training server, and low-level execution control can be deployed on the target robot's low-level controller or actuator. The computing modules can interact with each other via communication links to jointly complete the migration of motion control policies and the control of the target robot.
[0135] The motion control transfer method for heterogeneous robots provided in this application has the following advantages: it can transfer the motion control strategy of the source robot to target robots with different structures and degrees of freedom based on the existing motion control strategy of the source robot, reducing the workload of redesigning and retraining the complete control strategy for each robot; by first observing the layout and aligning the joints at the kinematic level, the input and output of the motion control strategy on the target robot have consistent physical semantics, reducing control errors caused by differences in joint sequence, degrees of freedom, and mechanisms; by only adapting the parameters of the dynamically sensitive modules, it can retain the whole-body coordination, balance maintenance, and motion tracking capabilities already formed in the motion control strategy, and adjust them for the dynamics of the target robot; compared with training from scratch or re-optimizing the complete parameters, it can reduce the scale of training parameters and computational overhead in the transfer stage, improve the efficiency of strategy transfer, and shorten the cycle of forming a new robot control strategy; by separating and saving the kinematic mapping parameters and target dynamics adaptation parameters from the main body of the source strategy, lightweight configurations can be formed for multiple different target robots, improving the convenience of extending and deploying the same source strategy to multiple heterogeneous robots.
[0136] Based on the above description, the following examples will further illustrate the motion control transfer method for heterogeneous robots proposed in this application. Please refer to... Figure 6 , Figure 6 This is a schematic diagram of the overall motion control transfer process for heterogeneous robots provided in an embodiment of this application. The specific process can be as follows: Target robot state input. Acquire the target robot's state, including its own state and historical actions, as input for subsequent processing.
[0137] Observe the layout alignment. Align the target robot state with a unified variable order and semantics to ensure that the state of the input motion control strategy is consistent with the motion control strategy training phase in terms of variable order and semantics.
[0138] Mapping parameter generation. Mapping parameters are generated based on the target robot's body description information. The target robot's body description information includes information such as joints, mechanisms, and parameters; the mapping parameters may include joint correspondences, axis decoupling relationships, and mechanism correction relationships, which are used for subsequent joint kinematic alignment and motion inverse transformation.
[0139] Joint kinematic alignment. Based on the mapping parameters, the joint kinematic alignment of the target robot state after the observation layout is aligned is performed, handling the differences in degrees of freedom, axis systems, and closed-loop mechanisms between the target robot and the source robot, and transforming the target robot state into a unified semantic joint space.
[0140] Input the adapted motion control strategy. The kinematically aligned joint states are input into the adapted motion control strategy. The adapted motion control strategy consists of two parts: a pre-trained strategy body and a dynamic adaptation component. The parameters of the pre-trained strategy body are kept frozen, while the dynamic adaptation component is used to compensate for the dynamic differences between the target robot and the source robot.
[0141] Inverse motion transformation. After adaptation, the motion control strategy outputs actions in a unified semantic joint space, which are then transformed into the joint space of the target robot through inverse motion transformation to obtain joint control commands that the target robot can execute.
[0142] The target robot executes the commands. Joint control commands are sent to the target robot's underlying controller or actuator for execution, and the resulting state is fed back to the dynamic adaptive process.
[0143] Dynamics difference estimation. Based on information such as the target robot's mass, inertia, actuators, and contacts, the dynamic differences between the target robot and the source robot are estimated, providing a basis for dynamic adaptation.
[0144] Dynamic adaptation. Based on the dynamic difference estimation results and the state feedback after the target robot's execution, the low-rank adaptation parameters are updated so that the adapted motion control strategy can compensate for the dynamic differences of the target robot.
[0145] Through the above process, with the main parameters of the pre-trained strategy frozen, the structural differences between the target robot and the source robot can be eliminated by kinematic alignment, and the remaining dynamic differences can be compensated by dynamic adaptive compensation, thereby realizing the transfer of motion control strategy to heterogeneous target robots.
[0146] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a plurality of computer programs that can be loaded by a digital signal processor to execute steps in any of the motion control migration methods for heterogeneous robots provided in embodiments of this application.
[0147] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.
[0148] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0149] Since the computer program stored in the computer-readable storage medium can execute the steps in any of the motion control transfer methods for heterogeneous robots provided in the embodiments of this application, the beneficial effects that any of the motion control transfer methods for heterogeneous robots provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.
[0150] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. A motion control transfer method for heterogeneous robots, characterized in that, The method includes: Obtain the pre-trained motion control strategy corresponding to the source robot, as well as the ontological description information of the target robot; Based on the ontology description information, a kinematic mapping relationship is constructed between the joint space of the target robot and the unified semantic joint space of the motion control strategy. While keeping the main network parameters of the motion control strategy unchanged, trainable adaptive parameters are introduced into the motion control strategy. These adaptive parameters are only used to compensate for the dynamic differences between the target robot and the source robot. The state information of the target robot is obtained, and the state information is transformed into the unified semantic joint space according to the kinematic mapping relationship. A motion control strategy with the adaptive parameters is then input, and the target action output by the motion control strategy is obtained. The unified semantic joint space is defined according to the joint space of the source robot, and the joints of the source robot are arranged according to the unified joint semantics during the training phase to form the unified semantic joint space. The kinematic mapping relationship is used to map the joint space of the target robot to the unified semantic joint space. According to the inverse transformation relationship corresponding to the kinematic mapping relationship, the target motion is transformed into the joint space of the target robot to obtain the joint control command of the target robot, and the joint control command is constrained according to at least one of joint limit, velocity limit, torque limit and safety threshold. Based on the result of the target robot executing the joint control command, only the adaptive parameters in the motion control strategy are updated to obtain a target motion control strategy suitable for the target robot. The step of acquiring the state information of the target robot, transforming the state information to the unified semantic joint space according to the kinematic mapping relationship, and inputting a motion control strategy with the adaptive parameters incorporated therein to obtain the target action output by the motion control strategy includes: The state information of the target robot is obtained, and the state information includes at least one of the following: base state, joint position, joint velocity, historical actions, and reference motion information; According to the kinematic mapping relationship, the target robot joint state in the state information is transformed to the unified semantic joint space to obtain the unified semantic state; According to the observation layout definition of the motion control strategy, the variable order of the unified semantic state is aligned, wherein the observation layout definition includes the input variable order and input semantic conventions adopted by the motion control strategy during the training phase; The unified semantic state input, after aligning the variable order, has been incorporated into the motion control strategy of the adaptive parameters; Obtain the target action output by the motion control strategy in the unified semantic joint space.
2. The method according to claim 1, characterized in that, The step of constructing a kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space of the motion control strategy based on the ontology description information includes: Based on the ontology description information, establish the joint correspondence between the joints of the target robot and the unified joints in the unified semantic joint space; Based on the body structure of the target robot, determine the axis decoupling relationship between the joint axis system of the target robot and the unified semantic joint space, and the mechanism correction relationship between the mechanism constraints of the target robot and the unified semantic joint space; wherein, the axis decoupling relationship is used to compensate for at least one of the differences in joint axis direction, joint axis tilt angle, and mirror coordinate convention between the target robot and the source robot; the mechanism correction relationship is used to compensate for the inconsistency between the execution variables and kinematic joint variables in the parallel mechanism or closed mechanism of the target robot; Based on the joint correspondence, the axis decoupling relationship, and the mechanism correction relationship, a kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space is constructed.
3. The method according to claim 2, characterized in that, The step of establishing a joint correspondence between the joints of the target robot and the unified joints in the unified semantic joint space based on the ontology description information includes: The joints of the target robot are mapped to the unified joints in the unified semantic joint space to determine the position of each joint of the target robot in the unified semantic joint space. Based on the joint arrangement differences between the target robot and the source robot, the joint order of the target robot is adjusted so that the adjusted joint order is consistent with the unified joint order in the unified semantic joint space. For a unified joint that is missing in the target robot but present in the source robot, zeros are padded at the corresponding position in the unified semantic joint space; For any redundant degrees of freedom that the target robot has but the source robot does not have, the redundant degrees of freedom are discarded from the unified semantic joint space.
4. The method according to claim 2, characterized in that, The step of converting the target motion to the joint space of the target robot according to the inverse transformation relationship corresponding to the kinematic mapping relationship, to obtain the joint control commands of the target robot, includes: According to the joint correspondence, the target motion is mapped to the joint space of the target robot; According to the inverse axis transformation relationship corresponding to the axis decoupling relationship, the mapped target motion is subjected to an inverse axis transformation; According to the mechanism reverse correction relationship corresponding to the mechanism correction relationship, the target motion after the axis system inverse transformation is corrected by mechanism reverse correction to obtain the joint control command of the target robot.
5. The method according to claim 1, characterized in that, The step of introducing trainable adaptive parameters into the motion control strategy while keeping the main network parameters of the motion control strategy unchanged includes: In the motion control strategy, a dynamics-related network layer is determined, which includes at least one of the following: a body state input projection layer, a dynamics-sensitive layer in the strategy backbone, an action output related layer, and a value network. Keeping the original weight matrix of the motion control strategy unchanged, low-rank trainable parameters are introduced into the selected dynamics-related network layer, so that the introduced network weights are... ,in, The original weight matrix represents the motion control strategy, and A and B are newly added low-rank trainable matrices. Wherein, the rank of the low-rank trainable parameter is less than the input dimension and output dimension introduced into the network layer.
6. The method according to claim 1, characterized in that, The step of updating only the adaptive parameters in the motion control strategy based on the result of the target robot executing the joint control command, to obtain a target motion control strategy suitable for the target robot, includes: The joint control commands are executed in the simulation environment and / or closed-loop environment of the target robot, and the operating status of the target robot is collected. Based on the operating state, determine at least one of the following: motion tracking error, posture balance state, and motion smoothness of the target robot; An optimization objective is constructed based on at least one of the motion tracking error, the posture balance state, and the motion smoothness. While keeping the main network parameters of the motion control strategy unchanged, only the adaptive parameters in the motion control strategy are updated according to the optimization objective; When the optimization objective satisfies the preset convergence condition, the parameters of the kinematic mapping relationship and the updated adaptive parameters are fixed to obtain a target motion control strategy suitable for the target robot.
7. The method according to claim 1, characterized in that, The dynamic differences compensated by the adaptive parameters include at least one of the following: The mass matrix difference between the target robot and the source robot; The difference in Coriolis force and centrifugal force terms between the target robot and the source robot; The difference in gravity terms between the target robot and the source robot; The joint frictional resistance term of the target robot; The equivalent error caused by the actuator dynamics, gain, or response delay of the target robot; The equivalent error caused by the difference between the contact model of the target robot and the ground action.
8. A motion control migration system for heterogeneous robots, characterized in that, include: The source strategy acquisition module is used to acquire the pre-trained motion control strategy on the source robot. The target robot ontology description module is used to obtain ontology description information of the target robot; The observation layout alignment module is used to align the state information of the target robot according to the observation layout definition of the motion control strategy. The joint-level kinematic alignment module is used to construct a kinematic mapping relationship between the joint space of the target robot and the unified semantic joint space of the motion control strategy based on the ontology description information. A dynamics adaptive module is used to introduce trainable adaptive parameters into the motion control strategy while keeping the main network parameters of the motion control strategy unchanged. The adaptive parameters are only used to compensate for the dynamic differences between the target robot and the source robot. The reinforcement learning optimization module is used to update only the adaptive parameters in the motion control strategy based on the result of the target robot executing joint control commands, so as to obtain a target motion control strategy suitable for the target robot. The motion execution and safety control module is used to convert the target motion output by the motion control strategy into the joint space of the target robot according to the inverse transformation relationship corresponding to the kinematic mapping relationship, so as to obtain the joint control command of the target robot.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the motion control migration method for heterogeneous robots as described in any one of claims 1 to 7.
