Robot control method and device, robot, medium and product

Abstract

Embodiments of the present application provide a robot control method and device, a robot, a medium and a product. The method comprises: obtaining target motion information and current motion state information of a robot, the target motion information being determined based on operation input information of the robot; generating first whole-body motion reference information by a motion reference generation model based on the target motion information and the current motion state information; synchronously correcting root motion reference information in the first whole-body motion reference information according to the target motion information to obtain second whole-body motion reference information; and controlling the robot to move based on the second whole-body motion reference information. The embodiments of the present application can improve the adaptability between operation input information and whole-body motion control of the robot, and can control the consistency between motion reference and operation intention.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to robot control methods and devices, robots, media, and products. Background Technology

[0002] Robot control technology is widely used in humanoid robots, service robots, special-purpose robots, bionic robots, and industrial robots. When a robot performs movements, the various moving parts need to maintain coordination. In scenarios involving handle control, remote operation, or interactive control, insufficient adaptability between the input information and the robot's overall motion control, and difficulty in controlling the consistency between motion reference and operational intent, make it challenging to meet practical application requirements.

[0003] Based on this, embodiments of this application provide robot control methods and devices, robots, media, and products to improve related technologies. Summary of the Invention

[0004] The purpose of this application is to provide a robot control method and device, robot, medium, and product, to improve the adaptability between operation input information and robot whole-body motion control, and to control the consistency between motion reference and operation intention.

[0005] In a first aspect, embodiments of this application provide a robot control method, comprising: acquiring target motion information and current motion state information of a robot, wherein the target motion information is determined based on operation input information for the robot; generating first whole-body motion reference information through a motion reference generation model based on the target motion information and the current motion state information; synchronously correcting the root motion reference information in the first whole-body motion reference information according to the target motion information to obtain second whole-body motion reference information; and controlling the robot's motion based on the second whole-body motion reference information.

[0006] Secondly, embodiments of this application provide a robot control device, comprising: an information acquisition module for acquiring target motion information and current motion state information of the robot, wherein the target motion information is determined based on operation input information for the robot; a reference generation module for generating first whole-body motion reference information based on the target motion information and the current motion state information through a motion reference generation model; a root correction module for synchronously correcting the root motion reference information in the first whole-body motion reference information according to the target motion information to obtain second whole-body motion reference information; and a motion control module for controlling the robot's motion based on the second whole-body motion reference information.

[0007] Thirdly, embodiments of this application provide a robot, which includes a control module for performing the steps of any of the methods described above.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0009] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of any of the above methods.

[0010] This application provides robot control methods and devices, robots, media, and products. It adopts the concept of "generating whole-body reference information from a model and then synchronously correcting the root motion reference information", thereby improving the adaptability between operation input information and robot whole-body motion control, and enabling the consistency between motion reference and operation intention. Attached Figure Description

[0011] The embodiments of this application are further described below with reference to the accompanying drawings and specific implementation details.

[0012] Figure 1 This is a flowchart illustrating a robot control method provided in an embodiment of this application.

[0013] Figure 2 This is a structural block diagram of a robot control device provided in an embodiment of this application.

[0014] Figure 3 This is a structural block diagram of a robot provided in an embodiment of this application.

[0015] Figure 4 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this application.

[0017] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0018] Robot control technology is widely used in humanoid robots, service robots, special-purpose robots, bionic robots, and industrial robots. In multi-degree-of-freedom robots, such as humanoid robots, the robot typically includes multiple motion nodes, including the torso, legs, feet, arms, head, and root node. When the robot performs movements such as forward, backward, lateral, turning, standing transitions, and low-speed movement, the various moving parts need to maintain coordination; otherwise, problems such as discontinuous posture, foot slippage, movement jamming, body drift, or decreased balance may easily occur.

[0019] In scenarios involving handle control, remote operation, or interactive control, operators typically express their motion intentions through low-dimensional inputs such as forward / backward, lateral, and turning directions. There is a significant difference in information dimensionality between these low-dimensional inputs and high-dimensional whole-body motion reference information (such as the angles, velocities, and postures of the robot's torso, legs, feet, arms, head, and root nodes). If control data is generated directly based solely on these inputs, it may be difficult to simultaneously ensure motion intention response, whole-body posture coordination, and motion stability.

[0020] See Figure 1 , Figure 1 This is a flowchart illustrating a robot control method provided in an embodiment of this application.

[0021] To improve the adaptability between operation input information and robot whole-body motion control, as well as the consistency between control motion reference and operation intention, this application provides a robot control method, including steps S101 to S104.

[0022] Step S101: Obtain the target motion information and current motion state information of the robot, wherein the target motion information is determined based on the operation input information for the robot.

[0023] Step S102: Based on the target motion information and the current motion state information, generate first whole-body motion reference information through a motion reference generation model.

[0024] Step S103: Based on the target motion information, synchronously correct the root motion reference information in the first whole-body motion reference information to obtain the second whole-body motion reference information.

[0025] Step S104: Control the robot's movement based on the second whole-body motion reference information.

[0026] Target motion information refers to the robot's desired motion information determined based on operational input information. Target motion information describes how the robot "wants to move," for example, the robot wants to walk slowly forward at 0.15 m / s, or it wants to turn in place at 0.3 rad / s.

[0027] Current motion state information refers to the robot's actual motion state at the current moment or within the current control cycle. This information describes the robot's current state, enabling the motion reference generation model to generate continuous, smooth full-body motion reference information based on the robot's current posture.

[0028] Operation input information can refer to motion control information input from a gamepad, joystick, keyboard, touch interface, remote control device, or upper-level task control module.

[0029] A motion reference generation model can refer to a model used to generate first whole-body motion reference information based on target motion information and current motion state information.

[0030] The first whole-body motion reference information can refer to the whole-body motion reference result directly output by the motion reference generation model. The first whole-body motion reference information can include root motion reference information and robot body motion reference information.

[0031] Root motion reference information can refer to the reference information used to describe the overall motion reference of the robot in the first whole-body motion reference information. The root nodes involved in the root motion reference information can correspond to the skeletal reference nodes in the skeletal motion data that bear the overall displacement and orientation, such as skeletal root nodes, hip nodes, or pelvic nodes. The root nodes involved in the root motion reference information can also correspond to the base nodes in the robot's kinematic model, such as base links, pelvic links, or torso reference nodes. The root nodes involved in the root motion reference information can also correspond to virtual nodes that do not directly correspond to physical joints but are used to represent the overall displacement and orientation of the robot, such as simulation nodes, entity nodes, or simulation root nodes. Root motion reference information can originate from the output of the motion reference generation model and is used to characterize the root motion state or root motion changes predicted by the motion reference generation model. Root motion reference information can include one or more of the following: root position, root orientation, root velocity, and root angular velocity. It can also include one or more of the following relative to the current root state: position increment, orientation increment, velocity change, and angular velocity change. For humanoid robots, the root can be understood as the fundamental node connecting the body's motion posture with the overall spatial motion. The accuracy of the root motion reference information affects the overall motion stability of the robot in scenarios such as low-speed movement, turning in place, and switching between standing and walking.

[0032] Synchronous correction refers to the process of adjusting or fusing the root motion reference information in the first whole-body motion reference information based on the target motion information. For example, when the target motion information represents low-speed motion, the proportion of the expected root motion reference information in the synchronous correction result can be increased, thereby reducing the risk of the first whole-body motion reference information output by the model causing the root motion to be too fast, drifting, or stuttering.

[0033] The second whole-body motion reference information can refer to the whole-body motion reference information obtained by synchronously correcting the root motion reference information in the first whole-body motion reference information. The second whole-body motion reference information may include the corrected root motion reference information and the robot body motion reference information in the first whole-body motion reference information. In other words, the second whole-body motion reference information does not regenerate the reference pose of all robot motion nodes, but rather, while retaining the robot body motion reference information, synchronously corrects the root motion reference information that affects the overall motion reference.

[0034] In this embodiment, the robot control method first converts the operation input information into target motion information and acquires the robot's current motion state information. Then, the motion reference generation model generates first whole-body motion reference information based on the target motion information and the current motion state information, converting the low-dimensional operation intention into a whole-body motion reference result including root motion and robot body motion. Next, the root motion reference information in the first whole-body motion reference information is synchronously corrected based on the target motion information, ensuring that the root motion reference information aligns with the expected root motion trend corresponding to the target motion information. Finally, the robot's motion is controlled based on the second whole-body motion reference information.

[0035] This embodiment generates first whole-body motion reference information based on target motion information and current motion state information. This converts low-dimensional operation inputs into whole-body motion reference results for the robot, improving the adaptability between operation input information and robot whole-body motion control. By synchronously correcting the root motion reference information according to the target motion information, in scenarios such as low-speed movement, in-place turning, or stand-and-walk switching, the root motion reference information better matches the expected motion trend corresponding to the operation input information, thereby improving problems such as root drift, low-speed stuttering, or mismatch between root motion and operation intent. By generating robot control data based on second whole-body motion reference information, the downstream robot control process can track the synchronously corrected whole-body motion reference results, improving the stability and controllability of robot motion.

[0036] This embodiment addresses the problem that the dimension of the operation input information is low while the dimension of the robot's whole-body motion reference information is high, and the consistency between the root motion reference information generated by the model and the expected motion trend is difficult to control. It adopts the concept of "generating whole-body reference information by the model and then synchronously correcting the root motion reference information", thereby improving the adaptability between the operation input information and the robot's whole-body motion control, and being able to control the consistency between the motion reference and the operation intention.

[0037] In one example, the robot control cycle is 20ms. The operator inputs forward direction (0.20), lateral direction (0.02), and steering direction (0.10) via a handle. The robot control method performs dead-zone processing on the lateral direction input (0.02), sets the lateral direction input to 0, and determines the target forward speed as 0.15m / s based on the forward direction input (0.20) and the target steering angular velocity as 0.12rad / s based on the steering input (0.10). At this point, the robot's current motion state information includes the current root position (0.00m, 0.00m, 0.90m), the current root attitude angle (0°), and the left and right feet being in contact. The motion reference generation model generates the first whole-body motion reference information based on the target forward speed (0.15m / s), the target steering angular velocity (0.12rad / s), and the robot's current motion state information. In the first whole-body motion reference information, the forward displacement corresponding to the root motion reference information is 0.008m per control cycle, while the forward displacement corresponding to the desired root motion reference information of the target motion information is 0.003m per control cycle. The robot control method synchronously corrects the root motion reference information based on the target motion information, adjusting the corrected root forward displacement from 0.008m to approximately 0.003m. Simultaneously, it retains the robot body motion reference information for legs, feet, and torso from the first whole-body motion reference information, resulting in the second whole-body motion reference information. Subsequently, the robot control method generates target positions and target velocities for each joint based on the second whole-body motion reference information, controlling the robot to move forward slowly at a speed close to 0.15m / s and to make slight turns according to a target turning angular velocity of 0.12rad / s.

[0038] In some embodiments, the step of synchronously correcting the root motion reference information in the first whole-body motion reference information according to the target motion information to obtain the second whole-body motion reference information may include: determining desired root motion reference information based on the target motion information; determining root synchronization parameters based on the target linear velocity and / or target angular velocity corresponding to the target motion information; fusing the root motion reference information in the first whole-body motion reference information and the desired root motion reference information based on the root synchronization parameters to obtain corrected root motion reference information; and obtaining the second whole-body motion reference information based on the corrected root motion reference information and the robot body motion reference information in the first whole-body motion reference information.

[0039] In this embodiment, synchronous correction does not regenerate the poses of all robot motion nodes, but focuses on processing the root motion reference information that affects the robot's overall spatial motion, changing the consistency between the root motion reference information and the expected motion trend corresponding to the target motion information. For example, in scenarios such as low-speed forward movement, turning in place, and slow start-stop, synchronous correction can increase the proportion of the expected root motion reference information in the final result, thereby improving the problem of inconsistency between the model's output root motion reference information and the operational intent.

[0040] In this embodiment, the target motion information may include the target linear velocity and the target angular velocity.

[0041] Desired root motion reference information refers to reference information calculated based on target motion information, used to describe the robot's desired overall motion reference. Desired root motion reference information characterizes the root motion state or root motion change that the robot is expected to execute based on the operational input information. Desired root motion reference information may include one or more of the following: desired root position, desired root orientation, desired root velocity, and desired root angular velocity. It may also include one or more of the following relative to the current root state: desired position increment, desired orientation increment, desired velocity change, and desired angular velocity change. For example, the desired root position increment can be determined based on the product of the target linear velocity and the control cycle, and the desired root orientation increment can be determined based on the product of the target angular velocity and the control cycle, reflecting the motion change that the operational input information expects the robot's root to execute within the current control cycle.

[0042] Target linear velocity refers to the velocity information in the target motion information used to characterize the speed and direction of the robot's translational motion. Target linear velocity can include forward velocity, backward velocity, and lateral velocity, or it can include a two-dimensional translational velocity determined by both forward / backward and lateral input information. For example, the target linear velocity can be 0.15 m / s forward, 0.10 m / s to the right, or a composite velocity of 0.20 m / s forward and 0.05 m / s to the left.

[0043] Target angular velocity refers to the angular velocity information in the target motion information used to characterize the robot's turning speed and direction. The target angular velocity can be determined by the turning input information and describes the change in the robot's root posture per unit time. For example, a target angular velocity of 0.4 rad / s indicates whether the robot is turning left or right in the vertical direction.

[0044] Root synchronization parameters can refer to parameters used to control the fusion ratio between the root motion reference information in the first whole-body motion reference information and the desired root motion reference information. Root synchronization parameters can be determined based on the target linear velocity and / or target angular velocity.

[0045] Fusion processing refers to the process of combining the root motion reference information and the desired root motion reference information from the first whole-body motion reference information according to the root synchronization parameters. Fusion processing can be performed separately for root position, root attitude, root velocity, and root angular velocity. For example, root position can be fused linearly, root attitude can be fused using attitude interpolation, root velocity can be fused using velocity-weighted fusion, and root angular velocity can be fused using angular velocity-weighted fusion.

[0046] The corrected root motion reference information can refer to the root motion reference information obtained after fusion processing. The corrected root motion reference information retains the naturalness of the motion output by the motion reference generation model, while also incorporating the expected root motion trend corresponding to the target motion information.

[0047] Robot body motion reference information refers to the robot motion reference information in the first whole-body motion reference information, excluding the root motion reference information. This information can include reference positions, reference postures, and reference velocities for motion nodes such as the torso, head, arms, legs, and feet. The robot body motion reference information can continue to be derived from the output of the motion reference generation model, enabling the robot to maintain relatively natural whole-body posture changes.

[0048] This embodiment determines the desired root motion reference information based on the target motion information, converting the desired movement and turning trends corresponding to the operation input information into references at the root motion level, thus improving the targeting of subsequent root correction processing. By determining the root synchronization parameters based on the target linear velocity and / or target angular velocity, the degree of correction of the root motion reference information can be adaptively adjusted according to the robot's movement speed and turning intensity, improving adaptability under different motion states. By fusing the root motion reference information and the desired root motion reference information based on the root synchronization parameters, the root motion can be made more consistent with the target motion information while preserving the naturalness of the model's generated motion, improving the problems of root drift, excessively fast root output, or root motion stuttering during low-speed movement. By obtaining the second whole-body motion reference information based on the corrected root motion reference information and the robot's body motion reference information, root synchronization correction can be completed without regenerating the poses of all robot motion nodes, reducing processing complexity and improving the controllability of the whole-body motion reference information. This embodiment addresses the issue that the root motion reference information output by the motion reference generation model may be inconsistent with the target motion information in low-speed motion, stationary turning, or start-stop switching scenarios. It adopts the concept of "generating expected root motion reference information based on target motion information and synchronously fusing the root motion reference information output by the model", thereby improving the consistency between the robot's root motion and the operation intention, and improving the stability of the robot's slow motion and turning motion.

[0049] In one example, the robot control cycle is 20ms, and the target motion information includes a target linear velocity of 0.15m / s and a target angular velocity of 0.30rad / s. The robot control method calculates the desired root position increment as 0.003m based on the target linear velocity of 0.15m / s and the control cycle of 20ms; and calculates the desired root pose increment as 0.006rad based on the target angular velocity of 0.30rad / s and the control cycle of 20ms. In the first whole-body motion reference information generated by the motion reference generation model, the root position increment corresponding to the root motion reference information is 0.008m, and the root pose increment is 0.004rad. The robot control method determines the root synchronization parameters based on the target linear velocity of 0.15m / s and the target angular velocity of 0.30rad / s, for example, determining the root position synchronization parameter to be 0.85 and the root pose synchronization parameter to be 0.85, where the root position synchronization parameter and the root pose synchronization parameter represent the proportion of the desired root motion reference information in the fusion result. Subsequently, the robot control method fuses the root position increment of 0.008m with the desired root position increment of 0.003m to obtain a corrected root position increment of approximately 0.00375m; and fuses the root attitude increment of 0.004rad with the desired root attitude increment of 0.006rad to obtain a corrected root attitude increment of approximately 0.0057rad. Finally, the robot control method combines the corrected root position increment and the corrected root attitude increment with the robot body motion reference information in the first whole-body motion reference information to obtain the second whole-body motion reference information.

[0050] In some embodiments, determining the desired root motion reference information based on the target motion information may include: performing root motion estimation based on the target motion information to obtain the desired root motion reference information.

[0051] In some embodiments, the root motion reference information may include root motion state information and / or root motion change information. The root motion state information includes at least one of root position, root attitude, root velocity, and root angular velocity. The root motion change information includes at least one of position increment, attitude increment, velocity change, and angular velocity change relative to the current root state.

[0052] In some embodiments, the desired root motion reference information may include desired root motion state information and / or desired root motion change information. The desired root motion state information may include at least one of desired root position, desired root attitude, desired root velocity, and desired root angular velocity. The desired root motion change information may include at least one of desired position increment, desired attitude increment, desired velocity change, and desired angular velocity change relative to the current root state.

[0053] Root motion estimation refers to calculating the expected motion state or expected motion change of the robot's root within the current or future control cycles based on target motion information. Root motion estimation can be performed based on target linear velocity, target angular velocity, current root state, and control cycle. For example, the expected position increment can be calculated based on the target linear velocity and control cycle, the expected attitude increment can be calculated based on the target angular velocity and control cycle, or the expected root velocity can be directly determined based on the target linear velocity, or the expected root angular velocity can be directly determined based on the target angular velocity.

[0054] Root motion reference information and expected root motion reference information can be root motion information obtained from the model output side and the target intent side, respectively. Root motion reference information can originate from the first whole-body motion reference information and is used to characterize the root motion predicted by the motion reference generation model; expected root motion reference information can originate from the root motion calculation results of the target motion information and is used to characterize the motion that the robot is expected to execute at the root level based on the operational input information. Both can correspond to the same root node and can use the same or mutually matching data representation forms for subsequent fusion processing according to root synchronization parameters.

[0055] Root motion state information and expected root motion state information can be information describing the motion state of the root at a certain moment or a future moment. Root motion state information can include one or more of root position, root attitude, root velocity, and root angular velocity, used to represent the root state predicted by the motion reference generation model; expected root motion state information can include one or more of expected root position, expected root attitude, expected root velocity, and expected root angular velocity, used to represent the expected root state calculated based on target motion information. For example, root position can represent the root spatial position predicted by the model, and expected root position can represent the target root position calculated based on the target linear velocity, the current root position, and the control cycle.

[0056] Root motion change information and expected root motion change information can be information describing the amount of motion change of the root relative to the current root state. Root motion change information can include one or more of position increments, attitude increments, velocity changes, and angular velocity changes, used to represent the root change predicted by the motion reference generation model. Expected root motion change information can include one or more of expected position increments, expected attitude increments, expected velocity changes, and expected angular velocity changes, used to represent the expected root change calculated based on target motion information. For example, the expected position increment can be determined based on the product of the target linear velocity and the control cycle, and the expected attitude increment can be determined based on the product of the target angular velocity and the control cycle, thus characterizing the motion change that the robot root is expected to produce within the current control cycle based on the operational input information.

[0057] Root velocity refers to the translational velocity of the root node, also known as root linear velocity, and is used to characterize how quickly the root node's position changes in space. Root velocity can include one or more of the following: forward / backward velocity, lateral velocity, and vertical velocity. It can also be expressed as a velocity vector containing multiple directional velocity components. Root angular velocity characterizes how quickly the root node's attitude changes, and may include one or more of the following: yaw rate, pitch rate, and roll rate.

[0058] The current root state refers to the motion state information of the robot's root at the beginning of the current control cycle. The current root state can include one or more of the following: current root position, current root orientation, current root velocity, and current root angular velocity. The current root state can serve as the starting point for root motion estimation, used to calculate desired root motion state information or desired root motion change information.

[0059] The control cycle refers to the time interval corresponding to one control calculation performed by the robot control method. The control cycle can be 20ms, 10ms, 5ms, etc. It is used to convert velocity information into position changes and angular velocity information into attitude changes. For example, when the control cycle is 20ms and the target linear velocity is 0.15m / s, the expected position increment can be 0.003m; when the control cycle is 20ms and the target angular velocity is 0.30rad / s, the expected attitude increment can be 0.006rad.

[0060] This embodiment calculates root motion based on target motion information, converting target linear velocity, target angular velocity, and other motion intentions into desired root motion reference information, thus improving the correspondence between target motion information and root motion control. By dividing the desired root motion reference information into desired root motion state information and desired root motion change information, it can adapt to the input requirements of different control links for absolute state or relative changes, expanding the applicability of the desired root motion reference information. By calculating root motion based on the current root state and control cycle, the desired position increment, desired attitude increment, desired velocity change, and desired angular velocity change can be matched with the current control cycle, improving the continuity and feasibility of robot root motion calculation. Addressing the problem that operational input information is difficult to directly form robot root motion reference, this embodiment adopts the concept of "calculating desired root motion state information and / or desired root motion change information based on target motion information," enabling the robot control method to obtain desired root motion reference information corresponding to the operational intention and providing a clear reference basis for subsequent root synchronization correction or motion control.

[0061] In one example, the robot control cycle is 20 ms, the current root position is (1.000m, 0.000m, 0.900m), the current root yaw angle is 10°, the current root forward velocity is 0.05 m / s, and the current root yaw angular velocity is 0.10 rad / s. Target motion information includes the target linear velocity of 0.15 m / s and the target angular velocity of 0.30 rad / s. Based on the target linear velocity of 0.15 m / s and the control cycle of 20 ms, the robot control method calculates the desired position increment as 0.003 m; based on the target angular velocity of 0.30 rad / s and the control cycle of 20 ms, it calculates the desired attitude increment as 0.006 rad. The robot control method can also calculate the desired root position as (1.003m, 0.000m, 0.900m) based on the current root position and the desired position increment; and calculate the desired root yaw angle as approximately 10.34° based on the current root yaw angle and the desired attitude increment. In addition, the robot control method can take the target linear velocity of 0.15 m / s as the desired root velocity and the target angular velocity of 0.30 rad / s as the desired root angular velocity, and calculate the desired velocity change as 0.10 m / s and the desired angular velocity change as 0.20 rad / s.

[0062] In some embodiments, the root synchronization parameters may include root position synchronization parameters and root attitude synchronization parameters.

[0063] In some embodiments, determining the root synchronization parameters based on the target linear velocity and / or target angular velocity corresponding to the target motion information may include: determining an equivalent velocity based on the larger of the equivalent linear velocity corresponding to the target linear velocity and the equivalent linear velocity corresponding to the target angular velocity; determining a velocity ratio parameter based on the ratio of the equivalent velocity to a preset velocity; and determining root position synchronization parameters for root position synchronization based on the difference between 1 and the cube of the velocity ratio parameter.

[0064] In some embodiments, determining the root synchronization parameters based on the target linear velocity and / or target angular velocity corresponding to the target motion information may include: determining the root attitude synchronization parameters based on the root position synchronization parameters, wherein the root attitude synchronization parameters are greater than zero.

[0065] The root position synchronization parameter refers to a parameter used to control the fusion ratio between the root position reference information and the desired root position reference information. A larger root position synchronization parameter results in a root position or increment that is closer to the desired root position or increment; a smaller parameter results in a root position or increment that is closer to the root position or increment output by the motion reference generation model. The root position synchronization parameter and the target linear velocity can have a negative correlation. For example, when the target linear velocity is low, a larger root position synchronization parameter can be set to make the root position more closely follow the changes in the desired root position calculated from the target motion information; when the target linear velocity is high, a smaller parameter can be set to make the root position retain more of the natural motion results generated by the model.

[0066] Root attitude synchronization parameters refer to parameters used to control the fusion ratio between root attitude reference information and desired root attitude reference information. Root attitude synchronization parameters can be used for the synchronous correction of information such as root rotation, root orientation, root attitude increment, or root angular velocity. Root attitude synchronization parameters can be the same as root position synchronization parameters, or they can be determined based on root position synchronization parameters and a preset minimum value. For example, the larger of the root position synchronization parameter and the preset minimum value can be determined as the root attitude synchronization parameter, ensuring that the root attitude synchronization parameter remains greater than zero even when the root position synchronization parameter is small. The preset minimum value can be determined based on at least one of the following: robot structural stability requirements, root attitude drift tolerance, target angular velocity range, gait state, tracking capability of the reference trajectory tracking controller, simulation test results, and actual machine calibration results. As an example, the preset minimum value can be 0.05, 0.10, 0.12, or 0.15. For example, the root attitude synchronization parameter can be set to greater than or equal to 0.12, so that the robot retains a certain root attitude synchronization effect even when moving at high speed, thereby reducing the cumulative drift of root yaw or tilt during long-term movement.

[0067] Equivalent linear velocity refers to the parameter obtained by converting the target angular velocity into a velocity quantity corresponding to the dimension of linear velocity. Equivalent linear velocity is used to measure the impact of turning motion on the root synchronization strength. For example, the equivalent linear velocity can be obtained by multiplying the target angular velocity by a preset conversion factor, which can be 0.5m. Using equivalent linear velocity, even if the robot's target linear velocity is small or zero, as long as the target angular velocity is large, it can still reflect that the robot has significant turning motion, thereby reducing the situation where the root position is overly constrained by synchronization in stationary turning scenarios.

[0068] Equivalent velocity refers to the velocity parameter obtained by combining the equivalent linear velocities corresponding to the target linear velocity and the target angular velocity. The equivalent velocity can be determined based on the larger of the target linear velocity and the equivalent linear velocity corresponding to the target angular velocity. For example, when the target linear velocity is 0.10 m / s, the target angular velocity is 0.80 rad / s, and the preset conversion factor is 0.5 m, the equivalent linear velocity is 0.40 m / s, and this value can be taken as 0.40 m / s. The equivalent velocity is used to subsequently determine the velocity scaling parameters, ensuring that the root synchronization parameters simultaneously respond to translational and directional motions.

[0069] The preset speed refers to a speed threshold used to normalize the equivalent speed. The preset speed can be determined based on the robot's low-speed motion range, the root output characteristics of the model, or engineering calibration results. For example, the preset speed could be 0.5 m / s. When the equivalent speed is less than the preset speed, the robot can be considered to be in a low-speed or low-to-medium-speed state requiring strong root synchronization; when the equivalent speed is close to or greater than the preset speed, the root position synchronization effect can be gradually reduced, allowing the root position to retain more of the natural motion results generated by the motion reference generation model.

[0070] The speed proportional parameter can be a normalized parameter determined based on the ratio of the equivalent speed to the preset speed and after amplitude limiting. The value of the speed proportional parameter can range from 0 to 1. Specifically, when the equivalent speed is less than the preset speed, the speed proportional parameter can be equal to the ratio of the equivalent speed to the preset speed; when the equivalent speed is greater than or equal to the preset speed, the speed proportional parameter can be equal to 1. For example, when the equivalent speed is 0.15 m / s and the preset speed is 0.5 m / s, the speed proportional parameter can be 0.3; when the equivalent speed is 0.60 m / s and the preset speed is 0.5 m / s, the speed proportional parameter can be 1.

[0071] Determining the root position synchronization parameter based on the difference between 1 and the cube of the velocity proportionality parameter can refer to using a cubic curve to determine the root position synchronization parameter. For example, if the velocity proportionality parameter is t, then the root position synchronization parameter can be 1-t. 3 This method results in a larger root position synchronization parameter at low speeds, thereby enhancing the influence of the desired root position reference information. As the speed ratio parameter increases, the root position synchronization parameter decreases rapidly, allowing the root position to retain more of the natural motion results generated by the motion reference generation model at high speeds.

[0072] The preset minimum value can be 0.05, 0.10, 0.12, 0.15, etc. By setting the preset minimum value, even if the root position synchronization parameter is close to 0 at high speed, the root attitude synchronization parameter still retains a certain synchronization effect, thereby reducing the risk of continuous drift of the root attitude during long-term high-speed movement, continuous turning, or complex gait operation.

[0073] This embodiment determines the equivalent velocity based on the larger of the target linear velocity and the equivalent linear velocity corresponding to the target angular velocity. This allows both translational and turning motion intensity to be incorporated into the root synchronization parameter calculation, reducing the risk of excessive synchronization constraints on the root position due to a small target linear velocity in stationary turning scenarios. By determining the velocity ratio parameter based on the ratio of the equivalent velocity to a preset velocity, different motion velocities can be uniformly mapped to a range that facilitates the calculation of synchronization intensity, improving the stability of the root synchronization parameter calculation. By determining the root position synchronization parameter based on the difference between 1 and the cube of the velocity ratio parameter, the fusion ratio of the desired root position reference information can be increased at low speeds, and the root position synchronization intensity can be rapidly reduced as the speed increases, thus balancing low-speed controllability and high-speed naturalness. By setting the root attitude synchronization parameter to be greater than zero, the root attitude synchronization function can be retained even when the root position synchronization intensity is low, reducing attitude drift such as root yaw and roll. This embodiment addresses the issues of requiring strong root synchronization for low-speed motion, preserving the model's natural output for high-speed motion, and avoiding excessive restrictions on zero linear velocity during in-situ turning. It adopts the concept of "determining the equivalent velocity jointly by the target linear velocity and the target angular velocity, determining the root position synchronization parameters based on the cubic curve, and setting the root attitude synchronization parameters to be greater than zero," thereby improving the adaptability of root synchronization correction to different scenarios such as low-speed movement, in-situ turning, and high-speed motion.

[0074] In one example, the preset velocity is 0.5 m / s, the target linear velocity is 0.15 m / s, the target angular velocity is 0.30 rad / s, and the preset conversion coefficient corresponding to the target angular velocity is 0.5 m. The robot control method first converts the target angular velocity 0.30 rad / s into an equivalent linear velocity of 0.15 m / s, and determines the equivalent velocity as 0.15 m / s based on the larger of the target linear velocity 0.15 m / s and the equivalent linear velocity 0.15 m / s. Subsequently, the robot control method calculates the velocity proportionality parameter as 0.15 / 0.5 = 0.3, and determines the root position synchronization parameter as 1 - 0.3 based on the difference between 1 and the cube of the velocity proportionality parameter. 3 =0.973. If the preset minimum value is 0.12, then the root attitude synchronization parameter can be taken as 0.973.

[0075] In another example, the preset velocity is 0.5 m / s, the target linear velocity is 0 m / s, the target angular velocity is 0.80 rad / s, and the preset conversion factor corresponding to the target angular velocity is 0.5 m. The robot control method converts the target angular velocity of 0.80 rad / s into an equivalent linear velocity of 0.40 m / s and determines the equivalent velocity as 0.40 m / s. The velocity scaling parameter is 0.40 / 0.5 = 0.8, and the root position synchronization parameter is 1 - 0.8. 3=0.488. If the preset minimum value is 0.12, then the root attitude synchronization parameter can be 0.488. In this example, even if the target linear velocity is 0, the target angular velocity still makes the equivalent velocity reach 0.40 m / s, thereby reducing the root position synchronization parameter to 0.488 and reducing the situation where the stationary turning steps are overly constrained.

[0076] In another example, the preset velocity is 0.5 m / s, the target linear velocity is 0.70 m / s, the target angular velocity is 0.10 rad / s, and the preset conversion factor corresponding to the target angular velocity is 0.5 m. The equivalent linear velocity corresponding to the target angular velocity is 0.05 m / s, and the equivalent velocity is taken as 0.70 m / s. The velocity scaling parameter can be limited to 1, and the root position synchronization parameter is 1-1. 3 =0. If the preset minimum value is 0.12, then the root pose synchronization parameter can be 0.12. In this example, during high-speed movement, the root position retains more of the natural motion result generated by the motion reference generation model, while the root pose still retains the synchronization effect of 0.12.

[0077] In some embodiments, the operation input information may include at least one of forward / backward direction input information, lateral direction input information, and steering input information.

[0078] In this embodiment, the operation input information can be used to describe the direction of motion, speed of motion, direction of turn, or intensity of turn that the operator wants the robot to perform. For handle control scenarios, the operation input information may include one or more of forward / backward direction input information, lateral direction input information, and turn input information, so as to convert the operator's low-dimensional control intention into target motion information that can be used by the robot in subsequent motion control processes.

[0079] Forward and backward input information refers to input information used to characterize the robot's intention to move in the forward and backward directions. Forward and backward input information can be used to determine whether the robot moves forward, backward, or remains stationary in the forward and backward directions.

[0080] Lateral input information refers to input information used to characterize the robot's intention to move in the left-right or lateral direction. Lateral input information can be used to determine whether the robot will move laterally to the left, to the right, or remain laterally stationary.

[0081] Steering input information refers to the input information used to characterize the robot's intention to change its orientation. Steering input information can be used to determine whether the robot should turn left, right, or maintain its current orientation in the vertical direction.

[0082] This embodiment, by setting the operation input information to include at least one of forward / backward direction input information, lateral direction input information, and turning input information, can cover various operation scenarios such as forward, backward, lateral movement, turning in place, and walking / turning, thereby improving the robot control method's ability to express different operation intentions. By separately setting forward / backward direction input information, lateral direction input information, and turning input information, translational movement intentions and turning movement intentions can be expressed separately, reducing the risk of confusion between different movement intentions. By allowing the operation input information to include at least one of the above input information, it can adapt to the input dimensions of different operating devices and the movement capabilities of different robots, thus improving the applicability of the robot control method.

[0083] In one example, the input information comes from the handle. The values ​​for forward / backward input, lateral input, and steering input range from -1 to 1. If the forward / backward input is 0.40, the lateral input is 0.00, and the steering input is 0.20, the robot control method can determine that the robot intends to move forward and make a slight turn, for example, a target forward speed of 0.30 m / s and a target angular velocity of 0.20 rad / s. If the forward / backward input is 0.00, the lateral input is 0.35, and the steering input is 0.00, the robot control method can determine that the robot intends to move laterally, for example, a target lateral speed of 0.15 m / s. If the forward / backward input is 0.00, the lateral input is 0.00, and the steering input is 0.50, the robot control method can determine that the robot intends to turn in place, for example, a target angular velocity of 0.50 rad / s.

[0084] In some embodiments, the process of determining the target motion information may include: performing at least one of dead zone processing, speed level determination, and gait state determination based on the operation input information to obtain the target motion information.

[0085] Dead-zone processing refers to the filtering, zeroing, or reduction of response of input components with small amplitudes in the operational input information based on a preset dead-zone threshold. Since handles, joysticks, or other operating devices may have zero-point offsets or slight jitters when not actively operated, directly determining the target motion information based on these small input components may cause the robot to wobble slightly or move erroneously. Dead-zone processing treats input components smaller than the dead-zone threshold as invalid inputs. For example, when the dead-zone threshold is 0.05, the lateral input information of 0.02 can be zeroed, while the forward / backward input information of 0.20 can be retained, thus making the target motion information more consistent with the actual operational intent.

[0086] Speed ​​level determination refers to the process of converting continuous operational input information into discrete or semi-discrete speed level information. Speed ​​level information can include stationary level, low-speed walking level, medium-speed walking level, high-speed walking level, backward walking level, lateral movement level, and stationary turning level. Speed ​​level information not only characterizes the magnitude of the robot's desired movement speed but also serves as the basis for gait state determination and query feature updates. For example, robot control methods can determine the robot's gait state when the speed level information is stable and generate or update query features of the input motion reference generation model based on the stabilized speed level information. Through speed level determination, continuously changing operational input information can be converted into a more stable expression of movement intent. For example, input information in the forward / backward direction between 0.05 and 0.25 can be determined as a low-speed walking level; input information in the forward / backward direction between 0.25 and 0.60 can be determined as a medium-speed walking level; and input information in the forward / backward direction greater than 0.60 can be determined as a high-speed walking level. For example, after maintaining a preset number of control cycles or a preset duration at a speed level, the query features can be updated based on that speed level, thereby suppressing the impact of high-frequency jitter on the motion reference information generation process.

[0087] In some embodiments, velocity level information can be used as part of the target motion information or to determine relevant parameters within the target motion information. For example, when the velocity level information is a low-speed walking level, the target motion information may include the target linear velocity corresponding to low-speed walking and the future root trajectory; when the velocity level information is a stationary turning level, the target motion information may include the target angular velocity corresponding to turning and the future root orientation change. Thus, velocity level determination is not a separate input preprocessing step, but rather a means to stably convert operational input information into a target motion intent usable by the motion reference generation model.

[0088] Gait state determination refers to the processing of information such as operational input and / or velocity level to determine the appropriate gait state for a robot. Gait state information can include standing, forward walking, backward walking, lateral movement, stationary turning, and walking-turning states. Gait state determination can be completed by combining forward / backward, lateral, and turning input information, provided the velocity level information is stable. For example, if the forward / backward input is greater than the dead zone threshold and the turning input is small, it can be determined as a forward walking state; if the forward / backward input is small and the turning input is large, it can be determined as a stationary turning state; if both the forward / backward and turning inputs are large, it can be determined as a walking-turning state. Gait state information can be used as part of the target motion information or to determine the gait label, future root trajectory, or future root orientation in the query features of the input motion reference generation model, enabling the motion reference generation model to generate corresponding whole-body motion reference information based on different gait states.

[0089] Target motion information can refer to the robot's desired motion information obtained after processing at least one of dead-zone processing, velocity level determination, and gait state determination based on operational input information. Target motion information may include one or more of the following: target linear velocity, target angular velocity, target direction of movement, target turning direction, velocity level information, gait state information, and desired motion trajectory information. Target motion information is used to convert operational input information into a motion intent expression that can be used by the robot control process, enabling subsequent motion reference generation models or robot motion control processes to generate motion reference information or robot control data that matches the operational intent based on the target motion information. In some embodiments, target motion information can also be used to construct query features for the motion reference generation model, and the refresh timing of the query features can be controlled according to velocity level information, thereby reducing frequent changes in motion reference information caused by input jitter.

[0090] This embodiment uses dead-zone processing on the input information to filter out zero-point offsets and minor jitters in the operating device, reducing the risk of erroneous movements caused by invalid input. By determining the speed level, continuous input information can be converted into more stable speed level information, reducing frequent switching near speed boundaries. By determining the gait state, different operational intentions such as forward, backward, lateral, and turning can be converted into corresponding robot motion modes, enabling the motion reference generation model to generate matching full-body motion reference information based on the gait state. This embodiment addresses the problems of potential noise in the input information, the tendency for continuous input values ​​to jitter near boundaries, and the difficulty of directly using low-dimensional inputs for robot motion reference generation. It adopts a concept combining dead-zone processing, speed level determination, and gait state determination to obtain more stable target motion information that better matches the operational intention, thereby improving the stability of the robot's subsequent motion reference information generation and motion control processes.

[0091] In one example, the robot control method receives handle inputs: forward / backward input is 0.22, lateral input is 0.03, and steering input is 0.18, with a dead zone threshold set to 0.05. After dead zone processing, the robot control method sets the lateral input of 0.03 to zero, retaining the forward / backward input of 0.22 and the steering input of 0.18. Next, the robot control method determines the speed level: when the forward / backward input is between 0.05 and 0.25, it corresponds to a low-speed walking level; therefore, the forward / backward input of 0.22 corresponds to a low-speed walking level, and the target linear velocity can be determined to be 0.15 m / s. The robot control method further determines the gait state: since the forward / backward input is greater than the dead zone threshold and the steering input is also greater than the dead zone threshold, the gait state can be determined to be low-speed forward walking and steering, and the target angular velocity is determined to be 0.20 rad / s. If the low-speed walking level and the gait state of low-speed forward walking and turning are maintained continuously for 3 control cycles, the robot control method can update the query features of the input motion reference generation model based on the target linear velocity of 0.15 m / s, the target angular velocity of 0.20 rad / s, and the gait state of low-speed forward walking and turning. The query features can be determined based on the target motion information, which may include the target linear velocity of 0.15 m / s, the target angular velocity of 0.20 rad / s, the low-speed walking level, and the gait state of low-speed forward walking and turning.

[0092] In some embodiments, the motion reference generation model may include a feature mapping network, a temporal recursion network, and a pose decoding network.

[0093] In some embodiments, generating first whole-body motion reference information based on the target motion information and the current motion state information through a motion reference generation model may include: constructing query features based on the target motion information and the current motion state information; mapping the query features to latent space information through the feature mapping network; performing temporal recursion on the latent space information through the temporal recursion network; and decoding the temporally recursed latent space information through the attitude decoding network to obtain the first whole-body motion reference information.

[0094] In this embodiment, the motion reference generation model may include multiple networks, such as a feature mapping network, a temporal recursion network, and a pose decoding network. The motion reference generation model can be used to convert the low-dimensional motion intent corresponding to the operation input information into first whole-body motion reference information corresponding to multiple motion nodes such as the robot root, torso, legs, feet, and arms.

[0095] Query features can refer to the feature set input to the feature mapping network. Query features can be constructed from target motion information and current motion state information. Query features can be used to describe the combined relationship between "the robot's current body state" and "the robot's future motion intention," enabling the motion reference generation model to generate full-body motion reference information that conforms to the current posture and the target's motion intention.

[0096] Feature mapping networks can be defined as networks used to map query features to latent space information. Feature mapping networks can compress or transform input features of different dimensions and types into latent space information, enabling subsequent temporal recursion networks to perform motion state recursion in a more continuous and compact latent space.

[0097] Latent space information refers to the intermediate motion representation output by a feature mapping network. Latent space information can be in vector form, such as a 32-dimensional vector, or it can be a continuous feature representation of other dimensions. Latent space information does not necessarily correspond directly to a specific joint angle or root position, but rather represents a compressed representation of the robot's motion state within the model. Using latent space information can reduce the complexity of temporal prediction directly in a high-dimensional whole-body pose space.

[0098] A temporal recursive network is a network used to recursively predict latent space information over time. Based on the latent space information at the current moment, a temporal recursive network can predict the latent space information at the next moment or multiple subsequent moments, thus forming a continuous motion evolution process. The role of a temporal recursive network is to maintain the temporal continuity of motion reference information, enabling the robot to have a smoother whole-body motion reference result when transitioning from the current state to subsequent motion states.

[0099] A posture decoding network can be defined as a network used to decode temporally recursively derived latent space information into first whole-body motion reference information. Through a posture decoding network, a more abstract motion representation in latent space can be converted into whole-body motion reference information usable in the robot control process.

[0100] This embodiment, by setting up a feature mapping network, a temporal recursion network, and a posture decoding network, can complete the query feature mapping, latent space temporal recursion, and whole-body posture decoding in stages. This reduces the processing difficulty of directly generating high-dimensional whole-body motion reference information from target motion information and improves the continuity of the generation process. By constructing query features based on target motion information and current motion state information, the motion reference generation model can simultaneously perceive the robot's current body state and future motion intentions, thereby improving the matching degree between the first whole-body motion reference information and the current motion state and target motion information.

[0101] To improve the performance of the motion reference generation model in scenarios where the model transitions from standing to walking, in some embodiments, the training samples of the feature mapping network may include spliced ​​training samples; the query features of the spliced ​​training samples may include ontological pose features from standing state samples, and future trajectory features and / or gait features from walking state samples; wherein, the query features of the spliced ​​training samples may be features obtained by concatenating features in the target local feature space.

[0102] Chimeric training samples refer to training samples obtained by combining feature fragments from samples of different motion states. Chimeric training samples can be used to train feature mapping networks, enabling them to handle input scenarios where "the current body posture is still close to standing, but the future trajectory already represents walking intention." Chimeric training samples can include body posture features from standing state samples, and future trajectory features and / or gait features from walking state samples. Through chimeric training samples, the feature mapping network can be exposed to feature combinations that more closely resemble real-world operational inputs during the standing-to-walking transition phase during training.

[0103] Body posture features, future trajectory features, and gait features can be different categories of query features. Body posture features can be used to describe the robot's current body state, such as left foot position, right foot position, left foot velocity, right foot velocity, hip velocity, etc. Future trajectory features can be used to describe the robot's expected movement trend at several future moments, such as future root trajectory position, future root orientation, future direction of movement, etc. Gait features can be used to describe movement patterns such as walking, running, standing, and turning. These features collectively characterize the relationship between the robot's current posture and future movement intentions.

[0104] The target local feature space can refer to a local coordinate or normalized feature space used to uniformly represent features from different sources in the spliced ​​training samples. The target local feature space can be the feature space corresponding to the root coordinate system or the feature space corresponding to the simulated root coordinate system. By stitching the ontological pose features from standing state samples and the future trajectory features from walking state samples into the target local feature space, the impact of differences in world coordinates, orientation, or scale between different samples on the construction of training samples can be reduced.

[0105] This embodiment uses chimeric training samples to train the feature mapping network, which enables the feature mapping network to learn the combination relationship between the body posture in the standing state and the future trajectory in the walking state, reducing posture confusion when the robot starts or when it switches from standing to walking.

[0106] To ensure that training samples of preset category velocity segments appear more frequently during training, in some embodiments, the temporal recursive network and / or the pose decoding network can be trained based on training samples in a training sample set. The process of determining the training samples may include: during training, dividing the training sample set into multiple velocity segments according to the root motion velocity, and selecting training samples from the training sample set for training the temporal recursive network and / or the pose decoding network based on the sampling weights corresponding to each velocity segment. Wherein, the sampling weights corresponding to at least one preset category velocity segment can be set to be greater than sampling weights determined according to the proportion of the number of samples in the training sample set.

[0107] In some embodiments, the preset category of speed segments can satisfy a preset low-speed condition. In this case, the preset category of speed segments can be low-speed segments.

[0108] The training sample set can be a dataset used to train a temporal recursive network and / or a pose decoding network. The training sample set can include multiple training samples. Training samples can be samples selected from the training sample set according to a sampling strategy. For example, by adjusting the sampling weights of different velocity segments, the frequency of occurrence of different training samples during training can be changed.

[0109] Root motion velocity and velocity segmentation can be used as a basis for dividing the training sample set. Velocity segmentation can include low-speed segmentation, low-to-medium speed segmentation, medium-speed segmentation, high-speed segmentation, etc.

[0110] Sampling weights refer to the weights used when selecting training samples from different speed segments. The larger the sampling weight, the higher the frequency of the training sample in the corresponding speed segment during the training process.

[0111] Pre-defined velocity segments refer to velocity segments that are pre-assigned to be prioritized for training among multiple velocity segments. These pre-defined velocity segments can be determined based on weak points in robot motion control, sparse sample scenarios, or scenarios requiring focused optimization. By setting pre-defined velocity segments, the training process can be tailored to increase the frequency of training samples corresponding to specific motion scenarios.

[0112] Preset low-speed conditions refer to the conditions used to determine whether a speed segment belongs to a low-speed segment. Preset low-speed conditions can be determined based on at least one of the root movement velocity, target linear velocity, and root turning velocity. For example, preset low-speed conditions can include a root movement velocity less than 0.2 m / s, or a root movement velocity less than 0.1 m / s and a root turning velocity greater than 0.5 rad / s. By using preset low-speed conditions, scenarios prone to insufficient samples or unstable generation, such as slow walking, low-speed starts, and turning in place, can be included in the key training scope.

[0113] Root motion velocity can refer to a velocity scalar determined based on the root velocity, used to segment training samples according to their magnitude. For example, if the root velocity includes forward and backward velocities and lateral velocities, then the root motion velocity can be the composite velocity of the forward and backward velocities and the lateral velocities.

[0114] Root steering speed can refer to a steering speed scalar determined based on root angular velocity, used to characterize how fast the root rotates around a preset steering axis. For example, if the root angular velocity includes yaw rate, pitch rate, and roll rate, then the root steering speed can be determined based on the yaw rate.

[0115] Setting the sampling weight to be greater than the proportion determined by the number of samples in the training sample set can refer to enhancing the sampling of speed segments of a preset category, so that training samples in that speed segment appear more frequently during training than their original proportion. For example, the proportion of low-speed segments in the training sample set is 0.08, but the sampling weight of low-speed segments can be set to 0.20 during training. This can increase the exposure of low-speed training samples during training and improve the problem of low-speed motion samples being overwhelmed by medium- and high-speed motion samples.

[0116] This embodiment can increase the frequency of training samples of the preset category of speed segment in the training process by dividing the speed segment according to the root movement speed and selecting training samples based on sampling weight. In the case that the preset category of speed segment is a low speed segment, it can improve the problem of unstable generation of low speed reference information caused by the low speed training samples being submerged by medium and high speed training samples.

[0117] This embodiment addresses the challenges of directly generating high-dimensional whole-body motion reference information from low-dimensional target motion information, the lack of natural samples in the standing-to-walking transition scenario, and insufficient exposure to low-speed training samples. It adopts the concept of "three-network phased generation, chimeric training sample enhancement of feature mapping, velocity segmented weighted sampling to enhance temporal recursion, and posture decoding training," thereby improving the performance of the first whole-body motion reference information in terms of continuity, low-speed stability, and robustness in the standing-to-walking transition.

[0118] In one example, the motion reference generation model includes a feature mapping network, a temporal recursion network, and a pose decoding network. The robot control method acquires target motion information and current motion state information in the current control cycle and constructs 28-dimensional query features. Dimensions 0 to 14 represent body pose features, including left foot position, right foot position, left foot velocity, right foot velocity, and hip velocity; dimensions 15 to 26 represent future trajectory features, including the root trajectory position and root orientation in the 20th, 40th, and 60th frames; and dimension 27 represents gait features, with a value of 1.0 for walking and 0.0 for running. The feature mapping network maps the 28-dimensional query features to 32-dimensional latent space information, the temporal recursion network performs temporal recursion on the 32-dimensional latent space information, and the pose decoding network decodes the temporally recursed latent space information into first whole-body motion reference information.

[0119] When training the feature mapping network, the proportion of the chimeric training samples in the training samples of the feature mapping network is, for example, 15%. The 0 to 14-dimensional ontological pose features of the chimeric training samples are taken from standing state samples with root motion speed less than 0.05 m / s, and the 15 to 27-dimensional future trajectory features and gait features are taken from walking state samples with root motion speed between 0.15 m / s and 1.0 m / s, and are spliced ​​in the target local feature space corresponding to the root coordinate system.

[0120] When training the temporal recursive network and the pose decoding network, the training sample set can be divided into speed segments such as 0m / s to 0.2m / s, 0.2m / s to 0.5m / s, 0.5m / s to 1.0m / s, and 1.0m / s to 1.75m / s according to the root motion speed. Training samples for training the temporal recursive network and the pose decoding network are selected from the training sample set based on the sampling weights corresponding to each speed segment. For example, the sampling weight of the low-speed segment (0m / s to 0.2m / s) can be set to 0.20. This sampling weight is greater than the proportion of samples in this low-speed segment in the training sample set (0.08), thereby increasing the frequency of low-speed training samples during training.

[0121] In some embodiments, the training sample set of the motion reference generation model can be constructed based on the motion data of the target robot skeleton.

[0122] In some embodiments, the target robot skeleton motion data may include robot usable skeleton motion data obtained by skeleton retargeting of raw motion capture skeleton motion data from different motion capture systems.

[0123] In some embodiments, the skeleton relocation process may include adapting to at least one of different joint numbers, different rotation sequences, and different node channel layouts.

[0124] In this embodiment, the skeleton structure, joint naming, rotation order, and node channel layout of the training data for the motion reference generation model need to be as uniform as possible. If the data formats output by different motion capture systems differ significantly, the motion reference generation model will find it difficult to directly learn a consistent motion representation. Therefore, the motion reference generation model can be trained based on the target robot's skeleton motion data.

[0125] Target robot skeleton motion data refers to motion data organized according to the target robot skeleton structure, corresponding to the target robot skeleton. Target robot skeleton motion data can include one or more of the following: position information, posture information, velocity information, contact state information, and gait information of multiple robot motion nodes. For example, the number of joints, rotation order, and node channel layout of the target robot skeleton motion data are uniform. By constructing a training sample set for a motion reference generation model based on the target robot skeleton motion data, the impact of data differences between different motion capture systems on model training can be reduced.

[0126] A motion capture system refers to a system used to collect motion data from humans, robots, or other objects. Different motion capture systems may employ different numbers of skeletal nodes, different rotation sequences, and different node channel layouts. For example, one motion capture system might output 23 joints with a rotation sequence of first around the Y-axis, then around the X-axis, and then around the Z-axis; another might output 78 joints with a rotation sequence of first around the Z-axis, then around the Y-axis, and then around the X-axis. Without adaptation, the raw motion data of the skeletons output by different motion capture systems cannot be directly merged for training.

[0127] Raw motion capture skeleton data refers to the motion data output by the motion capture system corresponding to the original motion capture skeleton. Raw motion capture skeleton data can include skeleton hierarchy, joint names, joint offsets, root node positions for each frame, joint rotation angles for each frame, end-effector information, etc. Raw motion capture skeleton data can use a hierarchical skeleton motion data format, or other data formats capable of expressing skeleton structure and frame-by-frame motion posture. Since raw motion capture skeleton data may be used for human animation, game characters, or different robot skeletons, it can undergo skeleton retargeting processing before being used for robot control.

[0128] Skeleton retargeting refers to the process of converting raw motion data from the original motion-captured skeleton into motion data for the target robot skeleton. Skeleton retargeting can include joint mapping, skeleton scaling, root node translation, foot grounding, forward axis detection, rotational order conversion, node channel adaptation, joint naming conversion, unnecessary bone removal, and mirror data augmentation. Through skeleton retargeting, raw motion data from different motion-capture systems can be converted to a unified target robot skeleton, thus forming usable skeleton motion data for the robot.

[0129] Available skeleton motion data for a robot refers to skeleton motion data that can be used by the robot motion reference generation model, robot motion controller, or downstream motion processing module. Available skeleton motion data does not need to be completely identical to the original motion-captured skeleton, but rather needs to match the motion nodes, degrees of freedom, and control link input requirements of the target robot skeleton. For example, available skeleton motion data can include motion nodes such as root nodes, hips, spine, head, shoulders, arms, legs, feet, and toes, organized according to a node order and channel format recognizable by the target robot.

[0130] Adaptation to different joint numbers refers to the process of converting raw motion-captured skeletons with different joint numbers into target robot skeletons. Different motion-capture systems may output 24 bone nodes, 23 joints, 65 joints, 78 joints, or other numbers of skeleton nodes. Adaptation to different joint numbers may include deleting bone nodes that are not needed by the target robot, merging bone nodes with similar functions, mapping the joints of the raw motion-captured skeleton to the joints of the target robot skeleton, and retaining end-effector offsets. For example, for raw motion-captured skeleton motion data containing many finger, eye, or head end-effectors, unnecessary bones that are weakly related to the current robot's whole-body walking reference generation can be deleted, allowing the remaining bones to be mapped to a unified target robot skeleton.

[0131] Adaptation to different rotation orders refers to the conversion or standardization of joint rotation representations across different motion capture systems. Different motion capture systems may employ different rotation orders, such as rotating around the Y-axis first, then the X-axis, then the Z-axis, or vice versa. Different rotation orders result in the same Euler angle values ​​representing different actual poses, thus requiring adaptation during skeleton retargeting. Adaptation to different rotation orders may involve parsing the joint poses according to the rotation order of the source data and then converting it to the rotation representation required by the target robot skeleton.

[0132] Adapting different node channel layouts refers to parsing, transforming, and unifying the channel organization of nodes in the original motion capture skeleton data. For example, node channels can include position channels and rotation channels, such as three position channels plus three rotation channels. For this type of data, it is necessary to identify the channel organization of the nodes and convert it into a unified node channel layout for the target robot skeleton.

[0133] Adaptation refers to the transformation, mapping, or standardization performed to address data differences between different motion capture systems. Adaptation can include the number of joints, joint names, joint levels, rotation order, node channel layout, skeleton dimensions, foot orientation, and forward axis direction. The purpose of adaptation is not to preserve all details of the original motion capture skeleton data, but rather to ensure that original motion capture skeleton data from different sources can be mapped to the same usable skeleton representation for the robot, thereby improving the consistency of training data for the motion reference generation model.

[0134] In this embodiment, the training sample set of the motion reference generation model in the robot control method can be constructed based on the target robot skeleton motion data. The target robot skeleton motion data can be obtained by skeleton redirection processing of the original motion capture skeleton motion data from different motion capture systems. Specifically, during the training process of the motion reference generation model, the differences in data structure such as the number of joints, rotation order, and node channel layout in different original motion capture skeleton motion data can be identified first; then, through skeleton redirection processing, the original motion capture skeleton motion data of different motion capture systems can be converted into robot-usable skeleton motion data under the target robot skeleton; finally, the motion reference generation model is trained using the robot-usable skeleton motion data, enabling the motion reference generation model to learn the generation rules of robot whole-body motion reference information based on a unified data representation.

[0135] This embodiment constructs a training sample set for a motion reference generation model based on the target robot skeleton motion data. This reduces the impact of data format differences between different motion capture systems on model training and improves the consistency of training data. By retargeting the original motion capture skeleton motion data from different systems, original motion capture skeleton motion data with different numbers of joints, different rotation orders, and different node channel layouts can be included in the same training path, improving the compatibility of training data sources. Adapting to different numbers of joints reduces the difficulty of joint mapping caused by inconsistencies between the original motion capture skeleton and the target robot skeleton. Adapting to different rotation orders reduces motion posture deviations caused by joint posture resolution errors. Adapting to different node channel layouts ensures that the root position and root posture are consistently expressed in the target robot skeleton motion data. This embodiment addresses the problem that the original motion capture skeleton motion data output by different motion capture systems differ in the number of joints, rotation order, and node channel layout, making it difficult to directly train the motion reference generation model using multi-source original motion capture skeleton motion data. It adopts the concept of "first performing skeleton retargeting processing, and then training the motion reference generation model based on the target robot skeleton motion data", thereby improving the usability of multi-source original motion capture skeleton motion data and the training stability of the motion reference generation model.

[0136] In one example, the first motion capture system outputs raw motion data for a skeleton with 23 joints, rotating in the order of Y-axis, X-axis, and Z-axis. The second motion capture system outputs raw motion data for a skeleton with 78 joints, rotating in the order of Z-axis, Y-axis, and X-axis, with the hip node additionally including three position channels and three rotation channels. During the training of the motion reference generation model, the raw motion data output by the first motion capture system undergoes skeleton scaling and foot-grounding processing, and the 23 joints are mapped to corresponding motion nodes in the target robot skeleton. The raw motion data output by the second motion capture system undergoes node channel layout adaptation, deleting skeletal nodes such as fingers, eyes, and unnecessary end effectors that are weakly related to the current robot's whole-body walking reference generation, and mapping the remaining nodes to the target robot skeleton. After skeleton redirection, both types of raw motion data are converted into target robot skeleton motion data corresponding to 24 robot motion nodes, where the target robot skeleton height can be set to 1.61m. Subsequently, a motion reference generation model is trained based on the target robot skeleton motion data, enabling the motion reference generation model to generate first whole-body motion reference information based on a unified skeleton representation.

[0137] In one example, the raw motion capture skeleton data from different motion capture systems can first be processed by skeleton retargeting to obtain the target robot skeleton motion data; then, a training sample set can be constructed based on the target robot skeleton motion data. Each training sample corresponds to a root motion velocity. When training the temporal recursive network and / or the pose decoding network, the training sample set can be divided into multiple velocity segments according to the root motion velocity, and the sampling weight of the velocity segments that meet the preset low-speed conditions can be increased.

[0138] In some embodiments, the target robot skeleton motion data may include multiple skeleton motion frames arranged in time; the construction of a training sample set based on the target robot skeleton motion data may include: taking at least one skeleton motion frame as the current frame, extracting the body posture features, future trajectory features, gait features, root motion velocity, and whole-body motion reference target corresponding to the current frame, forming training samples for training the motion reference generation model.

[0139] In some embodiments, controlling the robot's movement based on the second whole-body motion reference information may include: generating a future reference trajectory based on the second whole-body motion reference information; inputting the future reference trajectory into a reference trajectory tracking controller to obtain robot control data; and controlling the robot's movement based on the robot control data.

[0140] A future reference trajectory can refer to motion reference data generated based on second whole-body motion reference information, covering one or more future control moments. The future reference trajectory can include information such as root position, root pose, root velocity, root angular velocity, robot motion node position, and robot motion node pose at multiple future moments. For example, in a scenario with a control frequency of 50 Hz, the future reference trajectory can include data from 20 future reference moments to provide continuous reference for the robot's subsequent motion.

[0141] A reference trajectory tracking controller can be a controller used to generate robot control data based on a future reference trajectory. Reference trajectory tracking controllers can be reinforcement learning reference trajectory tracking controllers, model prediction controllers, whole-body controllers, hierarchical quadratic programming controllers, inverse kinematics controllers, or combinations of the above. The function of the reference trajectory tracking controller is to enable the robot's actual motion to follow the future reference trajectory, thereby converting the second whole-body motion reference information into control data that the robot can execute.

[0142] Robot control data refers to the execution data used to control the robot's motion. Robot control data can include joint target positions, joint target velocities, joint target torques, joint angle increments, end effector target poses, whole-body control constraints, and inverse kinematics results. Robot control data can be generated by a reference trajectory tracking controller based on a future reference trajectory, or by a target motion control module based on output frames.

[0143] This embodiment generates a future reference trajectory based on a second whole-body motion reference information. This allows the whole-body motion reference result, after root synchronization correction, to be converted into a continuous reference input that the reference trajectory tracking controller can consume, improving the consistency between robot control data and the corrected motion reference result. By inputting the future reference trajectory into the reference trajectory tracking controller, the whole-body motion reference result can be converted into joint targets, torque targets, or other robot control data, improving the executability of robot motion control.

[0144] In some embodiments, controlling the robot's movement based on the second whole-body motion reference information may include: generating an output frame based on the second whole-body motion reference information and sending the output frame to a target motion control module, so that the target motion control module controls the robot's movement based on the output frame. The output frame may include pose information and motion channel information of multiple robot motion nodes. The motion channel information may include six channels corresponding to the root and three channels corresponding to each of the multiple subordinate motion nodes other than the root. The number of channels in the motion channel information can be determined based on the number of motion nodes in the target robot skeleton.

[0145] An output frame can refer to a frame of motion reference data encapsulated based on the second whole-body motion reference information. The output frame can be used to transmit motion reference information between the motion reference information generation side and the target motion control module. Through the output frame, the second whole-body motion reference information can be organized into a data format that is easy for the target motion control module to read and use.

[0146] A target motion control module can refer to a control module that receives output frames and controls the robot's motion based on those frames. This module may include a reference trajectory tracking controller, an inverse kinematics solver, a whole-body control module, a motion editing module, a low-level robot control module, or an external visualization and debugging module. The target motion control module can acquire output frames through network communication, in-process calls, shared memory, or file reading.

[0147] Robot motion nodes refer to nodes used to describe the overall motion of a robot. Robot motion nodes can correspond to nodes such as the root, hip, spine, head, shoulder, arm, hand, thigh, calf, foot, and toes of the robot, or they can correspond to virtual motion nodes in the robot's control chain. Multiple robot motion nodes together constitute the object representing the robot's overall motion reference information.

[0148] The pose information of a robot motion node refers to the information used to describe the spatial position and orientation of the robot motion node. Pose information can include the position coordinates and orientation representation of the robot motion node. The position coordinates can be three-dimensional coordinates, and the orientation representation can be quaternions, Euler angles, rotation matrices, or other orientation representation methods. For example, the pose information of a robot motion node can include its global position (0.10m, 0.02m, 0.95m) and its corresponding quaternion orientation.

[0149] Motion channel information for robot motion nodes refers to motion data arranged according to a preset channel format. This information describes the motion parameters of the root and multiple subordinate motion nodes, enabling the target motion control module to read and use the corresponding channels in a preset order. Motion channel information may include position channels, rotation channels, velocity channels, or other channels that can be consumed by the target motion control module.

[0150] The six channels corresponding to the root node can refer to the six motion channels associated with the root node. These six channels can include three position channels and three rotation channels. For example, the three position channels can represent the root node's lateral, longitudinal, and height positions in three-dimensional space, respectively, while the three rotation channels can represent the root node's rotation around the three coordinate axes. These six channels are used to express the robot's overall spatial displacement and overall spatial orientation.

[0151] The three channels corresponding to each subordinate motion node can refer to three rotational channels corresponding to each subordinate motion node other than the root. Since the position of the subordinate motion node can be calculated from the skeleton level, node offset, and joint rotation relationship, the subordinate motion node can mainly express posture changes through three rotational channels. For example, subordinate motion nodes such as left thigh, left calf, left foot, right thigh, right calf, and right foot can each correspond to three rotational channels.

[0152] The number of motion channel information channels is determined by the number of motion nodes in the target robot skeleton. This means that the total number of motion channel information channels is not fixed at a single value, but varies with the number of motion nodes in the target robot skeleton. If the target robot skeleton includes 1 root motion node and 23 subordinate motion nodes, then the motion channel information can include 6 + 23 × 3 = 75 channels; if the target robot skeleton includes 1 root motion node and 22 subordinate motion nodes, then the motion channel information can include 6 + 22 × 3 = 72 channels. This expression method can adapt to different skeleton structures and reduce the limitation of fixing the protection scope to a certain number of channels.

[0153] This embodiment generates an output frame based on the second whole-body motion reference information and sends it to the target motion control module. This allows the second whole-body motion reference information to be provided to different target motion control modules in a unified data format, improving the interface adaptability of the motion reference information. By simultaneously setting the pose information and motion channel information of multiple robot motion nodes in the output frame, both pose-based and channel-based control links can be accommodated, reducing the complexity of additional data conversion for the target motion control module. By setting the motion channel information to 6 channels at the root and 3 channels for each subordinate motion node, and making the number of channels determined by the number of motion nodes in the target skeleton, it can adapt to different target skeleton structures, improving the compatibility of the output frame with different robot skeletons.

[0154] In this embodiment, the robot control method can control robot movement based on second whole-body motion reference information. In one implementation, the robot control method organizes the second whole-body motion reference information into a future reference trajectory, inputs the future reference trajectory into a reference trajectory tracking controller, which generates robot control data and then controls robot movement based on the robot control data. In another implementation, the robot control method encapsulates the second whole-body motion reference information into an output frame and sends the output frame to a target motion control module. The target motion control module reads the pose information and motion channel information of multiple robot motion nodes in the output frame and generates robot control data based on the pose information and motion channel information or directly executes robot motion control. Both implementations use the second whole-body motion reference information as the motion reference source; the difference lies in that one inputs the reference trajectory into the reference trajectory tracking controller via a future reference trajectory, while the other inputs it into the target motion control module via an output frame.

[0155] The above embodiments address the problem that the second whole-body motion reference information needs to be consumed by different downstream control links, and that different target motion control modules have different input format requirements for reference trajectory, pose information, and channel information. They adopt two optional control methods: "inputting the future reference trajectory to the reference trajectory tracking controller" and "sending the output frame to the target motion control module", thereby improving the availability and interface compatibility of the second whole-body motion reference information in the robot motion control link.

[0156] In some embodiments, the output frame can be a motion matching frame in a preset serialization format, such as a motion matching frame in Protobuf format. The output frame may include pose information of multiple robot motion nodes and pre-stitched motion channel information. The target motion control module can obtain the output frame through network communication or in-process calls; the network communication method may include communication based on the User Datagram Protocol (UDP), and the in-process call method may include interface calls within the embedded control process. Therefore, the second whole-body motion reference information can be adapted to both independent process consumption scenarios and embedded in-process consumption scenarios.

[0157] In some embodiments, the robot control method may further include: generating a skeleton motion data file based on the second whole-body motion reference information; converting the skeleton motion data file into a training data file that can be used by a reference trajectory tracking controller, so as to train the reference trajectory tracking controller based on the training data file.

[0158] A skeleton motion data file refers to a data file generated based on second whole-body motion reference information, used to record the motion process of a robot skeleton. The skeleton motion data file can include root motion data, pose data of multiple robot motion nodes, frame timing information, hierarchical relationships of motion nodes, and motion node channel information. The skeleton motion data file can be organized according to a hierarchical skeleton motion data format or other formats that can express the robot skeleton structure and frame-by-frame motion data. Using the skeleton motion data file, the online-generated second whole-body motion reference information can be saved as reusable motion data.

[0159] Training data files refer to data files that the reference trajectory tracking controller can read and use during the training process. Training data files can be converted from skeleton motion data files and may include training fields such as future reference trajectories, root motion reference data, robot motion node pose data, joint reference data, velocity reference data, and gait labels. The data format of the training data file can be determined according to the training process of the reference trajectory tracking controller; for example, it can be an array file, a serialized data file, a trajectory sample file, or other data files in a preset training sample format.

[0160] Converting the skeleton motion data file into a training data file usable by the reference trajectory tracking controller can refer to the process of organizing the skeleton motion data in the skeleton motion data file into a training data file. The conversion process may include coordinate system transformation, unit transformation, frame rate resampling, motion node order adjustment, channel data parsing, reference trajectory segment extraction, joint reference data generation, and training field encapsulation. The purpose of the conversion is to transform data originally intended for motion recording or playback into data that the reference trajectory tracking controller can directly consume during its training process.

[0161] In this embodiment, the reference trajectory tracking controller can learn how to track the reference trajectory formed by the second whole-body motion reference information based on the training data file during the training phase, and can output robot control data based on the same reference trajectory during the deployment phase.

[0162] Training a reference trajectory tracking controller based on a training data file refers to using the training data file as input to the reference trajectory tracking controller's training process, enabling the controller to learn the motion reference distribution corresponding to a second whole-body motion reference. The training process may include reading the training data file, extracting future reference trajectories, constructing training samples, calculating tracking errors, and updating the reference trajectory tracking controller parameters. Through this training process, the reference trajectory tracking controller can be exposed to the reference trajectory types generated by the robot control method in advance.

[0163] In this embodiment, after obtaining the second whole-body motion reference information, the robot control method can generate a skeleton motion data file based on it. This saves the root motion, robot motion node poses, and motion channels from the second whole-body motion reference information as reusable skeleton motion data. Subsequently, the skeleton motion data file is converted into a training data file usable by the reference trajectory tracking controller, organizing the motion information in the file into training fields, reference trajectory fragments, or training samples. Next, the reference trajectory tracking controller can be trained based on the training data file, allowing it to access the reference trajectory formed by the second whole-body motion reference information during the training phase. Therefore, the second whole-body motion reference information is not only used for online control but can also serve as a source of training data to feed back into the training process of the reference trajectory tracking controller.

[0164] This embodiment generates a skeleton motion data file based on second whole-body motion reference information, which can convert the online-generated whole-body motion reference results into a reusable data file, improving the reuse value of the second whole-body motion reference information. By converting the skeleton motion data file into a training data file that can be used by the reference trajectory tracking controller, the skeleton motion data can be matched with the training input format of the reference trajectory tracking controller, reducing the data adaptation cost before training the reference trajectory tracking controller. By training the reference trajectory tracking controller based on the training data file, the reference trajectory tracking controller can be exposed to the actual reference trajectory distribution generated by the robot control method during the training phase, reducing the mismatch risk caused by the inconsistency between the reference trajectory distribution during the training phase and the reference trajectory distribution during the deployment phase. This embodiment addresses the problem that the reference trajectory tracking controller may not have sufficient contact with the whole-body motion reference information generated during the deployment phase during the training phase, resulting in inconsistencies in the reference trajectory distribution between the training and deployment phases. It adopts the concept of "generating a skeleton motion data file from the second whole-body motion reference information and further converting it into a training data file to feed back into the training," thereby improving the adaptability of the reference trajectory tracking controller to the reference trajectory during the deployment phase.

[0165] In one example, the robot control method generates second whole-body motion reference information at a frequency of 50 Hz, continuously generating data for 10 seconds, resulting in 500 frames of second whole-body motion reference information. Based on these 500 frames, the robot control method generates a skeleton motion data file, which includes the root position, root pose, left foot pose, right foot pose, torso pose, and motion channel data for multiple robot motion nodes for each frame. Subsequently, the robot control method converts the skeleton motion data file into a training data file. During the conversion, the 50 Hz frame sequence is maintained as a 50 Hz training sequence, and the reference trajectory for the next 20 frames is extracted as a training field from each training moment. After reading the training data file, the reference trajectory tracking controller is trained based on the reference trajectory for the next 20 frames, the current robot state, and the corresponding target motion data, enabling the reference trajectory tracking controller to learn to track the future reference trajectory formed by the second whole-body motion reference information.

[0166] See Figure 2 , Figure 2 This is a structural block diagram of a robot control device provided in an embodiment of this application.

[0167] This application also provides a robot control device, including an information acquisition module, a reference generation module, a root correction module, and a motion control module.

[0168] The information acquisition module is used to acquire the target motion information and current motion state information of the robot. The target motion information is determined based on the operation input information for the robot.

[0169] The reference generation module is used to generate first whole-body motion reference information based on the target motion information and the current motion state information through a motion reference generation model.

[0170] The root correction module is used to synchronously correct the root motion reference information in the first whole-body motion reference information according to the target motion information, so as to obtain the second whole-body motion reference information.

[0171] The motion control module is used to control the robot's movement based on the second whole-body motion reference information.

[0172] See Figure 3 , Figure 3 This is a structural block diagram of a robot provided in an embodiment of this application.

[0173] This application also provides a robot, which includes a control module for performing the steps of any of the above methods.

[0174] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0175] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0176] The computer program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the computer program product of the embodiments of this application is not limited thereto, and the computer program product may be any combination of one or more computer-readable media.

[0177] See Figure 4 , Figure 4 This is a structural block diagram of a computer device provided in an embodiment of this application.

[0178] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0179] The embodiments of this application do not limit the computer device, which may be, for example, a local computer device, a cloud computer device, a distributed computer device, etc.

[0180] The computer device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through internal connection paths.

[0181] The memory 110 is used to store computer programs, which in some implementations may include code for implementing the methods of the embodiments of this application.

[0182] The processor 120 executes the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, and output operation results and other data. In some implementations, when the solutions of the embodiments of this application are implemented by software or firmware, the computer program used to implement the solutions of the embodiments of this application can be stored in the processor 120 and executed by the processor 120.

[0183] The memory 110 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes random access memory (RAM), cache memory, and read-only memory (ROM). The memory 110 stores a computer program that can be executed by processor 120, causing processor 120 to implement the steps of any of the methods described above.

[0184] The processor 120 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can be any conventional processor.

[0185] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 120 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor 120. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0186] In some implementations, in addition to the hardware units described above, computer devices may also include software modules, such as operating systems, basic input / output systems (BIOS), and application software.

[0187] An operating system is used to manage the hardware and / or software resources of a computer device; it is the kernel and foundation of the computer. The operating system handles fundamental tasks such as managing and configuring memory, determining the priority of system resource allocation, controlling input and output devices, operating the network, and managing the file system. To facilitate user operation, most operating systems provide a user interface for interaction with the system.

[0188] The BIOS is used to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display processor temperature and execute temperature protection strategies.

[0189] Application software, also known as an application program, is software written for a specific user application purpose and is one of the main categories of computer software. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0190] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0191] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0192] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0193] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0194] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design intervention conditions of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0195] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the embodiments described above can be referred to the corresponding processes in other embodiments, and will not be repeated here.

[0196] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0198] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to related technologies, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robot control method, characterized in that, include: Acquire the target motion information and current motion state information of the robot, wherein the target motion information is determined based on the operation input information for the robot; Based on the target motion information and the current motion state information, first whole-body motion reference information is generated through a motion reference generation model; Based on the target motion information, the root motion reference information in the first whole-body motion reference information is synchronously corrected to obtain the second whole-body motion reference information. The robot's movement is controlled based on the second whole-body motion reference information; Specifically, the root motion reference information in the first whole-body motion reference information is synchronously corrected to obtain the second whole-body motion reference information, including: Determine the desired root motion reference information based on the target motion information; Based on the target linear velocity and / or target angular velocity corresponding to the target motion information, the root synchronization parameters are determined. The root synchronization parameters are parameters used to control the fusion ratio between the root motion reference information in the first whole-body motion reference information and the desired root motion reference information. Based on the root synchronization parameters, the root motion reference information in the first whole-body motion reference information and the desired root motion reference information are fused to obtain the corrected root motion reference information. Based on the corrected root motion reference information and the robot body motion reference information in the first whole-body motion reference information, the second whole-body motion reference information is obtained.

2. The robot control method according to claim 1, characterized in that, The step of determining the desired root motion reference information based on the target motion information includes: Based on the target motion information, the root motion is calculated to obtain the desired root motion reference information; The root motion reference information includes root motion state information and / or root motion change information. The root motion state information includes at least one of root position, root attitude, root velocity, and root angular velocity. The root motion change information includes at least one of position increment, attitude increment, velocity change, and angular velocity change relative to the current root state. The desired root motion reference information includes desired root motion state information and / or desired root motion change information. The desired root motion state information includes at least one of desired root position, desired root attitude, desired root velocity, and desired root angular velocity. The desired root motion change information includes at least one of desired position increment, desired attitude increment, desired velocity change, and desired angular velocity change relative to the current root state.

3. The robot control method according to claim 1, characterized in that, The root synchronization parameters include root position synchronization parameters and root attitude synchronization parameters. Determining the root synchronization parameters based on the target linear velocity and / or target angular velocity corresponding to the target motion information includes: The equivalent velocity is determined based on the larger of the target linear velocity and the equivalent linear velocity corresponding to the target angular velocity. Based on the ratio of the equivalent speed to the preset speed, the speed ratio parameter is determined; Based on the difference between 1 and the cube of the speed ratio parameter, the root position synchronization parameter for root position synchronization is determined. Based on the root position synchronization parameters, the root attitude synchronization parameters are determined, and the root attitude synchronization parameters are greater than zero.

4. The robot control method according to claim 1, characterized in that, The operation input information includes at least one of forward / backward direction input information, lateral direction input information, and steering input information; The process of determining the target motion information includes: performing at least one of dead zone processing, speed level determination, and gait state determination based on the operation input information to obtain the target motion information.

5. The robot control method according to claim 1, characterized in that, The motion reference generation model includes a feature mapping network, a temporal recursive network, and a pose decoding network; The step of generating first whole-body motion reference information based on the target motion information and the current motion state information through a motion reference generation model includes: Query features are constructed based on the target motion information and the current motion state information; The query features are mapped into latent space information through the feature mapping network. The latent space information is recursively processed through the temporal recursion network. The attitude decoding network decodes the temporally recursively derived latent space information to obtain the first whole-body motion reference information.

6. The robot control method according to claim 5, characterized in that, The training samples of the feature mapping network include chimeric training samples; The query features of the chimeric training samples include ontological pose features from standing state samples, and future trajectory features and / or gait features from walking state samples. The query features of the chimeric training samples are features obtained by splicing together features in the target local feature space.

7. The robot control method according to claim 5, characterized in that, The temporal recursive network and / or the pose decoding network are trained based on training samples in the training sample set; The process of determining the training samples includes: during the training process, dividing the training sample set into multiple velocity segments according to the root motion velocity, and selecting training samples from the training sample set for training the temporal recursive network and / or the pose decoding network based on the sampling weights corresponding to each velocity segment. Wherein, the sampling weight corresponding to at least one preset category of speed segment is set to be greater than the sampling weight determined according to the proportion of the number of samples in the training sample set; the speed segment of the preset category satisfies the preset low speed condition.

8. The robot control method according to claim 1, characterized in that, The training sample set of the motion reference generation model is constructed based on the motion data of the target robot skeleton; The target robot skeleton motion data includes usable robot skeleton motion data obtained by skeleton redirection processing of raw motion capture skeleton motion data from different motion capture systems. The skeleton redirection process includes adapting to at least one of different numbers of joints, different rotation sequences, and different node channel layouts.

9. The robot control method according to claim 1, characterized in that, The step of controlling the robot's movement based on the second whole-body motion reference information includes: Generate a future reference trajectory based on the second whole-body motion reference information; The future reference trajectory is input into the reference trajectory tracking controller to obtain the robot control data; The robot's movement is controlled based on the robot control data.

10. The robot control method according to claim 1, characterized in that, The step of controlling the robot's movement based on the second whole-body motion reference information includes: An output frame is generated based on the second whole-body motion reference information, and the output frame is sent to the target motion control module so that the target motion control module controls the robot's motion based on the output frame; The output frame includes pose information and motion channel information of multiple robot motion nodes. The motion channel information includes 6 channels corresponding to the root and 3 channels corresponding to each of the multiple subordinate motion nodes other than the root. The number of channels in the motion channel information is determined according to the number of motion nodes in the target robot skeleton.

11. The robot control method according to claim 1, characterized in that, The method further includes: Generate a skeletal motion data file based on the second whole-body motion reference information; The skeleton motion data file is converted into a training data file that can be used by the reference trajectory tracking controller, so as to train the reference trajectory tracking controller based on the training data file.

12. A robot control device, characterized in that, include: The information acquisition module is used to acquire the target motion information and current motion state information of the robot, wherein the target motion information is determined based on the operation input information for the robot; The reference generation module is used to generate first whole-body motion reference information based on the target motion information and the current motion state information through a motion reference generation model; The root correction module is used to synchronously correct the root motion reference information in the first whole-body motion reference information according to the target motion information to obtain the second whole-body motion reference information. A motion control module is used to control the robot's movement based on the second whole-body motion reference information; Specifically, the root motion reference information in the first whole-body motion reference information is synchronously corrected to obtain the second whole-body motion reference information, including: Determine the desired root motion reference information based on the target motion information; Based on the target linear velocity and / or target angular velocity corresponding to the target motion information, the root synchronization parameters are determined. The root synchronization parameters are parameters used to control the fusion ratio between the root motion reference information in the first whole-body motion reference information and the desired root motion reference information. Based on the root synchronization parameters, the root motion reference information in the first whole-body motion reference information and the desired root motion reference information are fused to obtain the corrected root motion reference information. Based on the corrected root motion reference information and the robot body motion reference information in the first whole-body motion reference information, the second whole-body motion reference information is obtained.

13. A robot, characterized in that, The robot includes a control module for performing the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

Images