A control method of a humanoid robot, a humanoid robot, and a storage medium

Abstract

The application provides a control method of a humanoid robot, the humanoid robot and a storage medium. The method comprises: acquiring a current state and a speed instruction of the humanoid robot; retrieving a reference motion segment from a preset database according to the current state and the speed instruction, wherein the database comprises a plurality of motion segments, and the motion segments are obtained according to human motion data; inputting the current state, the speed instruction and the reference motion segment into a motion generation model to obtain a target state of the humanoid robot at a next moment; inputting the current state into a policy network to obtain a target action output by the policy network, and controlling the humanoid robot to perform the target action; and updating parameters of the policy network according to a new state of the humanoid robot after performing the target action and the target state. Through the above method, the action of the humanoid robot can be more humanized, and unnatural actions can be avoided.

Description

Technical Field

[0001] This application belongs to the field of robotics technology, and in particular relates to a control method for a humanoid robot, the humanoid robot, and a computer-readable storage medium. Background Technology

[0002] In existing technologies, one of the mainstream methods for achieving motion control of humanoid robots is the traditional model-based control method. The core of this method relies on a precisely established robot dynamics model and predefined stability criteria, such as the zero torque point (ZMP).

[0003] However, this method has significant technical drawbacks. Due to its reliance on a simplified physical model and rigid stability criteria, the generated gait often appears unnatural, characterized by excessive knee flexion, stiff gait cycles, and a lack of the flexibility and coordination inherent in human walking. This typical "robotic" gait is not only energy inefficient but, more importantly, severely limits the integration and acceptance of humanoid robots in everyday human environments. Summary of the Invention

[0004] In view of this, this application provides a control method for a humanoid robot, a humanoid robot, and a computer-readable storage medium, which can make the humanoid robot's movements more human-like and avoid making unnatural movements.

[0005] In a first aspect, this application provides a control method for a humanoid robot, the method comprising: Obtain the current state and speed command of the humanoid robot; Based on the current state and the speed command, a reference motion segment is retrieved from a preset database, wherein the database includes multiple motion segments, which are obtained based on human motion data; The current state, the speed command, and the reference motion segment are input into the motion generation model to obtain the target state of the humanoid robot at the next moment; The current state is input into the policy network to obtain the target action output by the policy network, and the humanoid robot is controlled to execute the target action. The parameters of the policy network are updated based on the new state of the humanoid robot after performing the target action and the target state.

[0006] In the above method, firstly, since the motion segments in the database are directly derived from real human motion data, they inherently possess the naturalness and coordination of human movement. When a reference motion segment is retrieved from this database based on the current state and speed command, the obtained segment already possesses a highly anthropomorphic foundation in terms of physical plausibility and motion style. Next, the current state, speed command, and reference motion segment are input together into the motion generation model, which generates the target state for the next moment. The target state inherits the naturalness of real human movement while dynamically adapting to the current speed command and current state. Finally, the policy network is driven to control the humanoid robot to track this target state, reproducing highly anthropomorphic movements.

[0007] Secondly, this application provides a humanoid robot, including a memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the humanoid robot to perform the humanoid robot control method provided in the first aspect above.

[0008] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the humanoid robot control method provided in the first aspect above.

[0009] Fourthly, this application provides a computer program product that, when run on a humanoid robot, causes the humanoid robot to perform the humanoid robot control method provided in the first aspect above.

[0010] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the control method for a humanoid robot provided in an embodiment of this application.

[0013] Figure 2 This is a schematic diagram of the structure of the humanoid robot provided in the embodiments of this application. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0015] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0016] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0017] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0018] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0019] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0020] The development of humanoid robot control technology has mainly gone through the following evolutionary process: Traditional model-based control methods, as the mainstream approach for early humanoid robot dynamic walking, rely on accurate dynamic models and predefined stability criteria (such as zero-moment point (ZMP)). They typically employ a hierarchical control architecture: an upper-level trajectory planner generates a reference trajectory between the center of mass and the foot based on task instructions and ZMP stability constraints; a lower-level whole-body controller decomposes the reference trajectory into torque commands for each joint using optimization algorithms (such as model predictive control, MPC). While this method has proven feasible in structured environments, it suffers from significant limitations: the generated gait is stiff and unnatural (manifested as excessive knee flexion), it is extremely sensitive to model errors and external disturbances, and its adaptability in unstructured environments is very limited.

[0021] Reinforcement learning-based control methods: To overcome the limitations of model-based methods and improve system adaptability, researchers have turned to data-driven reinforcement learning methods. This method allows robots to autonomously learn control strategies through trial and error in interaction with their environment. Its advantages lie in not relying on precise dynamic models and in its ability to simultaneously optimize task completion and movement style through carefully designed reward functions. However, reinforcement learning methods also have inherent drawbacks: the design of reward functions is extremely complex and requires extensive debugging; the training process is inefficient and time-consuming; and without effective guidance, policy networks can easily learn gaits that, while capable of completing tasks, exhibit unnatural or even "strange" characteristics.

[0022] Introducing imitation learning methods based on prior motion: To directly integrate the natural characteristics of human motion into robot control, researchers have begun to utilize human motion data to guide reinforcement learning training through imitation learning mechanisms. Specifically, this includes: Adversarial Motion Prior (AMP): This method introduces a discriminator to distinguish between robot and human motion, and provides an adversarial reward signal to the policy based on the discrimination result, indicating whether the motion is human-like. While it can significantly improve the naturalness of the motion, AMP suffers from problems such as unstable training process and sparse and ambiguous reward signals.

[0023] Generative Motion Prior (GMP): As an improvement to AMP, GMP pre-trains a generative model (such as a Conditional Variational Autoencoder CVAE) to directly generate reference trajectories for future motions during online runtime based on the current robot state and instructions. This provides finer-grained and more explicit action guidance for reinforcement learning strategies, thereby significantly improving training stability. However, methods based on pure generative models are prone to error accumulation and pattern drift when generating long-term motion, which may lead to distortion of the generated motion. Furthermore, their adaptability and system robustness still face significant challenges when dealing with scenarios not covered by training data.

[0024] Figure 1A flowchart illustrating a control method for a humanoid robot according to an embodiment of this application is shown. This control method for a humanoid robot is applied to a humanoid robot and is described in detail below: Step 101: Obtain the current state and speed command of the humanoid robot; Step 102: Based on the current state and the speed command, retrieve a reference motion segment from a preset database, wherein the database includes multiple motion segments, which are obtained based on human motion data; Step 103: Input the current state, the speed command, and the reference motion segment into the motion generation model to obtain the target state of the humanoid robot at the next moment; Step 104: Input the current state into the policy network to obtain the target action output by the policy network, and control the humanoid robot to execute the target action; Step 105: Update the parameters of the policy network based on the new state of the humanoid robot after performing the target action and the target state.

[0025] In this embodiment, the current state can be a multi-dimensional vector describing the body posture and motion of the humanoid robot at the current time t. Specifically, the current state may include, but is not limited to: joint positions and joint velocities obtained from the robot's joint encoders, base posture, base linear velocity, and base angular velocity obtained from the inertial measurement unit, and keypoint positions calculated based on joint positions using forward kinematics. Keypoints may include, for example, the coordinates of the feet, pelvis, and hands in three-dimensional space. The velocity command is a high-level control command from the user or the upper-level planning system, typically a two-dimensional vector, such as [v_x, ω_z], where v_x represents the desired forward or backward linear velocity, and ω_z represents the desired angular velocity about the vertical axis. The database is pre-built offline, and its multiple motion segments form the basis for realizing anthropomorphic motion. Understandably, these motion segments are generated through motion redirection technology, transforming and optimizing real human motion data (such as that obtained through optical or inertial motion capture systems) collected from human actors into a continuous temporal sequence of motion that conforms to the physical configuration, joint limits, link lengths, and other kinematic constraints of the target humanoid robot. Each motion segment contains multiple frames, and each frame contains various state information defined by the aforementioned current state. The motion generation model in this embodiment is a conditional variational autoencoder. It should be noted that during the inference phase, this model uses the reference motion segment retrieved in the previous step as a strong conditional input, combined with the current state and speed command, and outputs a refined target state for the next moment through calculation by its internal neural network. The target state is structurally identical to the current state, but it represents the state that the model considers most ideal, natural, and consistent with the speed command at the next moment t+1. The policy network is a deep neural network trained using a reinforcement learning algorithm, which parameterizes a control policy. It takes the current state as input, passes through the forward propagation within the network, and outputs the target action. For example, the target action can be a target joint position increment, a target joint torque, or a target joint velocity. The underlying controller (such as a PD controller) translates this target action into specific joint torque commands, driving the humanoid robot to execute them. Finally, based on the difference between the new state actually reached by the robot after executing the target action and the target state given by the action generation model, the reward signal required for reinforcement learning is calculated, and this signal is used to update the parameters of the policy network. Through the accumulation and learning of a large amount of such interaction experience, the parameters of the policy network are continuously optimized, so that its output actions can drive the robot's state to increasingly accurately track the target state provided by the action generation model. A reference motion segment is a motion segment retrieved from multiple motion segments in a database.The policy network is a deep neural network trained using reinforcement learning. Its input is the current state s_t of the humanoid robot (a multi-dimensional vector containing joint angles, velocities, base information, etc.), and its output is the target action a_t (the command the humanoid robot should execute). The action generation model is implemented using CVAE (Conditional Variational Auto-Encoder), an advanced generative model belonging to the field of deep learning.

[0026] In some embodiments, the speed command comes from the user.

[0027] In this embodiment, the user can be an operator who directly issues movement commands to the humanoid robot via physical devices (such as remote controls or joysticks) or software interfaces (such as mobile applications or computer command lines). For example, the user can slide a slider on the interface to set a forward speed of 0.5 m / s, or press and hold a left turn button to generate a continuous angular velocity command of 0.2 radians / second; these are all speed commands from the user. It is understood that the humanoid robot possesses the ability to interact with human operators in real time, responding to and executing the user's intuitive movement intentions, which enhances the practicality and ease of use of the humanoid robot in real-world application scenarios.

[0028] In some embodiments, retrieving a reference motion segment from a preset database based on the current state and the speed command includes: Determine the current feature vector based on the current state and the speed command; Calculate the initial feature vector of the initial state of each motion segment in the database; Based on the current feature vector and the initial feature vector, the reference motion segment that matches the current state is retrieved from the database.

[0029] In this embodiment, the purpose of constructing feature vectors is to uniformly encode raw state data (such as current state and initial state) that may have different dimensions or units into fixed-dimensional numerical vectors capable of efficient mathematical comparison. The current feature vector is dynamically calculated based on the real-time acquired current state and velocity commands. The initial feature vector for each motion segment in the database is pre-calculated and stored, corresponding to the state encoding of the initial frame of each motion segment in the database. Optionally, to accelerate the search process in a massive database, an efficient data structure (such as a KD-Tree) can be used to organize and index all these initial feature vectors.

[0030] At each control time t, a current feature vector is calculated in real time based on the current state s_t and the velocity command c_t. This current feature vector is compared with the starting feature vectors of all motion segments in the database (by calculating a weighted Euclidean distance). The starting feature vector with the smallest distance to the current feature vector is found. The entire motion segment (not just the starting frame) corresponding to this starting feature vector is retrieved and output as a reference motion segment for use by the subsequent motion generation model. Most of the information in the current feature vector (such as joint angles and base velocity) comes directly from the real-time sensor readings of the humanoid robot at time t, and the expected position is calculated in real time based on the velocity command c_t given by the user. The starting feature vector is a pre-calculated feature vector stored in the database, which encodes the state of a certain motion segment in the database at its starting frame (i.e., the moment the segment begins).

[0031] In some embodiments, determining the current feature vector based on the current state and the speed command includes: Extract the current joint angle from the current state; The desired position of a preset part of the humanoid robot is predicted based on the speed command after a preset time. Extract the current linear velocity and current angular velocity of the humanoid robot's base from the current state; The current feature vector is constructed by using the current joint angle, the desired position, the current linear velocity, and the current angular velocity as components.

[0032] In this embodiment, the preset location can be the foot, which is crucial for motion stability. For example, based on the velocity command [v_x, ω_z], assuming the humanoid robot moves at a constant speed indicated by the velocity command within a short future Δt (e.g., 0.3 seconds, 0.6 seconds), the future position of the pelvis is calculated. Then, combined with typical gait phase, the expected horizontal landing position of the left and right feet at the next Δt time is calculated; this horizontal landing position is the expected position. This component greatly improves the relevance of the retrieval results to the future motion intention. The current linear velocity and current angular velocity of the base describe the overall motion trend of the robot. It can be understood that concatenating these components with different attributes (current joint angle, expected position, current linear velocity, and current angular velocity) into a current feature vector constitutes a query feature that can be used to retrieve motion segments from the database. Components with the same attributes as the initial feature vector and the current feature vector are used to compare the similarity between the initial and current feature vectors during retrieval. The current joint angle is the angle of all or some joints of the humanoid robot at the current moment. The current linear velocity and current angular velocity of the base are the linear velocity and angular velocity of the humanoid robot's base at the current moment.

[0033] In some embodiments, retrieving the reference motion segment matching the current state from the database based on the current feature vector and the initial feature vector includes: Calculate the distance between the current feature vector and the initial feature vector; The motion segment corresponding to the starting feature vector in the database that has the smallest distance from the current feature vector is used as the reference motion segment.

[0034] In this embodiment, the aforementioned distance can be calculated using a weighted Euclidean distance, with the formula D(f_current, f_db^i)=||W· (f_current-f_db^i) ||_2. Here, D is the Euclidean distance, f_current is the current feature vector, and f_db^i is the starting feature vector of the i-th motion segment in the database. It should be noted that W is a diagonal weight matrix, and the weight values ​​w_j on its diagonal are used to adjust the importance of different feature dimensions in the overall similarity calculation. For example, a higher weight can be assigned to the expected future position of the foot to ensure that the retrieved segment highly matches the current speed command in terms of gait phase and foot placement; simultaneously, lower weights can be assigned to the angles of certain non-critical joints. The motion segment corresponding to the starting feature vector with the smallest distance is the best matching result retrieved, and this motion segment serves as the reference motion segment. Optionally, this reference motion segment is a continuous sequence starting from the initial frame, such as the subsequent 10 frames of data (approximately 0.1 seconds of motion). This reference motion fragment M_match, derived from real human data, serves as input to the subsequent motion generation model, ensuring high naturalness and physical plausibility. The reference motion fragment M_match is a temporal sequence containing multiple consecutive frames, each representing a state of the humanoid robot at a specific moment.

[0035] In some embodiments, updating the parameters of the policy network based on the new state after the humanoid robot performs the target action and the target state includes: Calculate the reward value based on the new state and the target state; Store the current state, the target action, the new state, and the reward value in the cache. The parameters of the policy network are updated based on the data in the cache.

[0036] In this embodiment, the reward value *r* reflects the quality of the target action, and its calculation core lies in evaluating the degree of agreement between the new state (actual result) and the target state (expected result). It can be understood that the interaction experience tuple (current state, target action, new state, reward value) is stored in the buffer of the humanoid robot's memory (called the experience replay buffer). Updating the parameters of the policy network based on the data in the buffer involves learning from a batch of randomly sampled historical experience data from the buffer. This method can break the temporal correlation between data, improve data utilization, and significantly enhance the stability and convergence of training.

[0037] In some embodiments, calculating the reward value based on the new state and the target state includes: Extract the actual joint angles and actual key point positions from the new state; Extract reference joint angles and reference key point positions from the target state; The joint tracking reward value is calculated based on the difference between the actual joint angle and the reference joint angle; The key point tracking reward value is calculated based on the difference between the actual key point location and the reference key point location. Extract the actual velocity of the humanoid robot's base from the new state; The task reward value is calculated based on the difference between the actual speed of the base and the reference speed of the speed command; Calculate the motion smoothness reward value and the safety constraint reward value based on the new state; The reward value is obtained by weighted summing of the joint tracking reward value, the key point tracking reward value, the task reward value, the motion smoothness reward value, and the safety constraint reward value.

[0038] In this embodiment, the joint tracking reward encourages each joint of the robot to accurately reproduce the reference joint angles defined in the target state. The keypoint tracking reward provides stronger morphological constraints in Cartesian space, preventing the policy network from learning "strange" postures that, while having correct joint angles, result in incorrect overall configuration. The task reward directly evaluates the completion of velocity commands, ensuring that the final effect of the robot's movement matches the user's intent. Optionally, the motion smoothness reward is used to penalize drastic changes in joint acceleration or control torque, encouraging smooth motion. The safety constraint reward is used to penalize states that may lead to instability (such as excessive trunk tilt) or hardware damage (such as joint over-limits or self-collisions). Finally, the total reward value is obtained by weighted summation of these sub-rewards.

[0039] The joint tracking reward value is calculated as exp(-k_dof * Σ(actual joint angle - reference joint angle)²). The function exp(-k_dof * ...) maps the difference (a large positive number) to a reward interval between (0, 1). The keypoint tracking reward value is calculated as exp(-k_pos * Σ||actual keypoint position - reference keypoint position||²). The function exp(-k_pos * ...) maps the difference (a large positive number) to a reward interval between (0, 1). The task reward value is calculated as exp(-k_vel *||actual speed of the base - reference speed of the speed command||²). The function exp(-k_vel * ...) maps the difference (a large positive number) to a reward interval between (0, 1).

[0040] In some embodiments, the method further includes: The human motion data is mapped to humanoid robot motion data using motion redirection technology; The motion segment is generated based on the motion data of the humanoid robot.

[0041] In this embodiment, it is understood that human motion data differs from humanoid robot data in skeletal proportions, joint degrees of freedom, and movement capabilities, and therefore cannot be used directly. Motion redirection technology first establishes a mapping relationship between the human skeletal model and the humanoid robot skeletal model. Then, through optimization algorithms, while preserving the original human motion semantics (such as gait characteristics and rhythm), the human joint trajectories are converted into the trajectories of each joint of the robot, while simultaneously satisfying the robot's kinematic constraints (such as joint angle limitations and fixed link lengths). Optionally, this process may also need to handle physically unreasonable phenomena such as foot slippage and ground penetration. Finally, state information is directly extracted from the mapped humanoid robot motion data or calculated to obtain state information. Multiple frames of state information are arranged in chronological order to generate motion segments.

[0042] As can be seen from the above, in this application, firstly, since the motion segments in the database are directly derived from real human motion data, they inherently possess the naturalness and coordination of human movement. When a reference motion segment is retrieved from this database based on the current state and speed command, the obtained segment already possesses a highly anthropomorphic foundation in terms of physical rationality and motion style. Subsequently, the current state, speed command, and reference motion segment are input together into the motion generation model, which generates the target state for the next moment. The target state inherits the naturalness of real human movement while dynamically adapting to the current speed command and current state. Finally, the policy network is driven to control the humanoid robot to track the target state, reproducing highly anthropomorphic movements.

[0043] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Figure 2 This is a schematic diagram of the structure of a humanoid robot provided in one embodiment of this application. Figure 2 As shown, the humanoid robot 2 in this embodiment includes: at least one processor 20 ( Figure 2 Only one is shown in the diagram), memory 21, and computer program 22 stored in the memory 21 and executable on at least one processor 20. When the processor 20 executes the computer program 22, it causes the humanoid robot to perform the steps of the humanoid robot control method.

[0044] The humanoid robot 2 described above may include, but is not limited to, a processor 20 and a memory 21. Those skilled in the art will understand that... Figure 2 This is merely an example of humanoid robot 2 and does not constitute a limitation on humanoid robot 2. It may include more or fewer parts than shown in the figure, or combine certain parts, or different parts, such as input / output devices, network access devices, etc.

[0045] The processor 20 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0046] In some embodiments, the aforementioned memory 21 may be an internal storage unit of the humanoid robot 2, such as a hard drive or memory of the humanoid robot 2. In other embodiments, the aforementioned memory 21 may be an external storage device of the humanoid robot 2, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the humanoid robot 2. Furthermore, the aforementioned memory 21 may include both internal storage units and external storage devices of the humanoid robot 2. The aforementioned memory 21 is used to store operating systems, applications, bootloaders, data, and other programs, such as the program code of the aforementioned computer programs. The aforementioned memory 21 may also be used to temporarily store data that has been output or will be output. It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0047] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0048] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the various method embodiments described above.

[0049] This application provides a computer program product that, when run on a humanoid robot, causes the humanoid robot to perform the steps described in the various method embodiments above.

[0050] This application provides a chip system, characterized in that it includes a processor, which is used to call and run a computer program from a memory, causing a humanoid robot equipped with the chip system to perform the steps in the above-described method embodiments.

[0051] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a humanoid robot, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0052] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0053] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0054] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0055] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0056] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A control method for a humanoid robot, characterized in that, include: Obtain the current state and speed command of the humanoid robot; Based on the current state and the speed command, a reference motion segment is retrieved from a preset database, wherein the database includes multiple motion segments, which are obtained based on human motion data; The current state, the speed command, and the reference motion segment are input into the motion generation model to obtain the target state of the humanoid robot at the next moment; The current state is input into the policy network to obtain the target action output by the policy network, and the humanoid robot is controlled to execute the target action. The parameters of the policy network are updated based on the new state of the humanoid robot after performing the target action and the target state. The step of retrieving a reference motion segment from a preset database based on the current state and the speed command includes: Determine the current feature vector based on the current state and the speed command; Calculate the initial feature vector of the initial state of each motion segment in the database; Based on the current feature vector and the initial feature vector, the reference motion segment matching the current state is retrieved from the database; Determining the current feature vector based on the current state and the speed command includes: Extract the current joint angle from the current state; The desired position of a preset part of the humanoid robot is predicted based on the speed command after a preset time. Extract the current linear velocity and current angular velocity of the humanoid robot's base from the current state; The current feature vector is constructed by using the current joint angle, the desired position, the current linear velocity, and the current angular velocity as components.

2. The method as described in claim 1, characterized in that, The speed command comes from the user.

3. The method as described in claim 1, characterized in that, The step of retrieving the reference motion segment matching the current state from the database based on the current feature vector and the initial feature vector includes: Calculate the distance between the current feature vector and the initial feature vector; The motion segment corresponding to the starting feature vector in the database that has the smallest distance from the current feature vector is used as the reference motion segment.

4. The method as described in claim 1, characterized in that, The step of updating the parameters of the policy network based on the new state after the humanoid robot performs the target action and the target state includes: Calculate the reward value based on the new state and the target state; Store the current state, the target action, the new state, and the reward value in the cache. The parameters of the policy network are updated based on the data in the cache.

5. The method as described in claim 4, characterized in that, The step of calculating the reward value based on the new state and the target state includes: Extract the actual joint angles and actual key point positions from the new state; Extract reference joint angles and reference key point positions from the target state; The joint tracking reward value is calculated based on the difference between the actual joint angle and the reference joint angle; The key point tracking reward value is calculated based on the difference between the actual key point location and the reference key point location. Extract the actual speed of the humanoid robot's base from the new state; The task reward value is calculated based on the difference between the actual speed of the base and the reference speed of the speed command; Calculate the motion smoothness reward value and the safety constraint reward value based on the new state; The reward value is obtained by weighted summing of the joint tracking reward value, the key point tracking reward value, the task reward value, the motion smoothness reward value, and the safety constraint reward value.

6. The method as described in claim 1, characterized in that, The method further includes: The human motion data is mapped to humanoid robot motion data using motion redirection technology; The motion segment is generated based on the motion data of the humanoid robot.

7. A humanoid robot, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the humanoid robot to perform the method of any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a processor, causes the processor to perform the method as described in any one of claims 1 to 6.

Images