Humanoid robot teaching behavior generation method and system based on hierarchical auto-encoder

Abstract

The invention discloses a humanoid robot teaching behavior generation method based on a hierarchical auto-encoder, and the method achieves the independent coding and cooperative generation of the movement of different parts of a body through the multi-stream design and hierarchical quantification. A user can accurately control details of gestures, types of gaits and overall postures by adjusting discrete codes of different levels; parallel decoding is carried out by adopting non-autoregressive Transform, so that the reasoning speed of long sequence generation is greatly improved; a two-stage generation and memory enhancement mechanism is combined, so that the common problem of monotony or poor coordination in parallel generation is effectively solved, and the fluency and naturalness of actions are ensured; inverse kinematics constraints are introduced in the representation learning stage, so that the whole generation process is established on the basis of physically feasible action representation, and the possibility of generating abnormal postures violating ergonomics is fundamentally reduced.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and robotics, and more specifically, to a method and system for generating teaching behaviors of humanoid robots based on hierarchical autoencoders. Background Technology

[0002] Generating natural and fluent sequences of human movements that match semantic descriptions is a core and challenging task in fields such as robotics and human-computer interaction. In specific scenarios such as teaching, demonstrations, and storytelling, robots can convey information, express emotions, and interact with the environment through nonverbal actions, significantly enhancing the immersion and communication efficiency of interactions, and thus possessing significant application value.

[0003] To achieve the above goals, existing human motion generation technologies can be categorized as follows: One approach is based on action retrieval and concatenation. This method pre-constructs a database of labeled action segments. During generation, the semantics of the input text are first parsed, then multiple semantically matching short segments are retrieved from the database, and finally, these segments are connected into a complete sequence using motion concatenation technology.

[0004] The second method is based on motion capture and mapping (imitation learning). This method records the motion data of real performers using sensors and applies it directly or after processing to virtual characters. In recent years, with the development of video-based motion estimation technology, human motion parameters can be extracted directly from monocular videos.

[0005] Thirdly, there are data-driven generative modeling methods. This is currently the mainstream research approach in both academia and industry. These methods utilize deep learning models to directly learn the distribution patterns of actions from large-scale action data and achieve end-to-end mapping from conditions (such as text and audio) to action sequences. Among these, diffusion models have become a hot topic due to their powerful generative capabilities.

[0006] It is evident that existing methods for generating human motion can be mainly categorized into two types: those based on continuous space and those based on discrete representation. Methods based on continuous space directly perform regression or sampling in a continuous high-dimensional space, making it difficult to precisely control details, and the generated motions often exhibit jitter, unnaturalness, or physical inconsistencies. Methods based on discrete representation encode motions into discrete symbol sequences and utilize sequence models for generation, improving controllability and robustness. However, existing methods generally suffer from the following shortcomings: 1) a lack of refined and decoupled modeling capabilities for human motions with clearly defined functional partitions; 2) difficulty in balancing global coherence and local detail realism when generating long sequence motions; 3) insufficient fusion and utilization of complex multimodal conditions, leading to insufficient consistency between generated motions and semantics, scene, and style; and 4) low inference efficiency, making it difficult to meet the needs of real-time interactive applications. Summary of the Invention

[0007] To address at least one deficiency or improvement need in the existing technology, this invention provides a method, system, device, and storage medium for generating teaching behaviors of humanoid robots based on a hierarchical autoencoder, in order to systematically solve the fragmentation and shortcomings of the current technology in terms of refined control, quality and efficiency of long sequence generation, deep fusion of multimodal conditions, and physical rationality of actions.

[0008] To achieve the above objectives, according to a first aspect of the present invention, a method for generating teaching behaviors for a humanoid robot based on a hierarchical autoencoder is provided, the method comprising the following steps: S1, based on a multi-stream hierarchical vector quantization variational autoencoder, performs hierarchical vector quantization encoding on human action sequences to construct a hierarchical action codebook space; S2, the text instructions, scene layout and action style are encoded into conditional vectors, and the conditional vectors are fused into a joint conditional representation; the joint conditional representation is converted into a discrete action sequence, and a three-dimensional continuous action sequence is reconstructed based on the multi-stream hierarchical vector quantization variational autoencoder; S3, the three-dimensional continuous human motion sequence is mapped to the robot joint space, and the mapped motion sequence is fine-tuned based on the reinforcement learning model to obtain the robot kinematic parameters. A control strategy adapted to the robot's dynamic characteristics is generated based on the robot kinematic parameters.

[0009] Furthermore, step S1 of the above-mentioned method for generating teaching behaviors for humanoid robots also includes: S101, based on human kinesiology, decomposes the continuous three-dimensional human joint movement sequence into multiple sub-sequences, and encodes each sub-sequence through the corresponding encoder to obtain the continuous feature sequence of each part. S102, perform global pose quantization and local detail quantization on the continuous feature sequence to obtain a global index action sequence and a local index action sequence; S103, the global index action sequence and the local index action sequence are fused to generate a fused feature sequence, and the fused feature sequence is synthesized into a complete human action sequence based on the multi-stream decoder and human kinematics prior. S104 uses a multi-stream hierarchical vector quantization variational autoencoder to perform hierarchical vector quantization encoding on human action sequences in order to construct a hierarchical action codebook space.

[0010] Furthermore, step S102 of the above-mentioned method for generating teaching behaviors for humanoid robots also includes: The global pose quantization includes: splicing and fusing the feature sequences of each part to obtain global features; using a predefined global codebook to perform neighborhood search, and quantizing the features of each time step into discrete indices, thereby generating a global index sequence that encodes the overall pose temporal evolution. The local detail quantization includes: extracting hand features from upper body features and quantizing them using a fine hand codebook to generate a gesture detail index sequence; and extracting gait features from lower body features and quantizing them using a gait codebook to generate a gait pattern index sequence.

[0011] Furthermore, in step S2 of the above-mentioned method for generating teaching behaviors for humanoid robots, text instructions, scene layout, and action styles are encoded into conditional vectors, and the conditional vectors are fused into a joint conditional representation, specifically including: The text instructions are processed by extracting semantic features from a pre-trained language model to obtain a representation sequence of the context. The representations marked in the representation sequence are then extracted as a global semantic summary vector. Graph neural networks are used to encode the scene space that identifies key objects into global scene feature vectors that describe the location of key objects. Action style is identified as a multidimensional continuous vector, and the multilayer perceptron is used to encode the multidimensional continuous vector into an action style embedding vector; The global semantic summary vector, the global scene feature vector, and the action style embedding vector are used as condition vectors, and the condition vectors are fused into a joint condition representation.

[0012] Furthermore, in step S2 of the above-mentioned humanoid robot teaching behavior generation method, the joint conditional representation is converted into a discrete action sequence, and a three-dimensional continuous action sequence is reconstructed based on the multi-stream hierarchical vector quantization variational autoencoder, specifically including: The joint conditional representation is input into the decoder to construct a global action planning network. The global action planning network is run to obtain a global action summary sequence. The global action summary sequence is then combined with a refined network consisting of encoders for all conditional inputs, and a complete prediction sequence is output in parallel. The predicted sequence is decomposed into discrete sequences, and the discrete sequences are input into a multi-stream hierarchical vector quantization variational autoencoder to reconstruct a three-dimensional continuous action sequence.

[0013] Furthermore, in step S3 of the above-mentioned method for generating teaching behaviors for humanoid robots, mapping the three-dimensional continuous human motion sequence to the robot joint space specifically includes: A mapping table between the generated motion joints and the robot joints is established, and the differences between the robot's degrees of freedom and those of the human body are considered to perform joint fusion and decomposition; for the pose at each time step, the robot joint angles are calculated using an inverse kinematics solver.

[0014] Furthermore, in step S3 of the above-mentioned method for generating teaching behaviors for humanoid robots, the kinematic parameters of the robot are obtained by fine-tuning the mapped action sequence based on a reinforcement learning model, specifically including: Define the robot's state and action space, design a multi-objective reward function, and then train a reinforcement learning model. Based on the pre-trained reinforcement learning model, fine-tune the mapped action sequence to obtain the robot's kinematic parameters.

[0015] According to a second aspect of the present invention, a system for generating teaching behaviors for a humanoid robot based on a hierarchical autoencoder is also provided, the system comprising: The action generation module is configured to perform hierarchical vector quantization encoding on human action sequences based on a multi-stream hierarchical vector quantization variational autoencoder to construct a hierarchical action codebook space. The motion reconstruction module is configured to encode text instructions, scene layout, and motion style into conditional vectors, and fuse the conditional vectors into a joint conditional representation; convert the joint conditional representation into a discrete motion sequence, and reconstruct a three-dimensional continuous motion sequence based on the multi-stream hierarchical vector quantization variational autoencoder; The motion control module is configured to map the three-dimensional continuous human motion sequence to the robot joint space, fine-tune the mapped motion sequence based on a reinforcement learning model to obtain robot kinematic parameters, and generate a control strategy adapted to the robot's dynamic characteristics based on the robot kinematic parameters.

[0016] According to a third aspect of the present invention, a humanoid robot teaching behavior generation system device based on a hierarchical autoencoder is also provided, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit performs the steps of any of the methods described above.

[0017] According to a fourth aspect of the present invention, a computer program product is also provided, comprising a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, they implement the steps of any of the methods described above.

[0018] Overall, compared with existing technologies, the technical solutions conceived in this invention achieve the following beneficial effects: 1. Refined and decoupled control: Through multi-stream design and hierarchical quantization, independent encoding and collaborative generation of movements in different parts of the body are realized. Users can precisely control the details of gestures, gait types, and overall posture by adjusting discrete codes at different levels. 2. High efficiency and high-quality parallelism: Parallel decoding using a non-autoregressive Transformer greatly improves the inference speed of long sequence generation; combined with two-stage generation and memory enhancement mechanisms, it effectively solves the common problems of monotony or poor coordination in parallel generation, ensuring the smoothness and naturalness of the movements. 3. Inherent physical rationality: Inverse kinematic constraints are introduced during the representation learning stage, making the entire generation process based on physically feasible action representations, fundamentally reducing the possibility of generating abnormal postures that violate ergonomics. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a method for generating teaching behaviors of a humanoid robot based on a hierarchical autoencoder, provided in an embodiment of this application; Figure 2 A flowchart illustrating a method for generating teaching behaviors of a humanoid robot based on a hierarchical autoencoder, provided in an embodiment of this application; Figure 3 This is a schematic diagram of the action representation learning process provided in the embodiments of this application; Figure 4 A schematic diagram of a humanoid robot teaching behavior generation system based on a hierarchical autoencoder is provided for an embodiment of this application; Detailed Implementation To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0022] Based on the analysis of existing technologies, particularly the bottlenecks in data-driven generative modeling methods, the purpose of this invention is to provide an end-to-end human motion sequence generation solution. This solution systematically addresses the shortcomings and deficiencies of current technologies in areas such as refined control, long sequence generation quality and efficiency, deep fusion of multimodal conditions, and physical plausibility of movements. It overcomes the deficiencies of existing human motion generation methods in terms of refined control, long sequence coherence, multimodal condition fusion, and generation efficiency, providing a method and system capable of automatically generating high-fidelity, interpretable, physically constrained, and coordinated full-body human motion sequences from multimodal inputs such as teaching content, scene, and style.

[0023] To achieve the above objectives, this invention aims to specifically solve the following four interrelated key technical problems that are directly caused by the deficiencies of the prior art: (1) To address the problem that existing generative models cannot decouple and independently and finely control the movements of different functional parts of the body because they encode human movements as a whole and uniformly. Existing methods usually treat the high-dimensional data of all human joints as a whole for feature learning and generation, which makes it difficult for the model to focus on and accurately control the local motion patterns that are crucial in the teaching scenario, thus limiting the controllability and applicability of action generation.

[0024] (2) To address the problems of slow inference speed and inability to meet real-time interaction requirements of existing autoregressive sequence generation models, and the problems of poor overall coherence caused by the lack of high-level temporal planning in existing non-autoregressive or parallel generation models when generating long sequences. Specifically, it is necessary to overcome the inherent sequential computation bottleneck of autoregressive models, while avoiding the splicing of action segments, rhythm chaos and loss of long-range dependencies that may occur in non-autoregressive models when all frames are generated in parallel at once.

[0025] (3) To address the problem that existing methods often use simple splicing or shallow feature alignment to fuse multimodal conditions such as text instructions, scene spatial layout, and personalized action styles, resulting in insufficient fit between generated actions and complex task contexts and personalized expression styles, and the generation results are monotonous. Existing technologies have failed to effectively establish a deep and structured relationship between multimodal conditions and multi-level action generation processes.

[0026] (4) The problem is that purely data-driven generative models lack explicit human kinematic and dynamic constraints, which can easily produce non-executable actions that violate physical laws, such as joint angle exceeding limits, foot sliding, and limb penetration. As a result, the generated results cannot be directly and reliably applied to virtual character driving or physical robot control, and often require complex and time-consuming post-processing correction.

[0027] This invention, through a hierarchical discrete generation architecture described later, aims to comprehensively and collaboratively solve the four technical problems mentioned above, providing a human motion sequence generation scheme that combines high controllability, high generation efficiency, strong contextual relevance, and inherent physical rationality. It is particularly suitable for generating complex, continuous, and physically constrained robot motion sequences using multimodal instructions (such as text, style, and scene).

[0028] To achieve the above objectives, the technical solution adopted by this invention is a method and system for generating teaching behaviors of humanoid robots based on hierarchical autoencoders. The system processing flow is shown in the attached figure. Figure 2 As shown, the core idea of ​​this method is to decompose the complex generation problem of high-dimensional continuous action space into two structured sub-stages: First, a hierarchical action codebook space with rich semantics is constructed through multi-granularity discretization representation learning; second, within this discrete space, reasonable action code sequences are predicted under various conditions using an advanced sequence generation model, and finally, the decoder restores them into continuous actions. This method not only improves the quality and controllability of generated actions but also significantly improves inference efficiency.

[0029] According to a first aspect of the present invention, a method for generating teaching behaviors of a humanoid robot based on a hierarchical autoencoder is provided, as shown in the appendix. Figure 1 As shown, the method includes the following steps: S1 involves using a multi-stream hierarchical vector quantization variational autoencoder to perform hierarchical vector quantization encoding on the human action sequence to construct a hierarchical action codebook space. It should be noted that this step primarily implements action representation learning based on the multi-stream hierarchical vector quantization variational autoencoder. The goal of this stage is to learn a compressed, discretized, and hierarchically semantic representation of human actions. The processing procedure is shown in the attached figure. Figure 3 As shown.

[0030] S2, text instructions, scene layout, and action style are encoded into conditional vectors, and the conditional vectors are fused into a joint conditional representation; the joint conditional representation is converted into a discrete action sequence, and a three-dimensional continuous action sequence is reconstructed based on the multi-stream hierarchical vector quantization variational autoencoder; it should be noted that this step mainly realizes action sequence prediction based on memory-enhanced non-autoregressive Transformer. In this stage, the discrete codebook space learned in the first stage is used to transform the action generation task into a conditional sequence to sequence prediction problem.

[0031] S3, the three-dimensional continuous human motion sequence is mapped to the robot joint space. Based on a reinforcement learning model, the mapped motion sequence is fine-tuned to obtain the robot's kinematic parameters. A control strategy adapted to the robot's dynamic characteristics is then generated based on these kinematic parameters. It should be noted that this step mainly realizes the mapping of motion sequences and the execution of physical robot actions. The goal of this stage is to generate a continuous human motion sequence... Convert the commands into control instructions that can be executed by the physical robot, and ensure the stability, safety and naturalness of the actions in the real physical environment.

[0032] Furthermore, step S1 of the above-mentioned method for generating teaching behaviors for humanoid robots also includes: S101, based on human kinesiology, decomposes the continuous three-dimensional human joint movement sequence into multiple sub-sequences, and encodes each sub-sequence using a corresponding encoder to obtain continuous feature sequences for each part; it should be noted that this step mainly realizes the decomposition of movement sequences and feature extraction. The input continuous three-dimensional human joint movement sequence... Based on human kinesiology, it is divided into three sub-sequences: the upper body sequence, which mainly carries expressive gestures. The lower body sequence that dominates movement and posture And the torso sequence that maintains balance and connects the upper and lower body. Each subsequence is processed by a dedicated encoder. The dedicated encoder employs a hybrid architecture combining one-dimensional convolutional neural networks and graph convolutional networks to simultaneously capture the temporal dynamics of the motion and the spatial topological relationships between joints, outputting continuous feature sequences of the corresponding parts. .

[0033] S102, the continuous feature sequence is subjected to global pose quantization and local detail quantization to obtain a global index action sequence and a local index action sequence; it should be noted that this step mainly implements hierarchical vector quantization encoding. This step maps continuous features to a discrete codebook space to form a multi-level action representation.

[0034] S103, the global indexed action sequence and the local indexed action sequence are fused to generate a fused feature sequence. Based on the multi-stream decoder and human kinematics prior, the fused feature sequence is synthesized into a complete human action sequence. It should be noted that this step mainly realizes action reconstruction and fusion. The quantized features of the above three levels (global, hand, gait) are mapped to a unified dimension through a learnable projection layer and then added element-wise to obtain the fused feature sequence. The fused features are input to a multi-stream decoder, which contains three branches symmetrical to the encoder structure, responsible for reconstructing the movements of the upper body, lower body, and torso, respectively. Finally, a pose fusion network receives the outputs of these three branches and, based on learned human kinematics priors, synthesizes a natural, coordinated, and physically plausible sequence of complete human poses. .

[0035] S104, based on a multi-stream hierarchical vector quantization variational autoencoder, the human action sequence is encoded using hierarchical vector quantization to construct a hierarchical action codebook space. It should be noted that this step mainly achieves multi-objective joint optimization. The multi-stream hierarchical vector quantization variational autoencoder is trained using a composite loss function. The function is defined as follows:

[0036] The functions of each part are as follows: The basic reconstruction loss uses a smoothed L1 loss to reduce the impact of outliers, and a velocity loss is added to force the generated actions to be smooth in time.

[0037] The vector quantization loss for each codebook consists of the codebook learning loss and the encoder commitment loss, ensuring that the encoder output aligns with the codebook vectors, while the codebook vectors converge towards the encoder output.

[0038] Loss of upper and lower body coordination. In teaching scenarios, such as when a finger points to the right, the torso and gaze usually turn slightly to the right as well. This loss is addressed by calculating the correlation between upper and lower body joint movements to encourage this natural coordination.

[0039] Inverse kinematics constraint loss. To ensure the physical plausibility of the generated motion and the coordination of joint movements, a differentiable constraint based on inverse kinematics is introduced. This loss function is defined as:

[0040] in, For the end effector at time The target location can be extracted from training data or obtained through scene condition inference. It is a positive kinematic function, based on the angle of the skeletal joints. Calculate the end position. For the set of key end effectors, To balance the weights.

[0041] Furthermore, step S102 of the above-mentioned method for generating teaching behaviors for humanoid robots also includes: The global pose quantization includes: concatenating and fusing the feature sequences of each part to obtain global features; performing neighborhood search using a predefined global codebook, and quantizing the features at each time step into discrete indices, thereby generating a global index sequence that encodes the overall pose temporal evolution; it should be noted that global pose quantization includes: firstly, concatenating and fusing the feature sequences of the three parts to obtain global features. Subsequently, a predefined global codebook is used. right Perform nearest neighbor search, quantizing the features at each time step into a discrete index. This generates a global index sequence that encodes the overall attitude temporal evolution. .

[0042] The local detail quantization includes: extracting hand features from upper body features and quantizing them using a fine hand codebook to generate a gesture detail index sequence; extracting gait features from lower body features and quantizing them using a gait codebook to generate a gait pattern index sequence. It should be noted that local detail quantization includes: to preserve fine motion features, two local quantization processes are performed in parallel: (a) from upper body features... Extracting hand features And using a specially designed, finely detailed codebook for hands. Quantization is performed to generate a sequence of gesture detail indexes. (b) From the characteristics of the lower body Extracting gait features And using a dedicated gait codebook Quantization is performed to generate a gait pattern index sequence. .

[0043] Furthermore, in step S2 of the above-mentioned method for generating teaching behaviors for humanoid robots, text instructions, scene layout, and action styles are encoded into conditional vectors, and the conditional vectors are fused into a joint conditional representation, specifically including: Text instructions are processed by extracting semantic features from a pre-trained language model to obtain a representation sequence of the context. The labeled representations within this sequence are then extracted as a global semantic summary vector. It should be noted that semantic features are extracted using a pre-trained language model. Define a text instruction as a... A sequence of words or sub-word tags: First, by looking up the table. Each tag Mapped to word embedding vectors Then embed the sequence Input pre-trained language model encoder Obtain a contextualized representation sequence ,in Finally, take the sequence. The representation of the tokens serves as a global semantic summary: .

[0044] A graph neural network is used to encode the scene space containing key objects into global scene feature vectors describing the locations of the key objects. It should be noted that this step primarily implements scene layout, encoding it into spatial features describing the locations of key objects. The defined scenario includes... One key object, each object Represented by a triple: .in: Indicates the object type identifier. The center position coordinates of the object in the normalized 3D scene coordinate system , Normalized dimensions of the object Object type is mapped through a learnable embedding layer: Geometric properties are mapped through a multilayer perceptron: Initial characteristics of the object Initial features of all objects Viewed as nodes in a graph, through Layered graph neural networks update node features in This refers to the adjacency matrix or relation weights. It applies to the updated node features. Pooling is performed to obtain global scene features. .

[0045] The movement style is identified as a multidimensional continuous vector, and the multilayer perceptron is used to encode the multidimensional continuous vector into a movement style embedding vector. It should be noted that this step mainly achieves movement style identification and encoding of the embedding vector describing the personalized movement style. Let action style be a multidimensional continuous vector. Each dimension controls a style attribute. Embedding vectors are obtained using a multilayer perceptron. .

[0046] The global semantic summary vector, the global scene feature vector, and the action style embedding vector are used as condition vectors, and these condition vectors are fused into a joint condition representation. It should be noted that, since the above condition vectors are inconsistent in dimension, this scheme fuses them into a unified joint condition representation. ,in For linear projection layers, These are the parameters for the fusion layer.

[0047] Furthermore, in step S2 of the above-mentioned humanoid robot teaching behavior generation method, the joint conditional representation is converted into a discrete action sequence, and a three-dimensional continuous action sequence is reconstructed based on the multi-stream hierarchical vector quantization variational autoencoder. This mainly includes a two-stage action sequence generation, employing a two-step strategy of planning first and then refining. Specifically, it includes: The joint conditional representation is input into the decoder to construct a global action planning network. Running the global action planning network yields a global action summary sequence. This global action summary sequence is then combined with a refined network composed of encoders, inputting all conditions, and outputting a complete prediction sequence in parallel. It should be noted that this step primarily achieves high-level planning generation. The fused multi-conditional representation... The input is an autoregressive Transformer decoder used as the planning network. This planning network generates a length of [length missing] in an autoregressive manner. High-level behavioral summary sequence Each It is a discrete symbol representing a macroscopic behavioral unit, providing a temporal structure and a global intent framework for subsequent detail generation.

[0048] The predicted sequence is decomposed into discrete sequences, and these discrete sequences are input into a multi-stream hierarchical vector quantization variational autoencoder to reconstruct a 3D continuous action sequence. It should be noted that this step primarily achieves detailed action generation. This step is accomplished by a memory-enhanced non-autoregressive Transformer encoder refinement network. First, the target to be generated, i.e., the sequence derived from the predicted sequence... A long sequence interwoven together along the time dimension A high-ratio random mask is applied. Then, the model input is constructed, which contains, in sequence: (1) A set of learnable knowledge prefix vectors , used to inject general action priors. It is a set of a fixed number of learnable vectors, denoted as Each of them , It is the dimension of the hidden layer of the Transformer model.

[0049] (2) It integrates text, scene, style and planning sequence Conditional feature representation ; (3) The masked target sequence The model uses its multi-layered attention mechanism to predict the probability distribution of all masked locations in parallel.

[0050] Furthermore, step S2 of the above-mentioned method for generating teaching behaviors of humanoid robots also includes a memory enhancement mechanism and a training and reasoning process.

[0051] In the memory enhancement mechanism, to improve the diversity of generated actions and the ability to generalize to uncommon conditions, an external memory retrieval module is introduced into a specific layer of the refined network. This mechanism includes: Memory construction: During the training phase, the real action sequences in the training data are discretized. and the corresponding conditions Stored as memory entries, forming a memory bank. .

[0052] Retrieval and Injection: During the generation process, a query vector is calculated based on the current condition representation, from... The most similar one was retrieved from the search results. Each memory entry is analyzed. The action sequence features corresponding to these entries are then aggregated into a memory context. Furthermore, it injects this into the intermediate hidden state of the Transformer through an additional cross-attention layer, thereby leveraging similar patterns from historical real data to guide the current generation.

[0053] In the training and inference process, training includes refining the network so that the training objective is to minimize the negative log-likelihood loss at the mask positions. This means teaching the model to predict the correct discrete action codes based on context and multiple conditions. Inference includes: during inference, first running the planning network to obtain... Then, the fully masked sequence and all conditions are input into the refinement network, and the complete prediction sequence is output in parallel at once. Then it is decomposed back into a three-channel discrete sequence. Finally, this discrete sequence is input into the VQ-VAE decoder trained in the first stage to reconstruct the final three-dimensional continuous action sequence. .

[0054] Furthermore, in step S3 of the above-mentioned method for generating teaching behaviors for humanoid robots, mapping the three-dimensional continuous human motion sequence to the robot joint space specifically includes: A mapping table is established between the generated motion joints and the robot joints. Considering the differences between the robot's degrees of freedom and those of the human body, joint fusion and decomposition are performed. For the pose at each time step, the robot joint angles are calculated using an inverse kinematics solver. It should be noted that this step primarily achieves the mapping from the motion sequence to the robot's joint space. The generated... It is a three-dimensional joint motion sequence based on a standard human skeletal model, which needs to be adapted to the specific configuration of the target robot. Assume the robot has... There are 4 joints, each with restricted degrees of freedom. The robot's joint states are defined as follows: The mapping process includes the following steps: (1) Establishing the correspondence between joints: Establishing a mapping table between the generated motion joints and the robot joints. Furthermore, taking into account the differences between the robot's degrees of freedom and those of the human body, appropriate joint fusion or decomposition is performed.

[0055] (2) Inverse Kinematics and Optimization: For the posture at each time step, the robot's joint angles are calculated using an inverse kinematics solver. Physical constraints such as joint angle limits, self-collision avoidance, and center of mass stability are introduced to construct an optimization problem:

[0056] in, and These are the forward kinematics functions for the robot and the human body, respectively. This is a stability loss term.

[0057] Furthermore, in step S3 of the above-mentioned method for generating teaching behaviors for humanoid robots, the kinematic parameters of the robot are obtained by fine-tuning the mapped action sequence based on a reinforcement learning model, specifically including: The robot's state and action space are defined, and a multi-objective reward function is designed to train a reinforcement learning model. Based on the pre-trained reinforcement learning model, the mapped action sequence is fine-tuned to obtain the robot's kinematic parameters. It should be noted that this step mainly achieves fine-tuning and control of actions based on reinforcement learning. To ensure robust execution of actions in a real environment, a reinforcement learning module is introduced to fine-tune the mapped actions in a simulation environment, learning a control strategy adapted to the robot's dynamic characteristics.

[0058] (1) Definition of state and action space. State This includes joint angles, angular velocity, end-effector position, contact force, and center of mass position. (Motion) Target joint position increment or torque output.

[0059] (2) Reward Function Design. Design a multi-objective reward function to encourage faithful execution of actions, energy efficiency, and stability:

[0060] in, To track the rewards for generated actions, As a penalty for energy consumption, Maintaining a balance to preserve rewards Rewards for smooth motion.

[0061] (3) Training and Deployment. The policy network was trained in the MuJoCo simulation environment using DDPG policy gradients. After training, the policy network was deployed to the real robot controller.

[0062] According to a second aspect of the present invention, a humanoid robot teaching behavior generation system based on a hierarchical autoencoder is also provided, as shown in the appendix. Figure 4 As shown, the system includes: The action generation module is configured to perform hierarchical vector quantization encoding on human action sequences based on a multi-stream hierarchical vector quantization variational autoencoder to construct a hierarchical action codebook space. The motion reconstruction module is configured to encode text instructions, scene layout, and motion style into conditional vectors, and fuse the conditional vectors into a joint conditional representation; convert the joint conditional representation into a discrete motion sequence, and reconstruct a three-dimensional continuous motion sequence based on the multi-stream hierarchical vector quantization variational autoencoder; The motion control module is configured to map the three-dimensional continuous human motion sequence to the robot joint space, fine-tune the mapped motion sequence based on a reinforcement learning model to obtain robot kinematic parameters, and generate a control strategy adapted to the robot's dynamic characteristics based on the robot kinematic parameters.

[0063] It should be noted that the humanoid robot teaching behavior generation system based on hierarchical autoencoder of the present invention is implemented in accordance with the steps in the aforementioned humanoid robot teaching behavior generation method based on hierarchical autoencoder, and will not be repeated here.

[0064] Overall, compared with existing technologies, the technical solutions conceived in this invention achieve the following beneficial effects: 1. Refined and decoupled control: Through multi-stream design and hierarchical quantization, independent encoding and collaborative generation of movements in different parts of the body are realized. Users can precisely control the details of gestures, gait types, and overall posture by adjusting discrete codes at different levels. 2. High efficiency and high-quality parallelism: Parallel decoding using a non-autoregressive Transformer greatly improves the inference speed of long sequence generation; combined with two-stage generation and memory enhancement mechanisms, it effectively solves the common problems of monotony or poor coordination in parallel generation, ensuring the smoothness and naturalness of the movements. 3. Inherent physical rationality: Inverse kinematic constraints are introduced during the representation learning stage, making the entire generation process based on physically feasible action representations, fundamentally reducing the possibility of generating abnormal postures that violate ergonomics.

[0065] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0066] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0067] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0069] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0070] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0071] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0072] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0073] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0074] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating teaching behaviors for humanoid robots based on hierarchical autoencoders, characterized in that, The method includes the following steps: S1, based on a multi-stream hierarchical vector quantization variational autoencoder, performs hierarchical vector quantization encoding on human action sequences to construct a hierarchical action codebook space; S2, the text instructions, scene layout and action style are encoded into conditional vectors, and the conditional vectors are fused into a joint conditional representation; the joint conditional representation is converted into a discrete action sequence, and a three-dimensional continuous action sequence is reconstructed based on the multi-stream hierarchical vector quantization variational autoencoder; S3, the three-dimensional continuous human motion sequence is mapped to the robot joint space, and the mapped motion sequence is fine-tuned based on the reinforcement learning model to obtain the robot kinematic parameters. A control strategy adapted to the robot's dynamic characteristics is generated based on the robot kinematic parameters.

2. The method for generating teaching behaviors for humanoid robots as described in claim 1, characterized in that, Step S1 also includes: S101, based on human kinesiology, decomposes the continuous three-dimensional human joint movement sequence into multiple sub-sequences, and encodes each sub-sequence through the corresponding encoder to obtain the continuous feature sequence of each part. S102, perform global pose quantization and local detail quantization on the continuous feature sequence to obtain a global index action sequence and a local index action sequence; S103, the global index action sequence and the local index action sequence are fused to generate a fused feature sequence, and the fused feature sequence is synthesized into a complete human action sequence based on the multi-stream decoder and human kinematics prior. S104 uses a multi-stream hierarchical vector quantization variational autoencoder to perform hierarchical vector quantization encoding on human action sequences in order to construct a hierarchical action codebook space.

3. The method for generating teaching behaviors for humanoid robots as described in claim 2, characterized in that, Step S102 also includes: The global pose quantization includes: splicing and fusing the feature sequences of each part to obtain global features; using a predefined global codebook to perform neighborhood search, and quantizing the features of each time step into discrete indices, thereby generating a global index sequence that encodes the overall pose temporal evolution. The local detail quantization includes: extracting hand features from upper body features and quantizing them using a fine hand codebook to generate a gesture detail index sequence; and extracting gait features from lower body features and quantizing them using a gait codebook to generate a gait pattern index sequence.

4. The method for generating teaching behaviors for humanoid robots as described in claim 1, characterized in that, In step S2, the text instructions, scene layout, and action style are encoded into condition vectors, and the condition vectors are fused into a joint condition representation, specifically including: Text instructions are processed by extracting semantic features from a pre-trained language model to obtain a representation sequence of the context. The marked representations in the representation sequence are then extracted as a global semantic summary vector. Graph neural networks are used to encode the scene space that identifies key objects into global scene feature vectors that describe the location of key objects. Action style is identified as a multidimensional continuous vector, and the multilayer perceptron is used to encode the multidimensional continuous vector into an action style embedding vector; The global semantic summary vector, the global scene feature vector, and the action style embedding vector are used as condition vectors, and the condition vectors are fused into a joint condition representation.

5. The method for generating teaching behaviors for humanoid robots as described in claim 1, characterized in that, In step S2, the joint conditional representation is converted into a discrete action sequence, and a three-dimensional continuous action sequence is reconstructed based on the multi-stream hierarchical vector quantization variational autoencoder, specifically including: The joint conditional representation is input into the decoder to construct a global action planning network. The global action planning network is run to obtain a global action summary sequence. The global action summary sequence is then combined with a refined network consisting of encoders for all conditional inputs, and a complete prediction sequence is output in parallel. The predicted sequence is decomposed into discrete sequences, and the discrete sequences are input into a multi-stream hierarchical vector quantization variational autoencoder to reconstruct a three-dimensional continuous action sequence.

6. The method for generating teaching behaviors for humanoid robots as described in claim 1, characterized in that, In step S3, the three-dimensional continuous human motion sequence is mapped to the robot joint space, specifically including: A mapping table between the generated motion joints and the robot joints is established, and the differences between the robot's degrees of freedom and those of the human body are considered to perform joint fusion and decomposition; for the pose at each time step, the robot joint angles are calculated using an inverse kinematics solver.

7. The method for generating teaching behaviors for humanoid robots as described in claim 1, characterized in that, In step S3, the mapped action sequence is fine-tuned based on the reinforcement learning model to obtain the robot's kinematic parameters, specifically including: Define the robot's state and action space, design a multi-objective reward function, and then train a reinforcement learning model. Based on the pre-trained reinforcement learning model, fine-tune the mapped action sequence to obtain the robot's kinematic parameters.

8. A humanoid robot teaching behavior generation system based on a hierarchical autoencoder, characterized in that, The system includes: The action generation module is configured to perform hierarchical vector quantization encoding on human action sequences based on a multi-stream hierarchical vector quantization variational autoencoder to construct a hierarchical action codebook space. The motion reconstruction module is configured to encode text instructions, scene layout, and motion style into conditional vectors, and fuse the conditional vectors into a joint conditional representation; convert the joint conditional representation into a discrete motion sequence, and reconstruct a three-dimensional continuous motion sequence based on the multi-stream hierarchical vector quantization variational autoencoder; The motion control module is configured to map the three-dimensional continuous human motion sequence to the robot joint space, fine-tune the mapped motion sequence based on a reinforcement learning model to obtain robot kinematic parameters, and generate a control strategy adapted to the robot's dynamic characteristics based on the robot kinematic parameters.

9. A humanoid robot teaching behavior generation device based on a hierarchical autoencoder, characterized in that, The method includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.

Images