Robot motion sequence generation method, storage medium and electronic equipment

Abstract

The invention provides a robot motion sequence generation method, a storage medium and electronic equipment, and relates to the field of computer animation. The robot motion sequence generation method comprises the following steps: determining a first motion sequence; a space-time mask matrix corresponding to the first motion sequence is constructed, the dimension of the space-time mask matrix corresponds to the time dimension and the space dimension of the first motion sequence, and elements in the space-time mask matrix are used for indicating to maintain or repair motion data of corresponding space-time positions in the first motion sequence; and processing the first motion sequence by using a diffusion model based on the space-time mask matrix to obtain a second motion sequence of the robot.

Description

Technical Field

[0001] This application relates to the field of computer animation technology, specifically to a method for generating robot motion sequences, a storage medium, and an electronic device. Background Technology

[0002] Since the introduction of motion capture technology, motion synthesis methods have evolved from early splicing schemes based on motion graphs and motion matching to a parameterized modeling system centered on deep learning. Currently, denoising diffusion models dominate applications, demonstrating unprecedented generative flexibility in scenarios such as trajectory control, multi-person interaction, and even non-human characters.

[0003] However, existing diffusion models still face two major shortcomings: first, keyframe constraints are difficult to embed flexibly, and the accuracy of local pose and end contact is insufficient; second, the inference steps are lengthy and the generation efficiency is low. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method for generating robot motion sequences, a storage medium, and an electronic device.

[0005] In a first aspect, one embodiment of this application provides a method for generating a robot motion sequence, comprising: determining a first motion sequence; constructing a spatiotemporal mask matrix corresponding to the first motion sequence, wherein the dimensions of the spatiotemporal mask matrix correspond to the time dimension and the spatial dimension of the first motion sequence, and the elements in the spatiotemporal mask matrix are used to indicate motion data at corresponding spatiotemporal positions in the first motion sequence for preservation or repair; and processing the first motion sequence using a diffusion model based on the spatiotemporal mask matrix to obtain a second motion sequence of the robot.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, determining the first motion sequence includes: obtaining a third motion sequence in a global coordinate system, where the global coordinate system represents a fixed reference system independent of the robot body and is used to describe the robot's absolute position and orientation; and performing a trajectory representation transformation on the third motion sequence in the global coordinate system to obtain the first motion sequence in a local coordinate system, where the local coordinate system represents a reference system based on the robot and is used to describe the motion of various parts of the robot relative to the robot body.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, a trajectory representation transformation is performed on the third motion sequence in the global coordinate system to obtain the first motion sequence in the local coordinate system, including: decomposing the root trajectory in the third motion sequence to obtain the horizontal plane component; performing rotational decoupling on the third motion sequence to extract the principal axis rotation component from the global orientation of the third motion sequence and obtain the local global orientation; obtaining the local velocity trajectory and velocity angle based on the horizontal plane component and the principal axis rotation component; and using the local global orientation, local velocity trajectory, and velocity angle as the first motion sequence in the local coordinate system.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, obtaining the third motion sequence in the global coordinate system includes: obtaining the fourth motion sequence in the global coordinate system; constructing a preset posture sequence, wherein the preset posture sequence includes a standing posture; adding the preset posture sequence to the start and end of the fourth motion sequence respectively to obtain the third motion sequence in the global coordinate system.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the spatiotemporal mask matrix is ​​used to implement at least one of the following spatial control granularities: body region level control for controlling a body region containing multiple joints; joint group level control for controlling a joint group performing a specific function; and single joint level control for controlling the motion parameters of a single joint.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, constructing a spatiotemporal mask matrix corresponding to the first motion sequence includes: assigning a first value and a second value to the elements in the spatiotemporal mask matrix, wherein the first value is used to indicate the motion data at the corresponding spatiotemporal position in the first motion sequence, and the second value is used to indicate the motion data at the corresponding spatiotemporal position in the first motion sequence to be repaired; wherein, the priority of assigning the first value to the motion data at the same spatiotemporal position is higher than the priority of assigning the second value.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, constructing a spatiotemporal mask matrix corresponding to the first motion sequence includes: when the first motion sequence contains an end joint whose angle value does not conform to the motion stability rule after motion redirection, identifying the end joint whose angle value does not conform to the motion stability rule and the target spatiotemporal position corresponding to the end joint; setting the elements in the spatiotemporal mask matrix of the target spatiotemporal position to a second value.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, based on the spatiotemporal mask matrix, a diffusion model is used to process the first motion sequence to obtain the robot's second motion sequence, including: if the length of the first motion sequence exceeds a target length threshold, dividing the first motion sequence into multiple first sequence segments with overlapping regions; based on the spatiotemporal mask matrix corresponding to each first sequence segment, processing the first sequence segments using a diffusion model to obtain second sequence segments; and weighted fusing adjacent second sequence segments in the overlapping region to obtain the robot's second motion sequence.

[0013] Secondly, one embodiment of this application provides a robot motion sequence generation device, comprising: a determining module for determining a first motion sequence; a matrix construction module for constructing a spatiotemporal mask matrix corresponding to the first motion sequence, wherein the dimensions of the spatiotemporal mask matrix correspond to the time dimension and the spatial dimension of the first motion sequence, and the elements in the spatiotemporal mask matrix are used to indicate motion data at corresponding spatiotemporal positions in the first motion sequence that are maintained or repaired; and a processing module for processing the first motion sequence based on the spatiotemporal mask matrix using a diffusion model to obtain a second motion sequence of the robot.

[0014] Thirdly, one embodiment of this application provides a computer-readable storage medium storing a computer program for executing the robot motion sequence generation method described in the first aspect.

[0015] Fourthly, one embodiment of this application provides an electronic device, which includes: a processor; a memory for storing processor-executable instructions; the processor is used to execute the robot motion sequence generation method described in the first aspect.

[0016] Fifthly, one embodiment of this application provides a computer program product including instructions that, when executed on an electronic device, cause the electronic device to implement the robot motion sequence generation method described in the first aspect.

[0017] In this application, by constructing a mask matrix aligned with the spatiotemporal dimension of the motion sequence, it is possible to specify with high precision the joints that need to be strictly preserved (such as foot contact points) and the joints that need to be repaired or filled. This provides a general and highly flexible solution for the accurate reproduction and flexible generation of robot motion sequences, enabling partial posture control of any joint at any time, thereby generating motion that satisfies both precise spatial constraints and natural, smooth movements. Furthermore, by combining spatiotemporal masks with a diffusion model, this method reduces the dependence on large-scale pairwise training data. Only a small number of unlabeled motion sequences are needed to perform adaptive fine-tuning on the target robot, thus significantly shortening the deployment cycle and reducing data costs. Attached Figure Description

[0018] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 The diagram shown is a flowchart of a robot motion sequence generation method provided in an embodiment of this application.

[0020] Figure 2 The diagram shown is an exemplary schematic of a robot motion sequence generation method provided in an embodiment of this application.

[0021] Figure 3 The diagram shown is a flowchart illustrating the determination of a first motion sequence according to an embodiment of this application.

[0022] Figure 4 This is a schematic diagram of the trajectory representation conversion process provided in an embodiment of this application.

[0023] Figure 5 This is an exemplary schematic diagram illustrating trajectory representation conversion provided in an embodiment of this application.

[0024] Figure 6 The diagram shown is a flowchart of obtaining a third motion sequence according to an embodiment of this application.

[0025] Figure 7 The diagram shown is a schematic flowchart of a diffusion model for processing a first motion sequence according to an embodiment of this application.

[0026] Figure 8 The diagram shown is a schematic diagram of the structure of a robot motion sequence generation device provided in an embodiment of this application.

[0027] Figure 9 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] Since the advent of motion capture technology, numerous methods have been developed to meet user needs by temporally stitching together captured motion fragments, thereby creating human motion data. Specifically, motion graphs can pre-compute feasible motion transitions and then synthesize motion through methods such as search and dynamic programming; motion matching can search for the most suitable frame data for the current context; motion blending methods further allow for motion interpolation, with radial basis function kernels used to interpolate motions of the same category.

[0030] Furthermore, deep learning methods have proliferated in the field of animation. Human motion synthesis models are typically trained using large-scale motion capture data collections. Numerous parameterized models, including Recurrent Neural Networks (RNNs), Autoencoders, and Generative Adversarial Networks (GANs), have been proposed for motion modeling. Inspired by flow-based image synthesis models, autoregressive normalization networks have been developed for motion sequence modeling. In addition to the above, denoising diffusion models have also been widely used in motion synthesis, enabling novel control over motion generation. Their flexibility has been validated in trajectory and joint control, multi-person interaction, and non-human motion synthesis.

[0031] However, current diffusion modeling techniques still have significant shortcomings. For example, some models lack sufficient control precision and flexibility, making it difficult to effectively integrate spatiotemporal keyframe constraints, often leading to unnatural motion and foot slippage. Moreover, most methods are limited to fixed keyframe modes or only support partial posture control such as root joints. Secondly, some end-effector contact point control capabilities are weak, failing to accurately handle the angles of key parts such as feet and hands, easily producing motion artifacts such as slippage and floating, severely affecting the realism of motion. Furthermore, although some models can achieve fine control, their inference efficiency is low. The additional embedding modules and repetitive guidance mechanisms introduced to strengthen constraints significantly increase computational complexity and generation time, limiting the effectiveness of practical applications.

[0032] In summary, the existing technologies are insufficient in terms of control precision and flexibility, and the few methods that can achieve fine control are too complex and slow. Therefore, this application is submitted. Figure 1 The diagram shown is a flowchart illustrating a robot motion sequence generation method according to an embodiment of this application. Figure 1 As shown, the method includes the following steps.

[0033] Step S110: Determine the first motion sequence.

[0034] The first motion sequence refers to a set of continuous motion data arranged in temporal order. For example, the first motion sequence may include a matrix composed of several discrete-time pose vectors of the object to be imitated, arranged in temporal order. The pose vectors include, but are not limited to, rotation angles, joint positions, root node displacements, or any combination thereof. Optionally, the first motion sequence is obtained from the motion data of real humans, animated characters, or other robots to be imitated.

[0035] In some embodiments, the first motion sequence can be determined by motion capture. For example, the first motion sequence can be acquired using an optical motion capture system, an inertial measurement unit (IMU) array, a depth camera, a magnetic sensing device, or a video-based pose estimation algorithm, and then transformed into a matrix matching the input dimensions of the diffusion model through preprocessing operations such as noise filtering and coordinate system optimization. Alternatively, the first motion sequence can be directly obtained from a third-party motion database. Alternatively, the first motion sequence can be determined based on a third motion sequence. For further explanation on how to determine the first motion sequence based on a third motion sequence, see [link to documentation]. Figure 3 And its related descriptions.

[0036] Step S120: Construct the spatiotemporal mask matrix corresponding to the first motion sequence.

[0037] In some embodiments, the dimensions of the spatiotemporal mask matrix correspond to the time and spatial dimensions of the first sequence, respectively. Elements in the spatiotemporal mask matrix are used to indicate the preservation or restoration of motion data at corresponding spatiotemporal positions in the first motion sequence. For example, the spatiotemporal mask matrix can be represented as a two-dimensional matrix M[T×D], where T represents the time dimension (number of motion frames) and D represents the spatial dimension (motion feature dimension). Each element M[t, d] in the mask matrix defines a control strategy in time t and spatial dimension d.

[0038] Step S130: Based on the spatiotemporal mask matrix, the first motion sequence is processed using a diffusion model to obtain the robot's second motion sequence.

[0039] The diffusion model is a probabilistic generative deep learning model inspired by the molecular diffusion process in physics. Specifically, the diffusion model generates high-quality sample data by gradually adding noise to simulated data (forward diffusion process) and then using a deep learning model to progressively remove noise in reverse (backward diffusion process).

[0040] In some embodiments, a control strategy for the first motion sequence in time t and spatial dimension d can be determined based on a spatiotemporal mask matrix. For example, the original data may be kept unchanged or it may need to be repaired and generated through a diffusion model. Based on the first motion sequence and the control strategy, the second motion sequence of the robot can be obtained through the diffusion model.

[0041] For example, firstly, the first motion sequence is concatenated with the spatiotemporal mask matrix M along the element-wise dimension to form the conditional input. Then, Gaussian noise is progressively added to the first motion sequence at a predetermined diffusion number N to obtain a noisy motion sequence. Subsequently, based on the conditional input, a reverse denoising network predicts the noise at each step. After each iteration, the positions marked as preserved in the spatiotemporal mask matrix M are replaced with the corresponding elements of the first motion sequence, while the positions marked as repair retain the model's predicted values, until the denoised second motion sequence is obtained. The spatiotemporal dimension of the second motion sequence corresponds to the first motion region, and without changing the preserved region, it naturally completes the region to be repaired, realizing the generation of conditional motion guided by the mask.

[0042] In this application, by constructing a mask matrix aligned with the spatiotemporal dimension of the motion sequence, it is possible to specify with high precision the joints that need to be strictly preserved (such as foot contact points) and the joints that need to be repaired or filled. This provides a general and highly flexible solution for the accurate reproduction and flexible generation of robot motion sequences, enabling partial posture control of any joint at any time, thereby generating motion that satisfies both precise spatial constraints and natural, smooth movements. Furthermore, by combining spatiotemporal masks with a diffusion model, this method reduces the dependence on large-scale pairwise training data. Only a small number of unlabeled motion sequences are needed to perform adaptive fine-tuning on the target robot, thus significantly shortening the deployment cycle and reducing data costs.

[0043] To further improve the level of fine control over robot motion sequences, this application provides an optional implementation method based on the aforementioned spatiotemporal mask matrix. This method can selectively preserve or repair motion data according to different spatial granularities, thereby meeting multi-level control requirements from macroscopic body posture to microscopic joint angles. The specific implementation method is as follows.

[0044] In some embodiments, the spatiotemporal mask matrix is ​​used to implement at least one of the following spatial control granularities: body region level control for controlling a body region containing multiple joints; joint group level control for controlling a joint group performing a specific function; and single joint level control for controlling the motion parameters of a single joint.

[0045] More specifically, body region-level control can divide the robot skeleton into large body regions such as full-body posture, arms, and legs, and mark all joints belonging to the same body region with the same hold or restore value on the corresponding elements of the spatiotemporal mask matrix. As another example, joint group-level control can divide joints into functional joint groups such as the left arm (7 joints), right leg (6 joints), ankle, and wrist joints based on function or kinematic chain. Mark all joints belonging to the same functional joint group with the same hold or restore value on the corresponding elements of the spatiotemporal mask matrix, allowing constraints to be applied only to functional joint groups performing specific tasks (such as grasping or walking), while the remaining joints are freely completed by the model. Furthermore, single-joint-level control can be refined to the element level in the spatial dimension, assigning independent values ​​to each degree of freedom, achieving pixel-level precise control of the motion parameters of specific joints, such as the left ankle pitch joint and the right ankle roll joint, in any frame.

[0046] In one example, if the robot's left knee joint is detected to require repair between frames 40 and 50, this joint can be repaired at specific points using a mask matrix. Specifically, the values ​​of all entries in the spatiotemporal mask matrix corresponding to the left knee joint within a specific time period (frames 40 to 50) are set to the second value to instruct the diffusion model to repair the joint's motion trajectory within this interval. Simultaneously, the values ​​of all other joint entries in the spatiotemporal mask matrix are set to the first value to ensure that the original motion data of other parts of the robot's body is strictly preserved. In this way, this method can intelligently repair abnormal data in local joints without interfering with the overall motion, ensuring the naturalness and smoothness of the generated motion.

[0047] In another example, data from frames 20 to 40 of a complete motion sequence is completely missing. In this case, the time range to be filled can be specified using a mask matrix. Specifically, the values ​​of all entries in the spatiotemporal mask matrix corresponding to all relevant nodes within a specific time period (frames 20 to 40) are uniformly set to the second value. This instructs the diffusion model to infer and generate coherent transition movements for all joints within this interval based on the starting state of frame 19 and the ending state of frame 41. For all data outside this time period, the corresponding mask values ​​are set to the first value to ensure strict preservation of the known motion sequence. In this way, this method can effectively fill in long periods of missing data, generate reasonable movements that conform to kinematic laws, and significantly improve the ability to process and utilize incomplete motion data.

[0048] In this embodiment, by dividing the spatial control granularity of the spatiotemporal mask matrix into three levels—body region level, joint group level, and single joint level—motion constraints at any level can be specified as needed without changing the diffusion model parameters. This avoids the high cost of retraining the network for specific parts or functions and is compatible with robot platforms with different degrees of freedom configurations, significantly improving the flexibility and accuracy of robot motion sequence generation.

[0049] To prevent conflicts between the hold mode and the repair mode, this application provides an optional implementation based on the above-mentioned spatiotemporal mask matrix. By assigning first and second values ​​with priority order to the matrix elements, it is guaranteed that the hold instruction will be executed first when both hold and repair instructions are received at the same spatiotemporal location. The specific implementation is as follows.

[0050] In some embodiments, elements in the spatiotemporal mask matrix can be assigned a first value and a second value, wherein the first value indicates the motion data at the corresponding spatiotemporal position in the first motion sequence, and the second value indicates the motion data at the corresponding spatiotemporal position in the first motion sequence that is repaired. For example, the first value is 1, the second value is 0, and each element M[t, d]∈{0,1}.

[0051] In some embodiments, a first value can be assigned to the elements in the spatiotemporal mask matrix based on user input, end-joint constraint detection, keyframe annotation files, and foot-foot height threshold, and a second value can be assigned to the remaining positions that need to be repaired or generated by the diffusion model.

[0052] Furthermore, assigning a first value to motion data at the same spatiotemporal location has a higher priority than assigning a second value. For example, firstly, based on user input, keyframe annotation files, contact point detection results, or automated semantic annotation, at least one initial hold set is generated to represent a specific location in the spatiotemporal mask matrix as the first value; then, for each location in the initial hold set that is not marked as the first value, a second value is generated as an initial repair set; for at least one initial repair set, if both the first and second values ​​exist at the same location, then according to the conflict resolution rule that assigns the first value has a higher priority than the second value, the final value at that location is forcibly set to the first value, ensuring that the diffusion model performs hard constraint preservation on that location during subsequent denoising.

[0053] In this embodiment, when repair and retention commands are received simultaneously at the same spatiotemporal location, the retention mode automatically takes priority, ensuring that critical parts or security constraints are never rewritten. This strategy enables fine-grained locking and unlocking switching within a single frame without the need to retrain the model, fundamentally eliminating the risk of control conflicts and significantly improving the security, stability, and engineering deployability of the generation process.

[0054] In order to automatically eliminate motion instability caused by abnormal angles of the end joint after retargeting, this application provides an optional implementation method that can identify and mark the target spatiotemporal position that needs to be regenerated in real time during the mask construction stage, so that the diffusion model can repair the position instead of maintaining it. The specific implementation is as follows.

[0055] In some embodiments, when the first motion sequence includes an end joint whose angle value does not conform to the motion stability rule after motion redirection, the end joint whose angle value does not conform to the motion stability rule and the target spatiotemporal position corresponding to the end joint can be identified; and the elements in the spatiotemporal mask matrix of the target spatiotemporal position are set to the second value.

[0056] Motion redirection is a technique that adapts the motion data of one character to another character with different body proportions.

[0057] Motion stability rules refer to the rules that a character's movement must meet to be considered reasonable. In some embodiments, motion stability rules include ensuring that the rotation angle of joints at the end of the kinetic chain does not exceed a preset physically feasible range. For example, end joints whose angle values ​​do not conform to motion stability rules include ankle joints with unreasonable plantar angles.

[0058] In some embodiments, the redirected Euler angle sequence of joint rotation can first be read, and the contact torque between each end joint and the ground can be calculated frame by frame. Then, the spatiotemporal position where the rate of change of torque is greater than the maximum value of the rate of change of torque of that joint is marked as the target spatiotemporal position.

[0059] The target spatiotemporal position corresponding to the end joint refers to the spatiotemporal position of the end joint in the first motion sequence whose angle value does not conform to the motion stability rule. For example, the target spatiotemporal position can be the (t, d) coordinate point jointly determined by the time frame index t in the entire motion sequence and its specific index in the motion feature dimension d (such as the left ankle roll joint). In some embodiments, the target spatiotemporal positions can be recorded to form a target spatiotemporal position set. Subsequently, when constructing the spatiotemporal mask matrix M, the elements corresponding to the target spatiotemporal position set are forcibly set to a second value to instruct the diffusion model to repair rather than maintain the position in the subsequent denoising process, thereby eliminating abnormal angles caused by redirection and ensuring the motion stability of the second motion sequence.

[0060] In this embodiment, by setting the elements in the spatiotemporal mask matrix of the target spatiotemporal location to a second value, a trained diffusion model can be used to infer the end joints with mask values ​​of the second value, generating natural and reasonable joint angles. By learning from a large amount of accurate human motion data, the diffusion model can infer foot plantar angles that conform to biomechanical principles, thereby correcting unreasonable postures generated during the repositioning process.

[0061] The foregoing steps have sequentially elucidated the complete logical flow of the robot motion sequence generation method in the embodiments of this application, so as to understand the mapping relationship of the process at the actual data level. Next, to intuitively demonstrate the synergistic effect between the spatiotemporal mask matrix and the diffusion model in the above steps, this application also provides... Figure 2 and Figure 3 This is used to illustrate how to map the raw captured data to a local coordinate system based on the robot itself, and then perform subsequent mask construction and diffusion processing to improve the accuracy and robustness of motion redirection and generation.

[0062] Specifically, Figure 2 The diagram shown is an exemplary schematic of a robot motion sequence generation method provided in an embodiment of this application. Figure 3 The diagram shown is a flowchart illustrating the process of determining a first motion sequence according to an embodiment of this application. Figure 3 As shown, the method includes the following steps.

[0063] Step 310: Obtain the third motion sequence in the global coordinate system.

[0064] A global coordinate system represents a fixed reference frame independent of the object being imitated, used to describe the absolute position and orientation of the object. For example, a global coordinate system is a three-dimensional Cartesian coordinate system with a fixed coordinate axis direction, based on a point in space for capturing the object.

[0065] A third motion sequence refers to a set of continuous motion data arranged in chronological order within a global coordinate system. For example, a third motion sequence may include a matrix composed of the pose vectors of the object to be imitated at several discrete moments, arranged in chronological order within the global coordinate system. For more information on the object to be imitated, see [link to documentation]. Figure 1 And related descriptions. In some embodiments, a third motion sequence can be obtained directly from a third-party motion database.

[0066] Step S320: Perform trajectory representation transformation on the third motion sequence in the global coordinate system to obtain the first motion sequence in the local coordinate system.

[0067] Trajectory representation transformation refers to the coordinate transformation of a motion trajectory. For example, trajectory representation transformation includes the operation of mapping the global pose data of each joint to its local pose data relative to its direct parent joint. In this case, the first motion sequence can refer to the motion data generated by coordinate transformation in the local coordinate system. For example, the first motion sequence includes a matrix composed of the pose vectors of the object to be imitated at several discrete moments arranged in time order in the local coordinate system.

[0068] In some embodiments, a trained trajectory transformation model can be used to convert a third motion sequence into a first motion sequence. The trajectory transformation model can be a machine learning model, such as a Deep Neural Network (DNN) model, a Convolutional Neural Network (CNN) model, a Recurrent Neural Network (RNN) model, or any combination thereof. Accordingly, the input to the trajectory transformation model includes the third motion sequence, and the output of the trajectory transformation model includes the first motion sequence.

[0069] In some embodiments, the trajectory conversion model can be trained using multiple labeled training samples. For example, multiple labeled training samples are input into an initial trajectory conversion model, a loss function is constructed using the labels and the results of the initial trajectory conversion model, and the parameters of the initial trajectory conversion model are iteratively updated based on the loss function. The model training is complete when the loss function of the initial trajectory conversion model satisfies a preset condition, resulting in a trained prediction model. The preset condition may be loss function convergence, the number of iterations reaching a threshold, etc.

[0070] In some embodiments, the training samples include a third motion sequence and the label is a first motion sequence. In some embodiments, the training samples and the label can be obtained by inertial motion capture and optical motion capture of the human body.

[0071] In some embodiments, by processing the third motion sequence through an initial trajectory conversion model, the self-learning capability of the machine learning model can be utilized to find patterns from a large number of motion captures, obtain the correlation between the first motion sequence and the third motion sequence, and improve the accuracy and efficiency of determining the first motion sequence.

[0072] To convert the global absolute motion into a local representation based on the robot itself without distortion, this application provides an optional embodiment that can sequentially decompose and decouple the root trajectory, global orientation, and velocity field to generate a first motion sequence that only reflects the differences in the body's kinematics. The specific implementation is as follows.

[0073] Figure 4 The diagram shown is a schematic representation of a trajectory representation conversion process according to an embodiment of this application. Figure 4 As shown, the implementation method includes the following steps.

[0074] Step S410: Decompose the root trajectory in the third motion sequence to obtain the horizontal plane component.

[0075] The root trajectory refers to the position-orientation sequence of the root link (e.g., pelvis or base) of the object being imitated, changing over time in the global coordinate system throughout the entire motion. For example, the root trajectory can be R(t) = [x(t), y(t), z(t), θ(t)], where t = 1…T, (x, y, z) are the coordinates of the root centroid, and θ is the yaw angle about the gravity axis (Z-axis).

[0076] The horizontal plane component refers to the orthogonal projection of the root trajectory onto the horizontal plane (i.e., the XY plane) of the global coordinate system. In some embodiments, the three-dimensional position data of the root link at each moment can be extracted first from the third motion sequence; then the Z-axis displacement component of the root trajectory is removed, and only the motion information in the X and Y axes is retained to obtain the horizontal plane component.

[0077] Step S420: Rotationally decouple the third motion sequence to extract the principal axis rotation component from the global orientation of the third motion sequence and obtain the local global orientation.

[0078] Rotational decoupling refers to a mathematical method that separates the rotational motion of the spindle from other forms of motion.

[0079] Global orientation refers to the three-dimensional orientation of the root link of the object to be imitated in the global coordinate system. For example, global orientation can be in the form of Euler angles ZYX.

[0080] The spindle rotation component refers to the rotation component of the root link about the gravity axis (Z-axis). It usually represents the orientation of the root link in the horizontal plane.

[0081] The local-global orientation represents the rotational component remaining after decoupling the principal axis rotational component from the global orientation. It typically represents body postures such as tilting, leaning forward, or leaning backward.

[0082] In some embodiments, the third motion sequence can be rotationally decoupled to extract the principal axis rotation component from the global orientation of the third motion sequence, and the remaining tilt component can be used as the local global orientation. For example, the Euler angles ZYX of the global orientation of the root link are first read frame by frame. Then, the Euler angles ZYX are decomposed into a Z-axis rotation component and a composite rotation component with the YX plane axis. Subsequently, the rotation component about the gravity axis (Z-axis) is used as the principal axis rotation component, and the remaining rotation component XY-axis rotation component is used as the local global orientation.

[0083] Step S430: Based on the horizontal plane component and the main axis rotation component, obtain the local velocity trajectory and velocity angle.

[0084] Local velocity trajectories are used to describe the speed and direction of the object being imitated in the local coordinate system, and are used to characterize the forward and lateral movement velocities of the object being imitated.

[0085] The velocity angle describes how fast the object being imitated rotates around the gravity axis (Z-axis), quantifying the degree of deviation between the motion direction and the body's orientation. In some embodiments, the horizontal plane component and the principal axis rotation component at multiple discrete moments can be differentially analyzed, and the results can be transformed into a local coordinate system based on the global orientation to obtain the local velocity trajectory and velocity angle. For example, firstly, the horizontal plane component is rotated in coordinate system based on the principal axis rotation component to eliminate the orientation effect. The first-order difference is calculated frame by frame and divided by the sampling interval to obtain the global horizontal velocity. Then, the first-order difference is calculated frame by frame for the principal axis rotation component and divided by the sampling interval to obtain the velocity angle.

[0086] Step S440: The local global orientation, local velocity trajectory, and velocity angle are used as the first motion sequence in the local coordinate system.

[0087] In some embodiments, the first motion sequence in the local coordinate system includes a matrix obtained by stitching together the local and global orientation, local velocity trajectory, and velocity angle. In some embodiments, the first motion sequence in the local coordinate system may further include a matrix obtained by stitching together the Z-axis displacement component and at least one joint degree of freedom (DOF) frame by frame.

[0088] This application also provides embodiments that... Figure 5 For comparison and understanding Figure 4 The data transformation relationships and key technical details of each step. Specifically, Figure 5 This is an exemplary schematic diagram illustrating trajectory representation conversion provided in an embodiment of this application. Figure 5 This demonstrates the complete technical process by which the third motion sequence in the global coordinate system gradually eliminates the influence of absolute displacement and orientation through root trajectory decomposition, rotational decoupling, and differential operations, ultimately extracting the local velocity trajectory, velocity angle, and local-global orientation of the robot as the main body, and generating the first motion sequence.

[0089] In this embodiment, the absolute trajectory, which originally relied on world coordinates and was susceptible to global orientation interference, is transformed into a local velocity trajectory and velocity angle based on the robot itself. This allows the diffusion model to learn gait patterns only within a local coordinate system, significantly improving the versatility of motion imitation.

[0090] To avoid abrupt changes at fragment boundaries and ensure sufficient motion constraints for the diffusion model at the start and end frames, this application provides an optional embodiment in which a preset buffer segment with a standing posture is first spliced ​​into the fourth motion sequence in the global coordinate system, and then the whole sequence is input into the subsequent transformation process as the third motion sequence to improve motion coherence and generation stability. The specific implementation is as follows.

[0091] Figure 6 The diagram shown is a schematic flowchart of obtaining a third motion sequence according to an embodiment of this application. Figure 6As shown, the implementation method includes the following steps: Step S610: Obtain the fourth motion sequence in the global coordinate system.

[0092] For more information on the global coordinate system, see [link to documentation]. Figure 3 And its related descriptions.

[0093] The fourth motion sequence refers to the original motion dataset that requires motion transition. For example, the fourth motion sequence can be a video demonstrating human movements. In some embodiments, the acquisition of the fourth motion sequence is similar to that of the first or third motion sequence, and will not be elaborated here. In some embodiments, a large amount of data can be acquired through a third-party platform, and the fourth motion sequence can be obtained after statistical analysis and other processing.

[0094] Step S620: Construct a preset pose sequence.

[0095] Preset pose sequences are motion sequences used to provide transition buffers at the start, end, or keyframe switching stages of motion. Their purpose is to smoothly connect the main motion sequence with static states or different action segments, reducing the generation error of the diffusion model at boundary frames.

[0096] In some embodiments, the preset posture sequence may include a standing posture. In some embodiments, frames of the object to be imitated in the fourth motion sequence in the preset posture can be retrieved, and the preset posture sequence is obtained by copying N frames. For example, the retrieval criteria include a pelvic height threshold, a joint angular velocity less than a preset velocity threshold, or a joint angle that is a preset joint angle. In some embodiments, if there are no frames in the preset posture that meet the retrieval criteria, the skeleton of the object to be imitated can be obtained based on the fourth motion sequence, and frames of the object to be imitated in the preset posture can be generated by pelvic alignment, root link rotation alignment, and inverse kinematics solution. The preset posture sequence is obtained by copying N frames. Here, N is a positive integer and can be dynamically set according to the diffusion model input length or transition duration requirements.

[0097] Optionally, pelvic alignment refers to mapping the position and orientation of the pelvic coordinate system in a preset pose to the root coordinate system of the object to be imitated in one go. Inverse kinematics retargeting refers to the process of reversely calculating the angles of each joint or the length of each link of the robot when the pose of the end effector is known, so as to adapt the captured motion of the object to be imitated to robot models of different scales.

[0098] Step S630: Add the preset posture sequence to the start and end of the fourth motion sequence respectively to obtain the third motion sequence in the global coordinate system.

[0099] In some embodiments, preset pose sequences can be directly added to the start and end points of the fourth motion sequence to obtain a third motion sequence in the global coordinate system. The start and end points correspond to the time positions before the first frame and after the last frame of the fourth motion sequence, respectively.

[0100] In some embodiments, a temporal continuity check is further performed on the spliced ​​sequence. If the difference in root node position or joint rotation between adjacent frames exceeds the corresponding difference threshold, a smooth transition (e.g., linear interpolation or spherical linear interpolation) is set for the preset pose region connecting the start and end points to the fourth motion sequence, while maintaining the pose constraints at the boundaries of the added motion sequence to ensure the naturalness of the motion transition. For more information on the third motion sequence, see [link to documentation]. Figure 3 And its related descriptions.

[0101] In some embodiments, when the third motion sequence is obtained by splicing a preset posture sequence and a fourth motion sequence, constructing the spatiotemporal mask matrix includes: setting the elements of the spatiotemporal positions corresponding to the preset posture sequence to a second value, and setting the elements of the spatiotemporal positions corresponding to the splicing points of the fourth motion sequence and the preset posture sequence to a first value.

[0102] In some embodiments, when the third motion sequence is spliced ​​from a preset posture sequence and a fourth motion sequence, the diffusion model performs overall repair on this segment by uniformly setting all spatiotemporal positions corresponding to the preset posture sequence to the second value in the mask matrix, thereby avoiding the introduction of a stiff transition due to the incorrect maintenance of the standing buffer posture. Simultaneously, the mask elements of adjacent frames at the splicing point are set to the first value to forcibly maintain key boundary data, ensuring the continuity of the main motion segment and the buffer segment in joint space and root trajectory. This setting significantly reduces velocity jumps and foot slippage at the splicing point in the generated results, allowing the robot's output second motion sequence to exhibit a natural deceleration and stable landing visual effect at both the start and end stages, and reducing the workload of subsequent manual adjustments.

[0103] For further explanation regarding the spatiotemporal mask matrix, please refer to [link / reference]. Figure 1 And its related descriptions.

[0104] In this embodiment, by splicing a preset attitude sequence at the beginning and end of the fourth motion sequence, the third motion sequence is made to form a zero-velocity, zero-acceleration boundary condition in the time domain. This boundary serves as a repair region in the subsequent diffusion model input, which can eliminate boundary artifacts and significantly improve the smoothness of the generated motion and deployment efficiency.

[0105] To achieve high-quality generation of long motion sequences even with limited single inference length, this application provides an optional embodiment that first segments the extremely long first motion sequence into overlapping segments, then generates it segment by segment through conditional diffusion, and finally eliminates boundary seams through weighted fusion, thereby balancing the preservation of local details with global coherence. The specific implementation is as follows.

[0106] Figure 7 The diagram shown is a schematic flowchart illustrating the processing of a first motion sequence using a diffusion model according to an embodiment of this application. Figure 7 As shown, the implementation method includes the following steps: Step S710: If the length of the first motion sequence exceeds the target length threshold, the first motion sequence is divided into multiple first sequence segments with overlapping regions.

[0107] Optionally, the target length threshold refers to the maximum number of frames that the diffusion model can accept in a single calculation. In some embodiments, the target length threshold can be determined based on historical experience to avoid memory overflow and ensure generation stability.

[0108] The first sequence segment refers to a motion sequence segment generated piecewise using an independent input diffusion model. In some embodiments, a segment length and an overlap length can be set, and the first sequence segment is truncated along the time axis using a sliding window based on the segment length and overlap length. For example, first, the segment length L and the overlap length O are set, where L is less than a target length threshold and O is less than L. Then, a step size is used... A sliding window is used along the time axis, where segment i is the cutoff time. The first sequence consists of consecutive frames, with adjacent segments sharing 0 frames at the end / beginning, forming an overlapping area. If the last segment is less than L frames, it is padded to L frames (by copying the last frame or zero-speed padding) to ensure that all first sequence segments are of equal length. Furthermore, each first sequence segment is assigned a segment index and its starting frame number in the original sequence is recorded, forming at least one first sequence segment with overlap.

[0109] Step S720: Based on the spatiotemporal mask matrix corresponding to each first sequence segment, the first sequence segment is processed using a diffusion model to obtain the second sequence segment.

[0110] For more information on spatiotemporal mask matrices, see [link to documentation]. Figure 1 And its related descriptions.

[0111] In some embodiments, a portion corresponding to a spatiotemporal position can be extracted based on the spatiotemporal mask matrix corresponding to the first motion sequence, and used as the spatiotemporal mask matrix corresponding to the first sequence segment.

[0112] The second sequence segment refers to the equal-length output sequence generated by the model after processing the first sequence segment based on a mask. The diffusion model processes the first sequence segment in a similar way to it processes the first motion sequence; see [link to details]. Figure 1 And its related descriptions.

[0113] Step S730: Weighted fusion of adjacent second sequence segments in the overlapping region to obtain the robot's second motion sequence.

[0114] The robot's second motion sequence refers to the final output motion sequence of the same length as the first motion sequence. In some embodiments, adjacent second sequence segments can be weighted and fused in the overlapping region. For example, in the overlapping region, the weight of the preceding segment decreases linearly from 1 to 0, and the weight of the following segment increases linearly from 0 to 1, thus obtaining the robot's second motion sequence. That is, for each frame in the overlapping region, a fusion weight is assigned in chronological order, monotonically increasing from 0 to 1, with the weight of the preceding segment being (1... w), the weight of the subsequent segment is w, where w=t' / ΔT, and t' is the relative sequence number of the frame in the overlapping area.

[0115] In this embodiment, when the length of the first motion sequence exceeds the target length threshold, the sequence is first divided into several equal-length first sequence segments according to a fixed step size. Each segment uses the corresponding slice of the original spatiotemporal mask matrix. After processing by the diffusion model, the second sequence segment is output. Then, weighted fusion is performed in the overlapping area, and finally, a complete and temporally continuous second motion sequence of the robot is spliced ​​together. This eliminates boundary discontinuities while satisfying the model length constraint, ensuring the smoothness and consistency of long-term motion generation.

[0116] The following examples, through embodiments one to three, exemplify the complete technical process by which this application converts global motion data into local features and combines them with spatiotemporal masks and diffusion models to generate a natural, controllable, and robot-adaptive second motion sequence.

[0117] Example 1: Motion data of the object to be imitated is acquired by an optical motion capture system or an inertial measurement unit array, and the root translation offset (root_trans_offset), global orientation (pose_aa), and joint degrees of freedom (dof) are obtained in chronological order based on the global coordinate system, which serve as the raw input for subsequent processing.

[0118] Then, a kinematic decoupling algorithm is used to decompose the root trajectory into a horizontal plane, decompose the global orientation into ZXY Euler angles and extract the principal axis rotation component. Then, the displacement Z-axis component, local velocity trajectory, velocity angle, local global orientation and joint degrees of freedom are obtained by frame-by-frame difference and coordinate system rotation. Finally, they are stitched together into a standardized local feature matrix with dimension T×D to complete the transformation from global to local coordinate system.

[0119] Next, a mask matrix M[T×D] corresponding one-to-one with the local feature dimensions is instantiated, where M[t, d]∈{0,1}, and three mask writing modes are supported: body region level, joint group level, and single joint level. For example, a mask value of 0 indicates repair, and 1 indicates preservation.

[0120] Furthermore, a dual-mode control strategy is applied to the mask matrix, where the hold mode task has higher priority than the repair mode task. That is, when both hold and repair instructions are received at the same (t, d) position, a priority decision is made, and a 1 is forcibly written to ensure that keyframes or end contact points are not modified by the diffusion model.

[0121] If the length T of the acquired motion sequence of the target object is greater than a preset length threshold Lmax, a sliding window is used to divide T into K segments, each segment being shorter than Lmax. Adjacent segments overlap by ΔT frames, and a global time offset is recorded to ensure that the mask control information remains consistent before and after segmentation. Then, each segment and its corresponding mask matrix are concatenated and input into a pre-trained Denoising Diffusion Probabilistic Model (DDPM). During the N-step reverse denoising process, positions with a mask value of 1 are forcibly restored to the original segment data, while positions with a mask value of 0 retain the model prediction value, until the second segment is generated. After being converted into robot motion data, weighted fusion is performed on the overlapping areas of adjacent second segments. The weight of the previous segment decreases linearly from 1 to 0 within ΔT frames, while the weight of the current segment increases linearly from 0 to 1. Spherical linear interpolation is used for joint rotations, and linear interpolation is used for root node positions to eliminate discontinuities at segment boundaries.

[0122] Finally, the local motion features of the fused robot are summed by accumulating the velocity angles to reconstruct the Z-axis rotation angle; the local velocity is converted back to the global coordinate system using a rotation matrix, and the trajectory is reconstructed by integration; the complete global orientation is reconstructed by combining the local and global orientations and the Z-axis rotation obtained by integration, and then the joint degrees of freedom are superimposed to obtain global coordinate system motion data consistent with the original capture data format, which is used as the robot's second motion sequence output for driving or display.

[0123] Example 2: Building upon Example 1, this example adds a motion transition generation step to eliminate jumps between the start and end phases. Specifically, standard standing posture parameters are created; these posture parameters are copied N frames to form a default posture sequence, which is then marked as a transition buffer. Then, the default posture sequence is placed before and after the input second motion sequence in time, constructing a "default posture - second motion sequence - default posture" splicing structure, expanding the total number of frames to T+2N. Furthermore, it is understood that the frame rate remains constant during the splicing process to ensure timeline continuity.

[0124] Finally, a smooth task mask is generated for the default pose regions of the front-end and back-end. Specifically, the elements of the spatiotemporal mask matrix corresponding to this region are initialized to the second value to instruct the diffusion model to perform repair. Simultaneously, at least one frame element at the boundary between the default pose sequence and the input sequence is forcibly set to the first value to maintain boundary pose constraints. In subsequent diffusion denoising, the boundary-preserving frames are passed as hard constraints to adjacent repair frames, achieving a smooth transition of velocity and acceleration, thereby outputting a second motion sequence with natural start and stop.

[0125] Example 3: Building upon Embodiment 1, this embodiment further provides a method for repairing end-joint angles in motion retargeting to address the problem of abnormal plantar angles caused by differences in skeletal proportions. Specifically, a quality assessment is performed on the motion sequence after motion retargeting, including reading the global orientation and joint degrees of freedom data for each frame after retargeting; calculating the pitch and roll angles of the plantar plane relative to the ground in the ankle joint; if the angle exceeds a preset biomechanically reasonable range and the duration exceeds a threshold number of frames, then that time period is marked as an unreasonable angle range.

[0126] Then, in the spatiotemporal mask matrix M[T×D] corresponding to the action sequence, the mask value of the ankle joint with unreasonable foot plantar angle is set to 0 in the corresponding time period to indicate that the diffusion model will perform repair generation on this part in the future, while the mask values ​​of the other channels remain unchanged.

[0127] Next, the local feature matrix with the problematic joint mask is input into the trained diffusion model. During the reverse denoising process, the model predicts the ankle joint channel with a mask value of 0 frame by frame and outputs the repaired rotation amount to replace the original abnormal angle.

[0128] Finally, the plantar angle and foot height off the ground are recalculated for the repaired motion sequence. If the ankle pitch and roll angles are within a reasonable range and the foot does not penetrate the ground or hang abnormally, the repair is deemed qualified; otherwise, the mask range is adjusted and the repair is re-executed until motion artifacts such as sliding, penetrating the ground, or hanging are eliminated, and finally, a second motion sequence that conforms to physical constraints is output.

[0129] The above text combined Figures 1 to 7 The present application describes in detail embodiments of the robot motion sequence generation method, which are illustrated below in conjunction with... Figure 8 This application provides a detailed description of embodiments of the robot motion sequence generation apparatus. It should be understood that the descriptions of the robot motion sequence generation method embodiments correspond to the descriptions of the robot motion sequence generation apparatus embodiments; therefore, any parts not described in detail can be found in the preceding method embodiments.

[0130] Figure 8 The diagram shown is a structural schematic of a robot motion sequence generation device provided in an embodiment of this application. Figure 8 As shown, the robot motion sequence generation device 80 provided in this application embodiment includes: The determination module 810 is used to determine the first motion sequence.

[0131] The matrix construction module 820 is used to construct the spatiotemporal mask matrix corresponding to the first motion sequence. The dimensions of the spatiotemporal mask matrix correspond to the time dimension and spatial dimension of the first motion sequence, respectively. The elements in the spatiotemporal mask matrix are used to indicate the motion data of the corresponding spatiotemporal position in the first motion sequence to maintain or repair.

[0132] The processing module 830 is used to process the first motion sequence based on the spatiotemporal mask matrix and the diffusion model to obtain the robot's second motion sequence.

[0133] In one embodiment of this application, the determining module 810 is further configured to: obtain a third motion sequence in a global coordinate system, wherein the global coordinate system represents a fixed reference system independent of the robot body and is used to describe the absolute position and orientation of the robot; and perform trajectory representation transformation on the third motion sequence in the global coordinate system to obtain a first motion sequence in a local coordinate system, wherein the local coordinate system represents a reference system based on the robot and is used to describe the motion of each part of the robot relative to the robot body.

[0134] In one embodiment of this application, the determining module 810 is further configured to: decompose the root trajectory in the third motion sequence to obtain the horizontal plane component; perform rotational decoupling on the third motion sequence, extract the principal axis rotation component from the global orientation of the third motion sequence, and obtain the local global orientation; obtain the local velocity trajectory and velocity angle based on the horizontal plane component and the principal axis rotation component; and use the local global orientation, local velocity trajectory, and velocity angle as the first motion sequence in the local coordinate system.

[0135] In one embodiment of this application, the determining module 810 is further configured to: obtain a fourth motion sequence in the global coordinate system; construct a preset posture sequence, wherein the preset posture sequence includes a standing posture; and add the preset posture sequence to the start and end of the fourth motion sequence respectively to obtain a third motion sequence in the global coordinate system.

[0136] In one embodiment of this application, the spatiotemporal mask matrix is ​​used to implement at least one of the following spatial control granularities: body region level control for controlling a body region containing multiple joints; joint group level control for controlling a joint group performing a specific function; and single joint level control for controlling the motion parameters of a single joint.

[0137] In one embodiment of this application, the matrix construction module 820 is further configured to assign a first value and a second value to the elements in the spatiotemporal mask matrix, wherein the first value is used to indicate the motion data at the corresponding spatiotemporal position in the first motion sequence, and the second value is used to indicate the repair of the motion data at the corresponding spatiotemporal position in the first motion sequence; wherein, the priority of assigning the first value to the motion data at the same spatiotemporal position is higher than the priority of assigning the second value.

[0138] In one embodiment of this application, the matrix construction module 820 is further configured to, in the case where the first motion sequence includes an end joint whose angle value does not conform to the motion stability rule after motion redirection, identify the end joint whose angle value does not conform to the motion stability rule and the target spatiotemporal position corresponding to the end joint; and set the elements in the spatiotemporal mask matrix of the target spatiotemporal position to a second value.

[0139] In one embodiment of this application, the processing module 830 is further configured to, when the length of the first motion sequence exceeds a target length threshold, divide the first motion sequence into multiple first sequence segments with overlapping regions; process the first sequence segments using a diffusion model based on the spatiotemporal mask matrix corresponding to each first sequence segment to obtain a second sequence segment; and perform weighted fusion of adjacent second sequence segments in the overlapping region to obtain the robot's second motion sequence.

[0140] Below, for reference Figure 9 This describes an electronic device according to embodiments of the present application. Figure 6 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application.

[0141] like Figure 9 As shown, the electronic device 90 includes one or more processors 901 and memory 902.

[0142] The processor 901 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0143] The memory 902 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 901 may execute the program instructions to implement the robot motion sequence generation methods of the various embodiments of this application described above, and / or other desired functions. Various contents, such as a first motion sequence, a spatiotemporal mask matrix, and a second motion sequence, may also be stored in the computer-readable storage medium.

[0144] In one example, the electronic device 90 may also include an input device 903 and an output device 904, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0145] The input device 903 may include, for example, a keyboard, a mouse, etc.

[0146] The output device 904 can output various information to the outside, including a first motion sequence, a spatiotemporal mask matrix, and a second motion sequence. The output device 904 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0147] Of course, for the sake of simplicity, Figure 9 Only some of the components of the electronic device 90 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 90 may include any other suitable components depending on the specific application.

[0148] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the robot motion sequence generation methods according to various embodiments of this application described above.

[0149] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0150] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the robot motion sequence generation method according to various embodiments of this application described above.

[0151] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0152] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0153] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0154] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0155] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0156] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A robot motion sequence generation method characterized by, The method comprises the following steps: determining a first motion sequence; constructing a space-time mask matrix corresponding to the first motion sequence, the space-time mask matrix having dimensions corresponding to the time dimension and the space dimension of the first motion sequence, and elements in the space-time mask matrix being used to indicate whether to keep or repair motion data at corresponding space-time positions in the first motion sequence; processing the first motion sequence based on the space-time mask matrix using a diffusion model to obtain a second motion sequence of the robot.

2. The robot motion sequence generation method according to claim 1, characterized by, The determination of the first motion sequence comprises: obtaining a third motion sequence in a global coordinate system, the global coordinate system representing a fixed reference frame independent of the robot body and being used to describe the absolute position and orientation of the robot; performing trajectory representation conversion on the third motion sequence in the global coordinate system to obtain a first motion sequence in a local coordinate system, the local coordinate system representing a reference frame with the robot as a reference and being used to describe the motion of each part of the robot relative to the robot body.

3. The robot motion sequence generation method according to claim 2, characterized by, The trajectory representation conversion on the third motion sequence in the global coordinate system to obtain the first motion sequence in the local coordinate system comprises: decomposing a root trajectory in the third motion sequence to obtain a horizontal plane component; performing rotational decoupling on the third motion sequence to extract a main axis rotation component from the global orientation of the third motion sequence and obtain a local global orientation; obtaining a local velocity trajectory and a velocity angle based on the horizontal plane component and the main axis rotation component; taking the local global orientation, the local velocity trajectory and the velocity angle as the first motion sequence in the local coordinate system.

4. The robot motion sequence generation method according to claim 2, characterized by, The obtaining of the third motion sequence in the global coordinate system comprises: obtaining a fourth motion sequence in a global coordinate system; constructing a preset posture sequence, wherein the preset posture sequence comprises a standing posture; adding the preset posture sequence to the starting end and the ending end of the fourth motion sequence respectively to obtain the third motion sequence in the global coordinate system.

5. The robot motion sequence generation method according to any one of claims 1 to 4, characterized by, The space-time mask matrix is used to implement at least one of the following spatial control granularities: body region level control, used to control a body region comprising multiple joints; joint group level control, used to control a joint group performing a specific function; single joint level control, used to control the motion parameters of a single joint.

6. The robot motion sequence generation method according to any one of claims 1 to 4, characterized by, The construction of the space-time mask matrix corresponding to the first motion sequence comprises: allocating a first value and a second value to elements in the space-time mask matrix, wherein the first value is used to indicate keeping the motion data at corresponding space-time positions in the first motion sequence, and the second value is used to indicate repairing the motion data at corresponding space-time positions in the first motion sequence; wherein the priority of allocating the first value to the motion data at the same space-time position is higher than the priority of allocating the second value.

7. The robot motion sequence generation method according to any one of claims 1 to 4, characterized by, The construction of the space-time mask matrix corresponding to the first motion sequence comprises: in the case that the first motion sequence contains motion reorientation and an end joint has an angle value that does not comply with the motion stability rule, identifying the end joint with the angle value that does not comply with the motion stability rule and a target space-time position corresponding to the end joint; Set an element in the spatiotemporal mask matrix of the target spatiotemporal position to a second numerical value.

8. The robot motion sequence generation method according to any one of claims 1 to 4, characterized by, The first motion sequence is processed by using a diffusion model based on the spatiotemporal mask matrix to obtain a second motion sequence of the robot, including: In the case that the length of the first motion sequence exceeds a target length threshold, the first motion sequence is segmented into a plurality of first sequence segments with overlapping regions; Each first sequence segment is processed by using the diffusion model based on the corresponding spatiotemporal mask matrix to obtain a second sequence segment; Adjacent second sequence segments are fused in the overlapping regions to obtain the second motion sequence of the robot.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the robot motion sequence generation method in any one of claims 1 to 8.

10. An electronic device, comprising: Comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to execute the robot motion sequence generation method in any one of claims 1 to 8.

11. A computer program product, characterised in that, The computer program product comprises instructions for causing an electronic device to implement the robot motion sequence generation method in any one of claims 1 to 8 when the instructions are executed on the electronic device.

Images