Robot attitude generation method and device, electronic equipment and storage medium

Abstract

The invention discloses a robot attitude generation method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a first attitude vector; performing attitude prediction on the first attitude vector to obtain a second attitude vector; carrying out constraint and smooth adjustment on the second attitude vector to obtain a third attitude vector; and adjusting the posture of the robot according to the third posture vector. According to the method, attitude vectors are generated according to feature information of different dimensions, the attitude vectors are predicted, a prediction result is constrained and smoothed to obtain a third attitude vector, and attitude adjustment is performed on the robot according to the third attitude vector. According to the method, the obtained posture of the robot can better fit the change posture of the object needing to be represented, and meanwhile, the stability and interactivity of the robot can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, and in particular to a robot posture generation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the expansion of the application scenarios of social robots, it is necessary for robots to be able to interact naturally with contact objects, and the robots need to be able to accurately learn co-speech gestures. As an important part of human natural expression, co-speech gestures play a key role in enhancing semantic understanding, emotional expression and social interaction. Therefore, in order to improve the implementation capability of robots for co-speech gestures, the robots need to be able to automatically generate natural gestures driven by speech according to the current posture state.

[0003] Further, the prior art can be divided into three categories: rule-based methods, early statistical methods, and deep model / diffusion model methods. Rule-based gesture generation techniques mostly use a matching method of predefined vocabulary and gesture templates. This type of method is simple to implement, but completely lacks cultural adaptability; the gestures are single, rigid, and unnatural, and are difficult to cover complex sentences, and cannot be adjusted according to the scene and context. Although deep learning models can achieve posture prediction of the upper body 2D key points of robots, they lack finger and spatial details, and are prone to jitter and instability problems. While diffusion models can generate high-quality motion gestures, the generated results are unstable and lack motion constraints, which can cause self-collision problems of robots during operation. SUMMARY

[0004] The present application provides a robot posture generation method, device, electronic equipment and storage medium to solve the problem that the generated results of robot posture do not meet the motion requirements.

[0005] According to an aspect of the present application, a robot posture generation method is provided, comprising:

[0006] determining a first posture vector; the first posture vector is a spliced vector of different dimension feature vectors, and is used to describe the change state of the robot from different dimensions;

[0007] performing posture prediction on the first posture vector to obtain a second posture vector; the second posture vector is used to represent the motion change of the robot at a preset number of frames after the first posture; the first posture is a pre-set number of posture changes of the robot at a pre-set time;

[0008] performing constraint and smoothing adjustment on the second posture vector to obtain a third posture vector;

[0009] performing posture adjustment on the robot according to the third posture vector.

[0010] According to another aspect of the present application, there is provided a robot pose generation apparatus comprising:

[0011] a first pose vector determination module configured to determine a first pose vector, the first pose vector being a concatenation vector of different dimension feature vectors, and configured to describe a change state of the robot from different dimensions;

[0012] a second pose vector determination module configured to perform pose prediction on the first pose vector to obtain a second pose vector, the second pose vector being configured to represent a motion change of the robot at a preset number of frames after the first pose, the first pose being a pre-set number of pose changes of the robot at a preset time;

[0013] a third pose vector determination module configured to perform constraint and smoothing adjustment on the second pose vector to obtain a third pose vector;

[0014] an adjustment module configured to perform pose adjustment on the robot according to the third pose vector.

[0015] According to another aspect of the present application, there is provided an electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected with the at least one processor; wherein,

[0018] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the robot pose generation method according to any one of the embodiments of the present application.

[0019] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to perform the robot pose generation method according to any one of the embodiments of the present application when executed by the processor.

[0020] The technical scheme of the embodiment of the present application determines a first attitude vector; the first attitude vector is a splicing vector of different dimension feature vectors, used to describe the changing state of the robot from different dimensions, and the determination of the first attitude vector can fuse feature information of different dimensions to provide reference for the attitude prediction of the robot from different angles, so that the obtained attitude is more suitable for the object to be represented; attitude prediction is performed on the first attitude vector to obtain a second attitude vector; constraint and smoothing adjustment are performed on the second attitude vector to obtain a third attitude vector, and the smoothing and constraint adjustment can make the obtained third attitude vector more suitable for the action rule of the robot while improving the smoothness of the robot action; and attitude adjustment is performed on the robot according to the third attitude vector. The method generates an attitude vector according to feature information of different dimensions, performs prediction on the attitude vector, performs constraint and smoothing processing on the prediction result to obtain a third attitude vector, and performs attitude adjustment on the robot according to the third attitude vector, so that the obtained attitude of the robot can be more suitable for the changing attitude of the object to be represented while the stability and interactivity of the robot can be improved.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 A flowchart of a robot attitude generation method provided by the embodiment of the present application is shown in the figure.

[0024] Figure 2 A mapping schematic diagram of a cultural label provided by the embodiment of the present application is shown in the figure.

[0025] Figure 3 A structural schematic diagram of a robot attitude generation device provided by the embodiment of the present application is shown in the figure.

[0026] Figure 4 A structural schematic diagram of an electronic device for implementing the robot attitude generation method of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Figure 1 This is a flowchart illustrating a robot posture generation method provided in an embodiment of the present invention. This embodiment is applicable to situations where changing postures are generated in real time based on the robot's movements. The method can be executed by a robot posture generation device, which can be implemented in hardware and / or software. This robot posture generation device can be configured in any electronic device with network communication capabilities. Figure 1 As shown, the method includes:

[0030] S110. Determine the first attitude vector; the first attitude vector is a concatenated vector of feature vectors of different dimensions, used to describe the robot's changing state from different dimensions.

[0031] Specifically, the process involves acquiring feature data that influences the robot's movements, matching the corresponding transformation model according to the type of feature data, and performing the transformation to obtain feature vectors of different dimensions. These feature vectors of different dimensions are then concatenated to obtain the first pose vector.

[0032] The feature data includes: text data, audio data, cultural tags, and initial posture.

[0033] Among them, text data includes pre-input text data that can guide the robot's actions, text data identified from images taken from the outside world, or text data converted from instructions issued by other objects, such as text data converted from the "take a picture" instruction issued by an interactive object.

[0034] The audio data can be either voice data emitted by the robot or voice data provided by an external source. For example, the voice data emitted by the robot could be the sounds it makes when performing actions requiring voice input, such as "hello" or "goodbye." The voice data provided by an external source could be audio that guides the robot's movements, such as the rhythm of music when the robot is dancing.

[0035] In the above steps, the role of audio data is to determine the posture rhythm, so that the generated action is synchronized with the rhythm of the audio.

[0036] Cultural tags are used to characterize the motion features of different types of robots. Cultural tags can include at least: primary cultural features, secondary cultural features, and tertiary cultural features.

[0037] Among them, the first cultural characteristic is used to describe collective actions, such as... Figure 2 As shown, the cultural label corresponding to the second cultural characteristic is "JA / JP", and the corresponding robot's movement characteristics are: small gesture amplitude and steady rhythm; strong gesture convergence and delicate finger movements.

[0038] Among them, the second cultural characteristic is the representation of open-ended actions, such as... Figure 2 As shown, the cultural label corresponding to the second cultural characteristic is "IT", and the corresponding robot's movement characteristics are: large gesture range and active rhythm; strong gesture openness and exaggerated finger movements.

[0039] Among them, the third cultural characteristic represents structural actions.

[0040] The role of cultural tags in the above steps is to control differences in posture and style, making the robot suitable for cross-cultural social scenarios.

[0041] The initial pose is the initial pose seed from the previous M frames at the current time. The pose is represented by a three-dimensional unit direction vector of J joints. For example, J=43, which includes 13 upper body joints and 30 finger joints. Each pose can be represented as a (J-1)×3-dimensional vector.

[0042] The purpose of the initial pose in the above steps is to ensure that the generated pose is continuous with the current pose and to avoid abrupt changes in action.

[0043] The conversion models include: text data conversion model, audio data conversion model, and tag mapping model.

[0044] The text data transformation model processes text data, converting the acquired text data to be synthesized into a sequence of word embeddings. Each word embedding has a 300-dimensional dimension. The text data transformation model can employ a pre-trained word vector model, such as FastText or GloVe.

[0045] The audio data transformation model is used to process audio data, that is, to extract frame-level audio features from the original audio waveform or spectrogram, and output a 32-dimensional audio feature vector sequence. The audio data transformation model consists of a three-layer convolutional layer, each followed by a ReLU activation function and a one-dimensional adaptive average pooling layer.

[0046] The label mapping model handles cultural labels, mapping them to fixed-length cultural feature vectors. These vectors represent style parameters such as amplitude, openness, and rhythm of the encoded posture. The label mapping model can employ a cultural embedding matrix, which is structured around cultural dimensions, robot action scenarios, and cultural embedding levels.

[0047] Among them, the cultural dimension is the cultural core that the robot's actions must adapt to, which determines the acceptability of the robot's actions and the degree of resonance with the interactive objects; the robot action scenario is the specific implementation scenario of cultural embedding, and different scenarios have different requirements for the cultural adaptation of actions; the cultural embedding level determines the degree of integration between robot actions and culture, and is the core benchmark for optimizing robot action design.

[0048] Furthermore, a multimodal fusion module is used to concatenate feature vectors of different dimensions. The hidden layer dimension of the multimodal fusion module is set to 512, containing 8 encoder layers, each containing a multi-head self-attention mechanism and a feedforward neural network. The encoder layers use a Transformer encoder.

[0049] S120. Perform attitude prediction on the first attitude vector to obtain the second attitude vector; the second attitude vector is used to characterize the robot's action changes after a preset number of frames following the first attitude; the first attitude is the robot's attitude changes before a preset number of frames at a preset time.

[0050] Specifically, the number of prediction frames is determined based on the prediction requirements. A vector is generated based on the number of prediction frames to record the predicted robot posture changes, resulting in the fourth posture vector. The third and fourth posture vectors are concatenated to obtain the fifth posture vector. Noise is added to the fifth posture data according to a preset number of additions and a preset time step, resulting in the first noise data. An addition number is randomly selected from the preset addition numbers as the first step length. Based on the first step length, noise data corresponding to the addition number is obtained from the first noise data, resulting in the second noise data. Starting from the second noise data, denoising is performed in reverse order until the addition number reaches 0. The data corresponding to the frame number of the fourth posture vector in the final data is used as the second posture vector.

[0051] S130. Constrain and smooth the second attitude vector to obtain the third attitude vector.

[0052] Specifically, the second attitude vector is smoothed using a filtering algorithm to smooth the motion trajectory, resulting in the sixth attitude vector. Angular velocity and angular acceleration are calculated based on the sixth attitude vector to obtain the first angle data. The first angle data is constrained and verified according to the robot's dynamics, yielding the first result. The sixth attitude vector is adjusted based on the first result to obtain the seventh attitude vector. The attitude vectors where the robot's joint angles abruptly change are extracted from the seventh attitude vector as the first attitude information. The timestamps for matching the first attitude information are determined, and motion beats are generated based on the obtained timestamps, resulting in the first beat sequence. The first beat sequence and the second beat sequence are verified beat by beat, and the first beat sequence is updated based on the verification results. Once all sequences have been compared, the attitude in the seventh attitude vector is adjusted based on the updated first beat sequence to obtain the third attitude vector.

[0053] S140. Adjust the robot's posture according to the third posture vector.

[0054] Specifically, the third pose vector is converted into joint angle changes through inverse dynamics, yielding the third angle data. This third angle data is then mapped to the second angle data along the joints and compared. Based on the comparison results, the third angle data is corrected to obtain the fourth angle data. The acquired fourth angle data is sent to the robot's underlying controller to drive the motors, thereby adjusting the robot's posture.

[0055] Inverse dynamics involves solving for joint angles based on the known motion trajectory of the robot's end effector.

[0056] The comparison dimensions are the changes in angular range and angular velocity.

[0057] Furthermore, the specific steps for acquiring the third-angle data are as follows: The third attitude vector is used to calculate the end effector's motion trajectory using a positive dynamics model. The ideal end effector motion trajectory x is defined. d The ideal end effector motion trajectory is solved based on the end effector motion trajectory to obtain the joint angle change and obtain the third angle data.

[0058] Optionally, attitude prediction is performed on the first attitude vector to obtain the second attitude vector, including steps A1-A5:

[0059] Step A1: Determine the fourth attitude vector; the fourth attitude vector is used to characterize the robot's attitude change within a preset number of frames after the second attitude.

[0060] Specifically, the number of prediction frames is determined based on the prediction requirements, and a vector is generated based on the number of prediction frames to record the changes in the robot's posture, thus obtaining the fourth posture vector.

[0061] Step A2: Concatenate the third attitude vector with the fourth attitude vector to obtain the fifth attitude vector.

[0062] Specifically, the third attitude vector and the fourth attitude vector are concatenated in the order of the third attitude vector first and the fourth attitude vector last to obtain the fifth attitude vector.

[0063] Step A3: Add noise to the fifth attitude vector according to a preset time step. When the number of additions equals the preset number of additions, the first noise data is obtained.

[0064] The preset time step is the time interval for adding noise to the fifth attitude vector.

[0065] Specifically, a second noise is added to the fifth attitude vector according to a preset number of additions and a preset time step. That is, a second noise is added to the fifth attitude vector once every preset time step. When the number of additions equals the preset number of additions, the addition stops and the first noise data is output.

[0066] The second noise is Gaussian noise, which follows a standard normal distribution.

[0067] Furthermore, the first noise data can be represented as: .in, The second type of noise can be represented as: ; This is the first noise data; Noise scheduling parameters are added to each predefined step; t is the number of times noise is added. Cumulative coefficient .

[0068] Step A4: Determine the first step length, and match the corresponding noise data from the first noise data according to the first step length to obtain the second noise data; the first step length is any number of additions selected from the preset addition counts.

[0069] Specifically, select any number of additions from the preset number of additions as the first step length. Based on the first step length, obtain the noise data corresponding to the number of additions from the first noise data to obtain the second noise data.

[0070] For example, assuming the preset number of additions is 500, the first step length can be any one of the 500 additions. If the first step length is chosen as t=500, then the second noise data is: .

[0071] Step A5: Predict the second noise data and update the first noise data based on the prediction results.

[0072] Specifically, the second noise data is input into the noise prediction model for noise prediction, resulting in predicted noise. The accuracy of the predicted noise is assessed using a loss function, and the first noise is determined based on the assessment result. The second noise data is then denoised using the first noise to obtain third noise data. The noise data corresponding to the second step size in the first noise data is updated using the third noise data. After the update, the noise data of the previous step is inferred based on the updated noise data, continuing until t=0. The data corresponding to the frame number of the fourth pose vector in the final obtained data is used as the second pose vector.

[0073] The second step length is the number of additions preceding the first step length.

[0074] Optionally, the second noise data is predicted, and the first noise data is updated based on the prediction results, including steps B1-B3:

[0075] Step B1: Based on the second noise data, perform noise prediction to obtain the first noise.

[0076] Specifically, the second noise data is input into the noise prediction model for noise prediction to obtain the predicted noise. A loss function is calculated based on the noise added the corresponding number of times in the predicted noise and the second noise. The loss function is used to determine whether the predicted noise needs to be re-predicted. If the loss function meets the preset requirements, the predicted noise is used as the first noise; if the loss function does not meet the preset requirements, the predicted noise is re-predicted until the preset requirements are met, and the first noise is obtained.

[0077] The loss function uses Huber loss instead of the traditional mean squared error (MSE) loss to improve the model's robustness to outliers in the training data.

[0078] Furthermore, the first noise can be expressed as: .

[0079] Step B2: Generate third noise data based on the second noise data and the first noise data; the third noise data is the noise data corresponding to the second step length generated based on the first noise; the second step length is the number of additions preceding the first step length.

[0080] Specifically, the second noise data is denoised based on the first noise to obtain the third noise data.

[0081] Furthermore, the third noise data can be represented as: Where c is the fifth attitude vector; For parameters.

[0082] Step B3: Update the first noise data based on the third noise data.

[0083] Specifically, the noise data corresponding to the second step size in the first noise data is updated based on the third noise data. During the update, the noise data corresponding to the third attitude vector part needs to remain unchanged, and only the part corresponding to the fourth attitude vector is updated. After the update is completed, the noise data of the previous step is inferred based on the updated noise data until t=0. The noise data corresponding to the frame number of the fourth attitude vector in the final noise data is used as the second attitude vector.

[0084] Optionally, the second attitude vector is constrained and smoothly adjusted to obtain the third attitude vector, including steps C1-C3:

[0085] Step C1: Perform smoothing filtering on the second attitude vector to obtain the sixth attitude vector.

[0086] Specifically, the second attitude vector is smoothed using a filtering algorithm to smooth the motion trajectory, thus obtaining the sixth attitude vector.

[0087] The filtering algorithm can be either Bézier curve interpolation or exponential moving average filtering.

[0088] Among them, Bézier curve interpolation flexibly adjusts the curve shape by selecting control points in the second attitude vector, so as to achieve smooth changes in joint angle and angular velocity and avoid motion stuttering or abrupt changes.

[0089] Among them, the control points are the position and orientation information of the robot joints.

[0090] Among them, the exponential moving average filter smooths the joint angles, angular velocities, and other data that are affected by noise interference in the second attitude vector, while preserving the dynamic change trend within the second attitude vector.

[0091] Step C2: Apply motion constraints to the sixth posture vector to obtain the seventh posture vector; the motion constraints are used to ensure that the joint changes of the robot represented by the sixth posture vector conform to the laws of physical motion.

[0092] Specifically, angular velocity and angular acceleration are calculated based on the sixth attitude vector to obtain the first angle data. The first angle data is then constrained and verified according to the robot's dynamics to obtain the first result. Based on the first result, the sixth attitude vector is adjusted to obtain the seventh attitude vector.

[0093] Furthermore, constraint verification mainly includes: verifying whether the angular velocity exceeds the limit and verifying whether the robot's joints meet the maximum torque of the motor.

[0094] The angular velocity verification process involves verifying the acquired angular velocity frame by frame, specifically checking whether the absolute value of the angular velocity is less than the preset maximum angular velocity. If it is less than the preset maximum angular velocity, the angular velocity of that frame is within the limit; if it is greater than or equal to the preset maximum angular velocity, the angular velocity of that frame exceeds the limit.

[0095] The verification of the maximum torque is as follows: angular velocity and angular acceleration are substituted into the dynamic model to calculate the torque change of the robot joints in different frames. The obtained torque is compared with the maximum torque of the motor. If it is greater than the maximum torque limit of the motor, the torque requirement is not met; if it is less than or equal to the maximum torque limit of the motor, the torque requirement is met.

[0096] Furthermore, adjusting the sixth attitude vector based on the first result includes: if the first result contains a constraint result of angular velocity exceeding the limit, then by stretching the time of the corresponding action in the sixth attitude vector for that frame, the amplitude of the angular velocity is reduced. If the first result contains a constraint result that does not meet the torque requirement, then by adjusting the interpolation control point of the sixth attitude vector using a Bézier curve, the peak value of the angular acceleration is reduced.

[0097] Step C3: Apply beat constraints to the seventh posture vector to obtain the third posture vector; the beat constraints are used to make the robot's posture changes represented by the seventh posture vector conform to the beat changes in the audio data.

[0098] Specifically, the posture vectors where the robot's joint angles change abruptly are obtained from the seventh posture vector, serving as the first posture information. The matching timestamps are determined based on the first posture information, and motion beats are generated based on these timestamps to obtain the first beat sequence. The first beat sequence and the second beat sequence are compared beat by beat. If a beat misalignment exists at the same moment, the first beat sequence is updated based on the beat in the corresponding second beat sequence. If no beat misalignment exists at the same moment, the second result indicates that the beat is not misaligned at that moment, and no update is needed. Once all sequences have been compared, the posture in the seventh posture vector is adjusted based on the updated first beat sequence to obtain the third posture vector.

[0099] Optionally, motion constraints are applied to the sixth attitude vector to obtain the seventh attitude vector, including steps D1-D3:

[0100] Step D1: Perform angle analysis on the sixth attitude vector to obtain the first angle data.

[0101] Specifically, angular velocity and angular acceleration are calculated based on the sixth attitude vector to obtain the first angle data.

[0102] Step D2: Constrain the first angle data according to the dynamic model to obtain the first result.

[0103] The dynamic model can be expressed as: .in, This is the driving torque vector of the robot joint, which is limited by the maximum torque of the motor, i.e. .

[0104] The first result is used to characterize whether the robot's angular velocity and angular acceleration satisfy the robot's dynamic model, that is, whether the robot's angular velocity and angular acceleration satisfy the laws of kinematics.

[0105] Specifically, angular velocity and angular acceleration are input into the dynamic model to calculate the torque changes of the robot joints in different frames. The acquired torque is compared with the maximum torque of the motor. If it exceeds the maximum motor torque limit, the torque requirement is not met; if it is less than or equal to the maximum motor torque limit, the torque requirement is met. The acquired angular velocity is verified frame by frame, that is, the absolute value of the angular velocity is checked to see if it is less than the preset maximum angular velocity. If it is less than the preset maximum angular velocity, the angular velocity in that frame is within the limit; if it is greater than or equal to the preset maximum angular velocity, the angular velocity in that frame exceeds the limit.

[0106] Step D3: Adjust the sixth attitude vector based on the first result to obtain the seventh attitude vector.

[0107] Specifically, if the first result contains a constraint that causes angular velocity to exceed the limit, the amplitude of the angular velocity is reduced by stretching the corresponding action in the sixth attitude vector for that frame over time. If the first result contains a constraint that does not meet the torque requirement, the interpolation control point of the sixth attitude vector is adjusted using a Bézier curve to reduce the peak value of the angular acceleration.

[0108] Optionally, a clock constraint is applied to the seventh attitude vector to obtain the third attitude vector, including steps E1-E4:

[0109] Step E1: Obtain the first posture information from the seventh posture vector; the first posture information is used to characterize the postures represented by the start, end and turning points of the robot's action; the turning point is the critical state point where the robot changes from one posture to another.

[0110] Specifically, the posture vectors at which the robot's joint angles change abruptly are obtained from the seventh posture vector, namely the posture vectors corresponding to the start of the action, the end of the action, and the turning point, and are used as the first posture information.

[0111] Step E2: Generate the first beat sequence based on the first posture information and the corresponding timestamp.

[0112] Specifically, the timestamp corresponding to the first posture information is obtained, and the movement beat is generated based on the obtained timestamp to obtain the first beat sequence.

[0113] Step E3: Compare the first beat sequence with the second beat sequence to obtain the second result; the second beat sequence is the beat sequence that matches the rhythm of the audio data.

[0114] Specifically, the first beat sequence and the second beat sequence are compared beat by beat. If there is a beat misalignment at the same moment, the second result is that the beat is misaligned at that moment; if there is no beat misalignment at the same moment, the second result is that the beat is not misaligned at that moment.

[0115] Step E4: Adjust the seventh attitude vector according to the second result to obtain the third attitude vector.

[0116] Specifically, if the second result indicates a beat misalignment at that moment, the first beat sequence is updated based on the beats in the second beat sequence corresponding to that moment, until all beats have been compared. The attitude in the seventh attitude vector is then adjusted based on the updated first beat sequence to obtain the third attitude vector.

[0117] Optionally, the robot's attitude is adjusted based on the third attitude vector, including steps F1-F3:

[0118] Step F1: Convert the third attitude vector into third angle data using inverse kinematics.

[0119] Specifically, the third attitude vector is used to calculate the end effector trajectory using a positive dynamics model. The ideal end effector trajectory x is defined. d The ideal end effector motion trajectory is solved based on the end effector motion trajectory to obtain the joint angle change and obtain the third angle data.

[0120] There is a mapping relationship between the end effector's motion trajectory and the third attitude vector; there is a mapping relationship between the ideal end effector's motion trajectory and the change in angle.

[0121] The positive dynamic model can be expressed as: x t =f(P t ); where P t This is the third attitude vector. t This represents the motion trajectory of the end effector.

[0122] Step F2: Perform self-collision detection on the third angle data to obtain the fourth angle data.

[0123] Specifically, the third angle data is matched with the second angle data according to the joints and compared. Based on the comparison results, the third angle data is corrected to obtain the fourth angle data.

[0124] The comparison dimension is the angular range.

[0125] Further, the comparison process is as follows: Select any joint from the third angle data and match the corresponding angle data to obtain the angle data to be compared. Compare the angle data to be compared with the angle range of the corresponding joint in the second angle data. If the angle data to be compared falls within the angle range of the joint, the angle change meets the requirements; if the angle data to be compared falls outside the angle range of the joint, the angle change does not meet the requirements and angle adjustment is needed. In other words, based on the overall angle data change corresponding to the joint, the excess angle data is limited to the angle range without affecting the overall movement of the robot.

[0126] Furthermore, after allocating the angle data of all robot joints to their corresponding angle ranges, an overlap judgment is made on the angle changes of all robot joints. That is, it is determined whether the links connecting different joints overlap under the same action. Based on the judgment result, the third angle data is further adjusted so that the angle changes meet the angle range and do not cause collisions.

[0127] For example, assuming the joint is the robot's elbow joint, the connecting links are the robot's upper arm and lower arm. In some action frames, the upper and lower arms overlap.

[0128] Step F3: Adjust the robot's posture based on the fourth angle data.

[0129] Specifically, the acquired fourth-angle data is sent to the robot's underlying controller to drive the motors and adjust the robot's posture.

[0130] The technical solution of this embodiment involves determining a first posture vector. This first posture vector is a concatenated vector of feature vectors from different dimensions, used to describe the robot's changing states from different dimensions. Determining the first posture vector allows for the fusion of feature information from different dimensions, providing a reference for robot posture prediction from various angles, thus making the acquired posture more closely match the object to be represented. Posture prediction is performed on the first posture vector to obtain a second posture vector. Constraints and smoothing adjustments are applied to the second posture vector to obtain a third posture vector. Smoothing and constraint adjustments make the acquired third posture vector more closely match the robot's motion patterns while also improving the smoothness of the robot's movements. The robot's posture is then adjusted based on the third posture vector. This method generates posture vectors based on feature information from different dimensions, predicts the posture vectors, constrains and smooths the prediction results to obtain a third posture vector, and adjusts the robot's posture based on the third posture vector. This enables the acquired robot posture to better match the changing postures of the object to be represented, while also improving the robot's stability and interactivity.

[0131] Figure 3 This is a schematic diagram of a robot posture generation device provided in an embodiment of the present invention. This embodiment is applicable to situations where changing postures are generated in real time based on the robot's movements. The robot posture generation device can be implemented in hardware and / or software, and can be configured in any electronic device with network communication capabilities. Figure 3 As shown, the device includes: a first attitude vector determination module 210, a second attitude vector determination module 220, a third attitude vector determination module 230, and an adjustment module 240, wherein:

[0132] First attitude vector determination module 210: used to determine the first attitude vector; the first attitude vector is a concatenated vector of feature vectors of different dimensions, used to describe the robot's changing state from different dimensions;

[0133] Second attitude vector determination module 220: used to perform attitude prediction on the first attitude vector to obtain the second attitude vector; the second attitude vector is used to characterize the robot's action change after a preset number of frames following the first attitude; the first attitude is the robot's attitude change before a preset number of frames at a preset time.

[0134] Third attitude vector determination module 230: used to constrain and smooth the second attitude vector to obtain the third attitude vector;

[0135] Adjustment module 240: used to adjust the robot's posture based on the third posture vector.

[0136] Optionally, the second attitude vector determination module 220 includes:

[0137] Fourth attitude vector determination unit: used to determine the fourth attitude vector; the fourth attitude vector is used to characterize the robot's attitude change within a preset number of frames after the second attitude;

[0138] Fifth attitude vector determination unit: used to concatenate the third attitude vector with the fourth attitude vector to obtain the fifth attitude vector;

[0139] First noise data determination unit: used to add noise to the fifth attitude vector according to a preset time step, and when the number of additions is equal to the preset number of additions, the first noise data is obtained;

[0140] Second noise data determination unit: used to determine the first step length, and match the corresponding noise data from the first noise data according to the first step length to obtain the second noise data; the first step length is any number of additions selected from the preset addition number;

[0141] Update unit: used to predict the second noise data and update the first noise data based on the prediction results.

[0142] Optional update units include:

[0143] First noise determination subunit: used to predict noise based on second noise data to obtain first noise;

[0144] The third noise data determination subunit is used to generate third noise data based on the second noise data and the first noise; the third noise data is the noise data corresponding to the second step size generated based on the first noise; the second step size is the number of additions preceding the first step size.

[0145] Update sub-unit: Used to update the first noise data based on the third noise data.

[0146] Optionally, the third attitude vector determination module 230 includes:

[0147] The sixth attitude vector determination unit is used to perform smoothing filtering on the second attitude vector to obtain the sixth attitude vector.

[0148] The seventh posture vector determination unit is used to apply motion constraints to the sixth posture vector to obtain the seventh posture vector. The motion constraints are used to ensure that the joint changes of the robot represented by the sixth posture vector conform to the laws of physical motion. The seventh posture vector is also used to apply beat constraints to obtain the third posture vector. The beat constraints are used to ensure that the posture changes of the robot represented by the seventh posture vector conform to the beat changes in the audio data.

[0149] Optionally, the seventh attitude vector determination unit includes:

[0150] First Angle Data Determination Subunit: Used to perform angle analysis on the sixth attitude vector to obtain the first angle data;

[0151] The first result determination sub-unit is used to constrain the first angle data according to the dynamic model to obtain the first result;

[0152] The seventh attitude vector determination sub-unit is used to adjust the sixth attitude vector based on the first result to obtain the seventh attitude vector.

[0153] Optionally, the seventh attitude vector determination unit includes:

[0154] First posture information determination subunit: used to obtain first posture information from the seventh posture vector; the first posture information is used to characterize the postures represented by the start, end and turning points of the robot's action; the turning point is the critical state point where the robot changes from one posture to another.

[0155] The first beat sequence determination subunit is used to generate the first beat sequence based on the first attitude information and the corresponding timestamp.

[0156] The second result determination subunit is used to compare the first beat sequence with the second beat sequence to obtain the second result; the second beat sequence is the beat sequence that matches the rhythm of the audio data.

[0157] The third attitude vector determination sub-unit is used to adjust the seventh attitude vector based on the second result to obtain the third attitude vector.

[0158] Optionally, adjustment module 240 includes:

[0159] Third Angle Data Determination Unit: Used to convert the third attitude vector into third angle data through inverse kinematics;

[0160] Fourth Angle Data Determination Unit: Used to perform self-collision detection on the third angle data to obtain the fourth angle data;

[0161] Adjustment unit: Used to adjust the robot's posture based on the fourth angle data.

[0162] The robot posture generation device provided in the embodiments of the present invention can execute the robot posture generation method provided in any of the embodiments of the present invention, and has the corresponding functions and beneficial effects of executing the robot posture generation method. For details, please refer to the relevant operations of the robot posture generation method in the foregoing embodiments.

[0163] Figure 4 This is a schematic diagram of an electronic device for implementing the robot pose generation method of this invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0164] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0165] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0166] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as robot pose generation methods.

[0167] In some embodiments, the robot pose generation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the robot pose generation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the robot pose generation method by any other suitable means (e.g., by means of firmware).

[0168] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0169] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0170] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0171] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0172] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0173] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0174] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0175] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating robot pose, characterized in that, include: Determine the first attitude vector; the first attitude vector is a concatenated vector of feature vectors of different dimensions, used to describe the robot's changing state from different dimensions; The first attitude vector is used to predict the attitude to obtain a second attitude vector; the second attitude vector is used to characterize the robot's motion change after a preset number of frames following the first attitude; the first attitude is the robot's attitude change before a preset number of frames at a preset time. The second attitude vector is constrained and smoothly adjusted to obtain the third attitude vector; The robot's posture is adjusted based on the third posture vector.

2. The method according to claim 1, characterized in that, The step of performing attitude prediction on the first attitude vector to obtain the second attitude vector includes: Determine the fourth attitude vector; the fourth attitude vector is used to characterize the robot's attitude change within a preset number of frames after the second attitude; The third attitude vector is concatenated with the fourth attitude vector to obtain the fifth attitude vector; Noise is added to the fifth attitude vector at preset time steps. When the number of additions equals the preset number of additions, the first noise data is obtained. The first step length is determined, and the corresponding noise data is matched from the first noise data according to the first step length to obtain the second noise data; the first step length is any number of additions selected from the preset addition number; The second noise data is predicted, and the first noise data is updated based on the prediction result.

3. The method according to claim 2, characterized in that, The step of predicting the second noise data and updating the first noise data based on the prediction result includes: Based on the second noise data, noise prediction is performed to obtain the first noise; Third noise data is generated based on the second noise data and the first noise; the third noise data is noise data corresponding to the second step size generated based on the first noise; the second step size is the number of additions preceding the first step size; The first noise data is updated based on the third noise data.

4. The method according to claim 1, characterized in that, The process of constraining and smoothing the second attitude vector to obtain the third attitude vector includes: The second attitude vector is smoothed and filtered to obtain the sixth attitude vector. A motion constraint is applied to the sixth posture vector to obtain a seventh posture vector; the motion constraint is used to make the joint changes of the robot represented by the sixth posture vector conform to the laws of physical motion; a beat constraint is applied to the seventh posture vector to obtain a third posture vector; the beat constraint is used to make the posture changes of the robot represented by the seventh posture vector conform to the beat changes in the audio data.

5. The method according to claim 4, characterized in that, The step of applying motion constraints to the sixth attitude vector to obtain the seventh attitude vector includes: The sixth attitude vector is analyzed to obtain the first angle data; The first angle data is constrained based on the dynamic model to obtain the first result; The sixth attitude vector is adjusted based on the first result to obtain the seventh attitude vector.

6. The method according to claim 4, characterized in that, The step of applying a beat constraint to the seventh attitude vector to obtain the third attitude vector includes: The first posture information is obtained from the seventh posture vector; the first posture information is used to characterize the postures represented by the start, end and turning points of the robot's action; the turning point is the critical state point where the robot changes from one posture to another. Based on the first posture information and the corresponding timestamp, generate the first beat sequence; The first beat sequence is compared with the second beat sequence to obtain a second result; the second beat sequence is a beat sequence that matches the rhythm of the audio data. The seventh attitude vector is adjusted based on the second result to obtain the third attitude vector.

7. The method according to claim 1, characterized in that, The step of adjusting the robot's attitude based on the third attitude vector includes: The third attitude vector is converted into third angle data through inverse kinematics; Perform self-collision detection on the third angle data to obtain the fourth angle data; The robot's posture is adjusted based on the fourth angle data.

8. A robot posture generation device, characterized in that, include: The first attitude vector determination module is used to determine the first attitude vector; the first attitude vector is a concatenated vector of feature vectors of different dimensions, used to describe the robot's changing state from different dimensions. The second attitude vector determination module is used to predict the attitude of the first attitude vector to obtain the second attitude vector; the second attitude vector is used to characterize the action change of the robot after a preset number of frames following the first attitude; the first attitude is the attitude change of the robot before a preset number of frames at a preset time. The third attitude vector determination module is used to constrain and smooth the second attitude vector to obtain the third attitude vector; An adjustment module is used to adjust the robot's posture based on the third posture vector.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the robot pose generation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the robot pose generation method according to any one of claims 1-7.

Images