A method and system for cross-robot action reorientation that preserves emotional expression

Abstract

The application discloses a cross-robot action reorientation method and system for keeping emotional expression, aiming to solve the problem of how to keep the emotional expression features carried by actions in the cross-robot action migration process, while taking into account the consistency, stability and smoothness of the actions. The technical solution points are as follows: a cross-robot action reorientation method for keeping emotional expression, comprising the following steps: S1, collecting source action data of a source action subject; S2, pre-processing the source action data and extracting basic action features; S3, extracting multi-dimensional emotional features from the basic action features and constructing a standardized emotional vector. The cross-robot action reorientation method for keeping emotional expression can accurately keep and restore the emotional expression features contained in the actions in the process of migrating the source actions to the target robot by constructing the standardized emotional vector and the optimization objective function containing the emotional constraint term, and the scheme has strong universality.

Description

Technical Field

[0001] This invention relates to the field of robot motion redirection processing technology, and more specifically, to a cross-robot motion redirection method and system that preserves emotional expression. Background Technology

[0002] With the rapid development of embodied intelligence and robotics, robots are gradually evolving from traditional industrial execution devices into intelligent agents with perception, interaction, and emotional expression capabilities. In the process of robot behavior learning and motion control, motion retargeting is a key technology. Its main purpose is to transfer the actions of the source action subject to the target robot body, achieving action reproduction and behavior control.

[0003] Most existing motion retargeting techniques focus on the consistency of motion geometry and trajectory, neglecting the emotional information conveyed within the motion. In reality, the same motion exhibits significant differences in amplitude, rhythm, speed, postural tension, joint acceleration, and spatial trajectory under different emotional states. For example, in a happy state, the motion amplitude is larger and the speed is faster, while in a sad state, the motion amplitude is reduced and the rhythm is slower.

[0004] Therefore, traditional motion retargeting methods lead to the following problems: loss of emotional features, distortion of movement style, inconsistent movement rhythm, and lack of emotional consistency in robot expression. Furthermore, due to significant differences in the number of joints, degrees of freedom, actuator performance, and motion structure among different robots, traditional motion retargeting schemes struggle to simultaneously achieve consistency in emotional expression, motion consistency, motion stability, and motion smoothness.

[0005] Therefore, how to maintain the emotional expression characteristics of actions during cross-robot motion transfer, while taking into account the consistency, stability and smoothness of actions, has become an urgent problem for those skilled in the art. Summary of the Invention

[0006] In order to maintain the emotional expression characteristics of actions during cross-robot motion transfer, while taking into account the consistency, stability and smoothness of the actions, this invention provides a cross-robot motion redirection method and system for maintaining emotional expression.

[0007] In a first aspect, the present invention provides a cross-robot action redirection method for maintaining emotional expression, employing the following technical solution: A method for cross-robot action redirection while preserving emotional expression includes: S1: Collect source action data of the source action subject; S2: Preprocess the source motion data and extract basic motion features; S3: Extract multi-dimensional emotional features from the basic action features and construct a standardized emotional vector; S4: Establish the skeleton mapping relationship between the source action subject and the target robot; S5: Based on the skeleton mapping relationship, the source action is redirected to the initial action of the target robot to obtain the initial posture sequence; S6: Construct an optimization objective function that includes an emotion constraint term, and perform nonlinear iterative optimization on the initial pose sequence with the goal of minimizing the composite loss function; S7: Generate the motion control sequence for the target robot based on the optimization results; S8: Drive the target robot to run using the motion control sequence, and dynamically correct subsequent actions based on the robot's real-time running feedback.

[0008] By adopting the above technical solution and constructing a standardized emotion vector and an optimized objective function containing emotion constraints, it is possible to accurately retain and restore the emotional expression characteristics contained in the action during the process of transferring the source action to the target robot. This breaks through the shortcomings of traditional cross-robot action transfer, which only replicates the appearance and loses the emotion of the action. It can complete the accurate transfer of actions between heterogeneous robots while fully preserving the original emotional expression, reducing the distortion of action style, improving the anthropomorphism of robot actions, optimizing the naturalness of human-computer interaction, and the solution has strong versatility and can be adapted to various types of devices such as humanoid, quadrupedal, and service robots.

[0009] The present invention is further configured such that the emotional feature extraction step in S3 includes: acquiring the temporal data of the skeletal nodes of the source action subject, calculating the instantaneous velocity, instantaneous acceleration and jerk of each skeletal node during the execution of the action; constructing a dynamic feature vector representing the force and smoothness of the action based on the calculation results, performing time-frequency domain analysis on the dynamic feature vector, and extracting spectral features representing the rhythm and energy distribution of the action; and fusing the dynamic feature vector and the spectral features to generate emotional expression features.

[0010] By adopting the above technical solution, emotion-related features are automatically extracted from objective motion data, breaking away from the model of relying on human subjective judgment of emotions, realizing the quantitative collection of action emotions, and providing reliable underlying data support for subsequent emotion vector construction.

[0011] The present invention is further configured such that the construction method of the standardized emotion vector in S3 specifically includes: normalizing the dynamic feature vector and the spectral feature to eliminate the dimensional differences of different action amplitudes and durations; mapping the normalized features to a preset multi-dimensional emotional semantic space, including three orthogonal dimensions: pleasure, arousal, and dominance; calculating the coordinate values ​​of the mapped features in the multi-dimensional emotional semantic space, and concatenating and reducing the dimensions of the coordinate values ​​to generate a standardized action-emotion vector.

[0012] By adopting the above technical solution, and through normalization and multi-dimensional emotional semantic space mapping, the emotions of joy, anger, sorrow and happiness are transformed into calculable standardized emotional vectors, thereby realizing the digitization of emotions.

[0013] The present invention is further configured such that the standardized action-emotion vector expression is: E = {e1,e2,e3,e4,...,en}, where: e1 represents action intensity, e2 represents action extensibility, e3 represents action rhythm, e4 represents action stability, and en represents other emotional feature parameters.

[0014] By adopting the above technical solution, the abstract emotion of movement is decomposed into multiple concrete physical indicators such as movement intensity, limb extension, movement rhythm, and movement stability, thereby transforming the invisible and intangible emotion into digital parameters that can be read and calculated by computers.

[0015] The present invention is further configured such that the skeleton mapping method in S4 specifically includes: acquiring the source skeleton topology and the target skeleton topology respectively, and establishing a correspondence table between the source joints and the target joints; acquiring the source skeleton length of the source action subject in the standard posture, and the target skeleton length of the target robot in the standard posture; and calculating the non-uniform local scaling factor of each corresponding joint based on the ratio of the source skeleton length to the target skeleton length, so as to construct a motion constraint mapping model.

[0016] By adopting the above technical solutions, we can adapt to heterogeneous robots with different link lengths, number of joints, and degree of freedom configurations, and solve the problems of motion scaling mismatch and joint over-limit caused by differences in body structure.

[0017] The present invention is further configured such that the optimization objective function expression in S6 is: Loss = λ1L motion +λ2L emotion +λ3L smooth +λ4L balance ; where: L motion Indicates the error in the motion trajectory; L emotion Indicates the error in sentiment characteristics; L smooth Indicates motion smoothing constraints; L balance λ1 to λ4 represent the robot's balance constraints; λ1 to λ4 represent the weighting coefficients.

[0018] By adopting the above technical solutions, the optimization process simultaneously takes into account four dimensions: motion appearance, emotion preservation, motion smoothness, and body stability. This solves the contradiction that traditional optimization only focuses on trajectory and abandons emotion, or focuses on emotion and causes the motion to exceed the limits of robot hardware and cannot be implemented.

[0019] Secondly, this invention provides a cross-robot action redirection system that preserves emotional expression, employing the following technical solution: A cross-robot motion redirection system that preserves emotional expression includes: Acquisition module: Used to acquire raw motion data of the source action subject in real time; A memory for storing programs that implement any of the above-mentioned methods for cross-robot action redirection while maintaining emotional expression; The processor is used to load and execute programs stored in memory.

[0020] In summary, the present invention has the following beneficial effects: By constructing standardized emotion vectors and an optimized objective function that includes emotion constraints, it is possible to accurately preserve and restore the emotional expression features contained in the action during the process of transferring the source action to the target robot. This breaks through the shortcomings of traditional cross-robot action transfer, which only replicates the appearance and loses the emotion of the action. It can complete the accurate transfer of actions between heterogeneous robots while fully preserving the original emotional expression, reducing the distortion of action style, improving the anthropomorphism of robot actions, optimizing the naturalness of human-computer interaction, and the solution has strong versatility and can be adapted to various types of devices such as humanoid, quadruped, and service robots. By automatically extracting emotion-related features from objective motion data, we can break away from the model of relying on human subjective judgment of emotions, realize the quantitative collection of action emotions, and provide reliable underlying data support for subsequent action retargeting. The optimization process simultaneously considers four dimensions: motion appearance, emotion preservation, smooth movement, and body stability. This solves the contradiction of traditional optimization that only focuses on trajectory and abandons emotion, or focuses on emotion and causes the motion to exceed the limits of the robot's hardware and become impossible to implement. Attached Figure Description

[0021] Figure 1 This is a flowchart of a cross-robot motion redirection method that preserves emotional expression. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0023] S1: Collect source action data of the source action subject; Source motion data includes human motion capture data, virtual character motion data, source robot motion data, video motion estimation data, and multimodal emotion-motion fusion data (such as voice emotion).

[0024] Motion acquisition methods can include optical motion capture, inertial motion capture, RGBD visual acquisition, depth camera acquisition, LiDAR acquisition, multi-camera pose estimation, and wearable motion acquisition devices.

[0025] The dimensions of motion data should include at least: joint position, joint rotation angle, posture trajectory, motion speed, motion acceleration, motion rhythm information, and motion timing information.

[0026] S2: Preprocess the source motion data and extract basic motion features; Basic motion features include the skeletal node trajectory of the source motion subject, joint rotation parameters, and motion timing features.

[0027] S3: Extract multi-dimensional emotional features from basic action features and construct standardized emotional vectors; The steps for extracting emotional features include: acquiring the temporal data of the skeletal nodes of the source action subject, calculating the instantaneous velocity, instantaneous acceleration, and jerk of each skeletal node during the execution of the action; constructing a dynamic feature vector representing the force and smoothness of the action based on the calculation results, performing time-frequency domain analysis on the dynamic feature vector, and extracting spectral features representing the rhythm and energy distribution of the action; and fusing the dynamic feature vector and spectral features to generate emotional expression features.

[0028] The construction of standardized emotion vectors specifically includes: normalizing the dynamic feature vectors and spectral features to eliminate the dimensional differences in the amplitude and duration of different actions; mapping the normalized features to a preset multi-dimensional emotional semantic space, including three orthogonal dimensions: pleasure, arousal, and dominance; calculating the coordinate values ​​of the mapped features in the multi-dimensional emotional semantic space, and then concatenating and reducing the dimensionality of the coordinate values ​​to generate standardized action-emotion vectors.

[0029] The expression for the action-emotion vector is: E = {e1,e2,e3,e4,...,en}, where: e1 represents action intensity, e2 represents action extensibility, e3 represents action rhythm, e4 represents action stability, and en represents other emotional feature parameters.

[0030] S4: Establish the skeleton mapping relationship between the source action subject and the target robot; The skeleton mapping method specifically includes: obtaining the source skeleton topology and the target skeleton topology respectively, and establishing a correspondence table between the source joints and the target joints; obtaining the source skeleton length of the source action subject in the standard posture, and the target skeleton length of the target robot in the standard posture; and calculating the non-uniform local scaling factor of each corresponding joint based on the ratio of the source skeleton length to the target skeleton length, so as to construct a motion constraint mapping model.

[0031] The mapping includes: joint correspondence, degree-of-freedom mapping, posture space mapping, range of motion constraints, joint limit constraints, and dynamic constraints. For cases where no corresponding joint exists, methods such as motion compensation, motion interpolation, posture approximation, local motion reconstruction, and trajectory replanning are employed.

[0032] S5: Based on the skeleton mapping relationship, the source motion is redirected to the initial motion of the target robot to obtain the initial posture sequence; The motion constraint mapping model is used to initially convert the joint parameters of the source action subject into the initial posture sequence of the target robot.

[0033] S6: Construct an optimization objective function that includes an emotion constraint term, and perform nonlinear iterative optimization on the initial pose sequence with the goal of minimizing the composite loss function; The objective function expression is: Loss = λ1L motion +λ2L emotion +λ3L smooth +λ4L balance ; where: L motion Indicates the error in the motion trajectory; L emotion Indicates the error in sentiment characteristics; L smooth Indicates motion smoothing constraints; L balance λ1 to λ4 represent the robot's balance constraints; λ1 to λ4 represent the weighting coefficients.

[0034] S7: Generate the motion control sequence for the target robot based on the optimization results; The motion control sequence includes: joint angle control sequence, actuator control signal, posture control parameters, motion time sequence, and balance control parameters.

[0035] S8: Drive the target robot to run using motion control sequences, and dynamically correct subsequent actions based on real-time feedback from the robot's operation.

[0036] Feedback information includes posture feedback, joint status feedback, force feedback, motion stability feedback, and movement deviation feedback.

[0037] Based on the same inventive concept, embodiments of the present invention provide a cross-robot action redirection system for maintaining emotional expression, comprising: Acquisition module: Used to acquire raw motion data of the source action subject in real time; Memory for storing programs that implement a method for redirecting actions across robots while preserving emotional expression; The processor is used to load and execute programs stored in memory.

[0038] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A cross-robot action redirection method that preserves emotional expression, characterized in that: include: S1: Collect source action data of the source action subject; S2: Preprocess the source motion data and extract basic motion features; S3: Extract multi-dimensional emotional features from the basic action features and construct a standardized emotional vector; S4: Establish the skeleton mapping relationship between the source action subject and the target robot; S5: Based on the skeleton mapping relationship, the source action is redirected to the initial action of the target robot to obtain the initial posture sequence; S6: Construct an optimization objective function that includes an emotion constraint term, and perform nonlinear iterative optimization on the initial pose sequence with the goal of minimizing the composite loss function; S7: Generate the motion control sequence for the target robot based on the optimization results; S8: Drive the target robot to run using the motion control sequence, and dynamically correct subsequent actions based on the robot's real-time running feedback.

2. The method for cross-robot action redirection while maintaining emotional expression according to claim 1, characterized in that: The emotional feature extraction step in S3 includes: acquiring the temporal data of the skeletal nodes of the source action subject, calculating the instantaneous velocity, instantaneous acceleration, and jerk of each skeletal node during the action execution process; constructing a dynamic feature vector representing the force and smoothness of the action based on the calculation results, performing time-frequency domain analysis on the dynamic feature vector, and extracting spectral features representing the rhythm and energy distribution of the action; and fusing the dynamic feature vector and spectral features to generate emotional expression features.

3. The method for cross-robot action redirection while maintaining emotional expression according to claim 1, characterized in that: The construction method of the standardized emotion vector in S3 specifically includes: normalizing the dynamic feature vector and spectral feature to eliminate the dimensional differences of different action amplitudes and durations; mapping the normalized features to a preset multi-dimensional emotional semantic space, including three orthogonal dimensions: pleasure, arousal, and dominance; calculating the coordinate values ​​of the mapped features in the multi-dimensional emotional semantic space, and concatenating and reducing the dimensionality of the coordinate values ​​to generate a standardized action-emotion vector.

4. A cross-robot action redirection method for maintaining emotional expression according to claim 3, characterized in that: The standardized action-emotion vector expression is: E = {e1,e2,e3,e4,...,en}, where: e1 represents action intensity, e2 represents action extensibility, e3 represents action rhythm, e4 represents action stability, and en represents other emotional feature parameters.

5. A cross-robot action redirection method for maintaining emotional expression according to claim 1, characterized in that: The skeleton mapping method in S4 specifically includes: acquiring the source skeleton topology and the target skeleton topology respectively, and establishing a correspondence table between the source joints and the target joints; acquiring the source skeleton length of the source action subject in the standard posture, and the target skeleton length of the target robot in the standard posture; and calculating the non-uniform local scaling factor of each corresponding joint based on the ratio of the source skeleton length to the target skeleton length, so as to construct a motion constraint mapping model.

6. A cross-robot action redirection method for maintaining emotional expression according to claim 1, characterized in that: The objective function expression in S6 is: Loss = λ1L motion +λ2L emotion +λ3L smooth +λ4L balance ; where: L motion Indicates the error in the motion trajectory; L emotion Indicates the error in sentiment characteristics; L smooth Indicates motion smoothing constraints; L balance λ1 to λ4 represent robot balance constraints; λ1 to λ4 represent weighting coefficients.

7. A cross-robot motion redirection system that preserves emotional expression, characterized in that: include: Acquisition module: Used to acquire raw motion data of the source action subject in real time; A memory for storing a program that implements a cross-robot action redirection method for maintaining emotional expression as described in any one of claims 1 to 6; The processor is used to load and execute programs stored in memory.

Images