Motion capture-based motion object control method and system

Abstract

The application provides a motion object control method and system based on motion capture. The method comprises the following steps: acquiring full-body posture data of a target operator collected by a motion capture device; performing coordinate system conversion on the full-body posture data to obtain an object target position sequence corresponding to a coordinate system of a target motion object; performing redundant motion detection on the object target position sequence; performing adaptive smoothing processing on the object target position sequence according to the result of the redundant motion detection to obtain a smoothed object control instruction; and inputting the smoothed object control instruction into a driving controller of the target motion object to control the target motion object to reduce unnecessary driving actions and lower energy consumption when performing the captured motion.

Description

Technical Field

[0001] This application relates to equipment control technology, and more particularly to a method and system for controlling moving objects based on motion capture. Background Technology

[0002] With the development of human-computer interaction and intelligent manufacturing, motion capture-based motion object control methods are widely used in systems such as humanoid robots, collaborative robots, and remotely operated robotic arms. Existing motion capture devices (including optical capture systems, inertial measurement units, and magnetic sensing systems) can collect the full-body posture data of the target operator in real time and map the captured posture information into joint position commands in the corresponding robot joint coordinate system through coordinate system transformation. This type of control method can realize the real-time reproduction of operator movements, enabling robots to achieve highly realistic human motion reproduction, and has strong application potential in entertainment, medical rehabilitation, virtual reality, and industrial production scenarios.

[0003] However, in the existing technology, the acquisition process of motion capture equipment is inevitably affected by the operator's non-purposeful micro-movements, and the obtained joint target position sequence often contains redundant motion data with high frequency, small amplitude and no relation to the actual task target.

[0004] When this redundant data is directly input into the robot's joint drive controller without processing, it causes the robot joints to perform a large number of meaningless, small-amplitude driving movements. This not only increases the ineffective energy consumption of the drive system but may also cause micro-jitter in the joint posture, and even affect the robot's endurance during long-term operation and the lifespan of the joint mechanical components. Summary of the Invention

[0005] This application provides a motion capture-based motion object control method and system to control the joints of a target robot to reduce unnecessary joint driving movements and lower energy consumption when performing capture actions.

[0006] In a first aspect, this application provides a method for controlling a moving object based on motion capture, including:

[0007] Acquire full-body posture data of the target operator collected by the motion capture device;

[0008] The whole-body posture data is transformed to obtain the object target position sequence corresponding to the target moving object coordinate system;

[0009] Redundant motion detection is performed on the target position sequence of the object, and based on the result of the redundant motion detection, adaptive smoothing processing is performed on the target position sequence of the object to obtain a smooth object control command;

[0010] The smooth object control command is input to the drive controller of the target moving object.

[0011] Secondly, this application provides a motion capture-based motion object control system, comprising:

[0012] The acquisition module is used to acquire the full-body posture data of the target operator collected by the motion capture device;

[0013] The processing module is used to perform coordinate system transformation on the whole-body posture data to obtain the object target position sequence corresponding to the target moving object coordinate system;

[0014] The processing module is used to perform redundant motion detection on the target position sequence of the object, and based on the result of the redundant motion detection, to perform adaptive smoothing processing on the target position sequence of the object to obtain a smoothed object control command;

[0015] The control module is used to input the smooth object control command to the drive controller of the target moving object.

[0016] The motion capture-based motion object control method and system provided in this application acquires the full-body posture data of the target operator collected by the motion capture device, then performs coordinate system transformation on the full-body posture data to obtain the joint target position sequence corresponding to the joint coordinate system of the target robot, and performs redundant motion detection on the joint target position sequence. Based on the results of the redundant motion detection, the joint target position sequence is adaptively smoothed to obtain smooth joint control commands. The smooth joint control commands are then input to the drive controller of the target robot joint, thereby controlling the target robot joint to reduce unnecessary joint drive movements and reduce energy consumption when performing capture actions. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario of a motion object control method based on motion capture, according to an example embodiment of this application.

[0019] Figure 2 This is a flowchart illustrating a motion object control method based on motion capture according to an example embodiment of this application;

[0020] Figure 3 This is a schematic flowchart illustrating the implementation of S140 according to an example embodiment of this application;

[0021] Figure 4This is a schematic diagram of the structure of a motion capture-based motion object control system according to an example embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application.

[0023] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] Figure 1 This is a schematic diagram illustrating an application scenario of a motion capture-based motion object control method according to an example embodiment of this application. Figure 2 This is a flowchart illustrating a motion object control method based on motion capture according to an example embodiment of this application. Figures 1-2 As shown, the motion capture-based motion object control method provided in this embodiment includes:

[0026] S110. Acquire the full-body posture data of the target operator collected by the motion capture device.

[0027] In this step, multiple spatial position sensors of a motion capture device can simultaneously acquire the three-dimensional position coordinates and three-dimensional attitude angle data of all joints of the target operator's body. Then, the acquired three-dimensional position coordinates and three-dimensional attitude angle data are synchronized according to timestamps to form continuous full-body attitude time-series data. The multiple spatial position sensors include inertial measurement units, optical tracking modules, or depth cameras, which provide raw measurements of acceleration, angular velocity, and the positions of spatial marker points, respectively.

[0028] Optionally, real-time posture data of the operator's entire body can be collected using optical motion capture devices, inertial measurement units, or depth cameras, and this posture data can be mapped into joint movement commands for the robot. However, in practical applications, factors such as changes in the operator's body position, mutual occlusion of body parts, occlusion by external objects, and limitations in the field of view of optical sensors can cause motion capture markers or measurement signals at certain joints to be temporarily obstructed, resulting in the loss of posture data for that joint or a significant decrease in acquisition accuracy.

[0029] To address this, when occlusion occurs at the target operator's joints, redundant sensors collect joint inertial measurement data under occlusion conditions. This data is then combined with real-time posture data from unoccluded adjacent joints. A joint kinematic constraint model is used to calculate the predicted posture of the occluded joint. The redundant sensors include inertial measurement units (IMUs) installed at the easily occluded joints of the target operator. An extended Kalman filter algorithm is employed to perform weighted fusion based on the covariance matrix of the observation data from the motion capture device and the data from the IMUs, outputting continuous pose data for robot joint control. This predicted posture is then fused with the full-body pose data collected by the motion capture device to obtain continuous, occlusion-compensated full-body pose data.

[0030] S120. Perform coordinate system transformation on the whole body posture data to obtain the target position sequence corresponding to the target moving object coordinate system.

[0031] In this step, the rotation transformation matrix and translation vector between the motion capture device coordinate system and the target robot joint coordinate system can be calculated based on the reference coordinate system parameters of the target robot joint. Optionally, the reference coordinate system parameters are obtained through the initial calibration process of the target robot joint to ensure that the target joint position sequence after coordinate system transformation matches the physical structure of the target robot joint in terms of spatial attitude and position. Then, the three-dimensional position coordinates and three-dimensional attitude angle data corresponding to the whole-body posture data are sequentially mapped to the target robot joint coordinate system using the rotation transformation matrix and translation vector to obtain the corresponding relationship with the target robot joint coordinate system.

[0032] S130, Perform redundant motion detection on the target position sequence of the object.

[0033] In real-time robot joint actuation, the operator's minute, high-speed joint jitters and unnecessary muscle detail movements are often directly transmitted to the robot's joint execution layer. These high-frequency, minimal-amplitude joint movements are usually not functionally meaningful to the operator, but they trigger ineffective actuation at the robot's execution end, causing frequent micro-movements of the joint motors, increasing energy consumption, and accelerating wear on mechanical components. If the filtering used is merely to smooth the overall trajectory, it is difficult to identify and eliminate these high-frequency, micro-amplitude redundant movements. This leads to the system losing some effective motion details while reducing noise, affecting control accuracy and response speed.

[0034] In this step, the redundant motion detection described above can be performed by determining high-frequency components above a preset cutoff frequency in the joint target position sequence based on frequency domain analysis. Then, it is determined whether the amplitude of the joint position change corresponding to the high-frequency components is less than a preset joint micro-motion threshold based on an amplitude threshold.

[0035] Specifically, frequency domain analysis methods (such as Fast Fourier Transform) can be used to perform spectral decomposition on the input joint target position sequence to extract components in the sequence that are above a preset cutoff frequency. These high-frequency components correspond to unintentional shaking and subtle muscle contractions in most natural human movements. The system then determines the amplitude of these high-frequency components to see if the amplitude of the corresponding joint position change is below a preset joint micro-motion threshold.

[0036] This threshold can be designed based on the robot joint execution accuracy and the physical response characteristics of the drive mechanism. Specifically, it can be based on the angular resolution parameters of the motion capture device used by the target operator to determine the minimum resolvable joint angle change in a static state. Then, a constant-temperature static test is conducted on the target robot joint to collect the amplitude distribution of joint position changes caused only by environmental vibration or micro-motion of the system itself without active drive input. Statistical analysis is performed on this amplitude distribution to obtain the noise peak amplitude. Finally, the minimum resolvable joint angle change and the noise peak amplitude are weighted and combined to shield micro-motion signals that have almost no impact on the position of the end effector. When both frequency and amplitude determinations meet the redundancy motion condition, the component is marked as redundant and filtered.

[0037] Optionally, for the above weighted combination, in one possible implementation, if the motion capture resolution value is much larger than the noise peak value (e.g., resolution 0.05°, noise peak value 0.005°), it indicates that the limited accuracy is mainly caused by the capture device. In this case, the weight corresponding to the minimum resolvable joint angle change can be configured as 0.7-0.9, while the weight corresponding to the noise peak amplitude can be configured as 0.1-0.3. In another possible implementation, if the two values ​​are of similar magnitude or the noise peak value is large, the joint noise has a more significant impact. In this case, the weight of the former can be configured as 0.4-0.6, and the weight of the latter as 0.4-0.6. In yet another possible implementation, if the noise peak value is much larger than the resolution, the weight of the former can be configured as 0.1-0.3, and the weight of the latter as 0.7-0.9.

[0038] Through the aforementioned redundant motion detection, high-frequency redundant motion components in the joint target position sequence can be identified without weakening effective actions. Combined with motion amplitude thresholds, it can be further determined whether the motion is effective, thereby reducing invalid drive actions generated at the robot joint end and improving the system's energy consumption and execution reliability.

[0039] S140. Based on the results of redundant motion detection, the target position sequence of the object is adaptively smoothed to obtain a smoothed object control command.

[0040] It is worth noting that the collected joint position data inevitably includes sensor noise, ambient light interference, inertial drift, and micro-jitter from the operator's natural movements. When these interference signals directly drive the robot joints, they can cause jitter, unevenness, or even overshoot in the joint trajectory. While fixed-parameter filtering methods (such as low-pass filtering and moving average) can reduce the impact of noise to some extent, the filtering strength remains fixed under different movement speeds and noise levels, which can easily lead to significant delays at high speeds and loss of detail in low-speed, fine movements. Furthermore, simple velocity interpolation algorithms have limited effectiveness in eliminating abrupt acceleration changes and cannot simultaneously address noise suppression and dynamic response.

[0041] Therefore, in one possible implementation, Figure 3 This is a schematic flowchart illustrating the implementation of S140 according to an example embodiment of this application. Figure 3 As shown, S140 includes:

[0042] S141. Perform adaptive Kalman filtering on the joint target position sequence.

[0043] In this step, the joint target position sequence is subjected to adaptive Kalman filtering to dynamically adjust the ratio of process noise to measurement noise based on real-time noise estimation.

[0044] Specifically, this can involve acquiring the most recent historical sampling data of the joint target position sequence and the corresponding sensor measurements. Then, a measurement noise covariance matrix is ​​calculated based on the sensor measurements, and a process noise covariance matrix is ​​calculated based on the dynamic characteristics of the joint target position sequence. Next, the Kalman gain is determined based on the measurement noise covariance matrix and the process noise covariance matrix, and the Kalman gain is updated in real time to generate filtered joint position data.

[0045] S142. Perform velocity smoothing interpolation based on third-order polynomial on the filtered joint position data.

[0046] In this step, the filtered joint position data is subjected to velocity smoothing interpolation based on a third-order polynomial to limit the rate of change of joint acceleration to less than a preset acceleration threshold.

[0047] Specifically, within each interpolation window, the corresponding filtered initial joint position, initial velocity, and initial acceleration, as well as the termination position, termination velocity, and termination acceleration, can be obtained. Then, a third-order polynomial interpolation function is constructed that satisfies the boundary constraints of the initial position, initial velocity, initial acceleration, termination position, termination velocity, and termination acceleration. A smooth velocity curve is generated based on the third-order polynomial interpolation function, and the corresponding rate of change of acceleration is calculated. During the interpolation process, the rate of change of acceleration is limited to a preset acceleration threshold range, resulting in a smoothed joint position data sequence.

[0048] For S141-S142, an adaptive Kalman filter is used to estimate the process noise covariance and measurement noise covariance of the input joint target position sequence in real time. The Kalman gain is adjusted according to the real-time changes of these two parameters: when the measurement noise increases, the smoothing weight is increased to suppress noise; when the measurement noise decreases, the smoothing intensity is reduced to preserve motion details. The filtered output data then enters a third-order polynomial velocity smoothing interpolation module. This module generates a smooth velocity curve in each interpolation interval by constructing a third-order polynomial function that satisfies the initial velocity, termination velocity, and acceleration constraints, and generates an acceleration curve by calculating the derivative, ensuring that the rate of change of acceleration does not exceed a preset threshold. This two-stage processing structure combines state estimation theory with trajectory generation optimization, achieving a balance between noise suppression and dynamic response, thereby preventing sudden shocks at the execution end.

[0049] Furthermore, regarding S142 above, although smooth interpolation can effectively reduce high-frequency noise and abrupt changes in joint position commands, in practical applications, since the interpolation window length is usually greater than a single sampling period, it inevitably introduces a phase delay, causing the actual movement of the robot joints to lag behind the operator's real-time actions. In application scenarios requiring high response speed and accurate trajectory following (such as remote surgical robots, human-machine motion simulation, etc.), this delay will cause distortion of the operating feel and a decrease in control precision.

[0050] To address this, a second-order kinematic prediction model based on joint position, joint velocity, and joint acceleration can be constructed before velocity smoothing interpolation. This model can then predict the future target joint position based on a preset delay compensation time, where the delay compensation time corresponds to the phase delay introduced by the interpolation window length used in velocity smoothing interpolation.

[0051] Specifically, this can involve acquiring the most recent historical sampled values ​​of the joint position, along with the corresponding joint velocity and acceleration. Then, a second-order Taylor expansion is performed on the joint position using the joint velocity and acceleration to obtain the instantaneous predicted value of the future joint position. A preset delay compensation time is then input as the prediction time step into the second-order Taylor expansion calculation formula to generate the future joint target position corresponding to the preset delay compensation time.

[0052] By constructing a second-order kinematic prediction model based on joint position, joint velocity, and joint acceleration before smoothing interpolation, and combining this with a preset delay compensation time to predict the future target joint position, the phase delay generated in the interpolation calculation can be effectively offset. This allows the robot joints to maintain the smoothness of velocity and acceleration changes while eliminating motion feedback lag during capture actions. These steps effectively improve the synchronization of trajectory following in high-speed control tasks and enhance the consistency of operator motion perception in low-speed, fine-grained operation tasks.

[0053] Furthermore, in medical rehabilitation training applications, rehabilitation robots typically utilize motion capture devices to collect patients' joint motion data in real time and convert it into robot joint control commands. However, because patients' joint movement speeds are generally low during rehabilitation, and the amplitude of joint position changes approaches the noise threshold of the motion capture system, the signal-to-noise ratio of the collected data is significantly reduced. When this low-speed joint position data containing high-frequency noise directly enters the joint control loop, the velocity differential calculation further amplifies the noise, causing frequent micro-vibrations in the actuators. This not only affects control accuracy but may also cause discomfort and safety risks to the patient. Therefore, in medical rehabilitation training applications, directly smoothing the data would result in noise amplification under low-speed motion capture conditions.

[0054] To address this, before performing adaptive smoothing on the joint target position sequence, velocity amplitude detection can be performed on the joint target position sequence to determine whether a low-speed motion state has been entered. This velocity amplitude detection includes calculating the instantaneous joint angular velocity based on the joint position difference between consecutive sampling frames, and comparing the instantaneous joint angular velocity with a preset low-speed threshold. If the instantaneous joint angular velocity is lower than the preset low-speed threshold, a low-speed state is determined. When a low-speed motion state is determined, the filtering time window is extended to the smoothing time length corresponding to the low-speed threshold to reduce the interference of high-frequency noise components on the velocity differential calculation. Then, the joint position data after parameter adjustment and filtering is interpolated and reconstructed to avoid high-frequency jumps in joint position commands during low-speed states. The reconstructed smoothed joint control commands are then input to the joint drive controller, thereby effectively suppressing joint shaking caused by low-speed motion capture noise amplification in rehabilitation training scenarios.

[0055] S150: Input the smooth object control command to the drive controller of the target moving object.

[0056] In this step, smooth joint control commands can be input to the drive controller of the target robot joint to control the target robot joint to reduce unnecessary joint drive movements and reduce energy consumption when performing capture actions.

[0057] It's worth noting that, in actual execution, the aforementioned smoothing process inevitably introduces a certain computational delay, creating a time lag between joint execution and motion capture. Furthermore, when robot joints execute control commands, their mechanical characteristics (including inertia and frictional damping) further cause dynamic response lag, resulting in the actual joint trajectory falling behind the target trajectory. This lag is particularly significant in fast-paced or frequently changing motion scenarios, significantly impacting motion fidelity and operational synchronization. Increasing control gain to reduce lag can easily lead to oscillations and instability in the mechanical system, negatively impacting system safety.

[0058] To address this, a drive feedforward component based on the predicted joint velocity can be configured in the smooth joint control command to compensate for the dynamic response lag of the target robot joint caused by inertia and frictional damping. Specifically, the target drive torque corresponding to the predicted joint velocity can be determined based on a second-order kinematic prediction model. Then, the target drive torque is linearly superimposed with the joint position control quantity of the smooth joint control command to obtain a combined control command that includes the joint position control quantity and the drive feedforward component.

[0059] Furthermore, the determination of the target driving torque can be achieved by obtaining the instantaneous predicted angular velocity and instantaneous predicted angular acceleration of the target robot joint. Then, the inertial driving torque is calculated based on the equivalent moment of inertia parameter of the target robot joint. The frictional driving torque is then calculated based on the viscous friction coefficient of the target robot joint. Finally, the inertial driving torque and the frictional driving torque are weighted and summed to obtain the target driving torque.

[0060] Optionally, the weighting of the inertial driving torque can be determined by measuring the equivalent moment of inertia of the joint and the typical acceleration range corresponding to motion capture. The greater the contribution of the inertial torque in the actual joint driving response, the higher the weighting should be. For example, during high-speed, large-amplitude movements, the inertial component dominates, and the weighting of the inertial driving torque can be configured as 0.6-0.85. Correspondingly, the weighting of the friction driving torque can be configured as 0.15-0.4.

[0061] Optionally, the weighting of the frictional driving torque can be determined by operating the joint at low or constant speeds and measuring the relationship between the viscous friction coefficient and the output torque. If frictional damping has a significant impact on the system response hysteresis, the weighting value needs to be increased. For example, in slow, precise movements or high-damping mechanical structures, the weighting of the frictional driving torque can be configured as 0.5-0.8, and the corresponding weighting of the inertial driving torque can be configured as 0.2-0.5.

[0062] It is worth noting that in scenarios such as rehabilitation training or precision assembly, the weight of the friction component should be appropriately increased to avoid joint overshoot and vibration; while in scenarios of rapid following or high dynamic movements, the weight of the inertial component should be appropriately increased.

[0063] Optionally, the most recent historical sampling point data can be selected from the joint target position sequence obtained by coordinate system transformation from the motion capture device, and the angular position, angular velocity, and angular acceleration parameters of the corresponding joint can be extracted respectively. Then, the angular position, angular velocity, and angular acceleration are input into the second-order Taylor expansion formula, and the prediction time step is set to be equal to the delay compensation time to obtain the predicted value of the future joint target position. Next, the instantaneous predicted angular velocity of the joint is obtained by calculating the difference between the predicted value and the historical value, and the instantaneous predicted angular acceleration is calculated based on the rate of change of the predicted angular velocity within the prediction time step.

[0064] Optionally, the aforementioned equivalent moment of inertia is the combined inertial parameter of the target robot's joint links and the load directly connected to them, which can be obtained through offline calibration or online parameter identification, and is usually a fixed constant. The inertial driving torque can be the product of the equivalent moment of inertia parameter and the instantaneous predicted angular acceleration.

[0065] Optionally, the viscous friction coefficient is the damping coefficient generated by the target robot's joint drive unit and joint structure during uniform rotation, which can be measured by the friction compensation module of the drive controller or obtained by fitting experimental data. The friction driving torque can be the product of the viscous friction coefficient and the instantaneous predicted angular velocity.

[0066] Furthermore, regarding the linear superposition of the target driving torque and the joint position control quantity of the smooth joint control command, specifically, the joint position control quantity can be converted into a position loop output torque, and the target driving torque can be used as the velocity loop output torque. Then, the position loop output torque and the velocity loop output torque are linearly superimposed within the same joint control channel according to a preset weighting coefficient to obtain the combined control torque. The combined control torque is then converted into a combined control command and input to the drive controller of the target robot joint.

[0067] The determination of the position loop output torque can be achieved by calculating the difference between the joint position control quantity and the real-time position feedback value of the target robot joint to obtain the joint position error. Then, the corresponding elastic restoring torque is calculated based on the joint position error and the joint stiffness parameters. Finally, the elastic restoring torque is weighted and summed with the damping torque corresponding to the joint damping parameters to obtain the position loop output torque.

[0068] Optionally, the initial weights can be determined based on the measured results of the stiffness parameters and damping coefficients of the joint drive system. For high-stiffness, low-damping joint structures, the elastic restoring force has a more significant effect in reducing positional errors. In this case, the weighting coefficient of the elastic restoring torque can be configured as 0.6-0.85, while the weighting coefficient of the damping torque can be configured as 0.15-0.4. Conversely, in high-damping structures, to prevent vibration and overshoot, the weighting of the damping torque should be increased, and it can be configured as 0.5-0.8, with the corresponding weighting coefficient of the elastic restoring torque configured as 0.2-0.5.

[0069] Furthermore, it is worth noting that in tasks involving rapid following and frequent acceleration and deceleration, the proportion of elastic restoring torque should be increased to ensure positional accuracy; while in precise, slow movements or rehabilitation training, the proportion of damping torque should be increased to stabilize the trajectory.

[0070] The aforementioned position loop output torque reflects the relationship between joint position error and mechanical elastic stiffness and damping characteristics, maintaining joint position stability. The velocity loop output torque, calculated based on a second-order kinematic prediction model, is used to counteract inertia and frictional damping effects during joint movement. By introducing a weighted linear superposition mechanism into the control system, the two are fused in the same joint control channel according to preset weight coefficients, forming a combined control torque that simultaneously includes position correction and dynamic compensation functions, achieving synchronous optimization of position accuracy and dynamic response. This fusion mechanism enables joint control to maintain precise spatial positioning while rapidly responding to motion capture commands, effectively solving the problems of response lag and insufficient control present in the prior art during high-speed, high-precision motion reproduction.

[0071] Furthermore, by linearly superimposing the position loop output torque generated by the joint position control variables and the velocity loop output torque generated by the target driving torque to form a combined control torque, and then converting it into a combined control command input to the joint drive controller, the dynamic response characteristics of the robot joints during real-time motion capture can be improved. This not only reduces joint motion lag caused by inertia and damping, but also reduces output torque fluctuations, thereby effectively improving the accuracy of motion reconstruction and system operational stability, while optimizing energy consumption while maintaining accuracy.

[0072] It is worth noting that the aforementioned joint target position sequences are typically filtered to suppress noise and combined with predictive compensation methods to calculate future joint positions in advance, thereby reducing communication latency and control system response lag. However, if filtering and predictive compensation are directly connected in series or parallel within the same signal link, phase conflicts and trend distortions can easily occur. Specifically, the filter introduces additional time delay when removing high-frequency noise, while predictive compensation relies on the instantaneous trend of the signal to shift time forward. Since their actions are opposite, there is a risk of mutual cancellation or superposition, resulting in overshoot. Especially when motion capture data simultaneously contains low-frequency main movements and high-frequency detailed movements (including instantaneous acceleration changes), it is difficult to achieve smooth low-frequency predictive output while retaining effective high-frequency information, leading to problems such as trajectory deviation, control jitter, or inaccurate lead during robot joint execution.

[0073] To address this, in the step of constructing a second-order kinematic prediction model based on joint position, joint velocity, and joint acceleration to predict the future joint target position according to a preset delay compensation time, the joint position can be further decomposed into trend and detail components, so that the low-frequency trend component and the high-frequency detail component are processed independently. Smoothing filtering is applied to the low-frequency trend component, and the future joint target position is predicted based on the second-order kinematic prediction model. Adaptive noise thresholding is used to suppress noise in the high-frequency detail component, and forward prediction is performed based on the delay compensation time. The predicted low-frequency trend component and the forward-compensated high-frequency detail component are fused to generate fused joint target position data for velocity smoothing interpolation.

[0074] Optionally, the decomposition of the joint position into trend and detail components can be performed by performing time-domain decomposition on the joint position data to separate different frequency components. The low-frequency trend component corresponds to the main movement trajectory, while the high-frequency detail component corresponds to subtle movement changes and noise components. The low-frequency trend component and the high-frequency detail component are extracted using an equivalent frequency-domain decomposition method. Specifically, the main movement trajectory, such as a large upward or downward movement of the upper limb, has a low frequency, a large peak-to-trough span, and slow dynamic changes. Subtle movement changes, such as finger adjustments or joint end-effector vibrations, have a medium-to-high frequency and a small amplitude. System noise, such as sensor quantization errors and environmental interference, is characterized by high frequency, small amplitude, and irregularity.

[0075] Specifically, wavelet transform can be used to perform multi-scale decomposition on the joint position data to separate different frequency components in the time domain. The lowest frequency scale component obtained from the decomposition is taken as the low-frequency trend component, which corresponds to the main motion trajectory. The high-frequency scale components obtained from the decomposition are taken as high-frequency detail components, which correspond to subtle motion changes and noise components. The center frequency of the mother wavelet selected in the wavelet transform is in a preset ratio to the sampling frequency of the joint target position sequence to ensure a balance between the separated low-frequency trend component and high-frequency detail component in terms of time resolution and frequency resolution.

[0076] Optionally, fusing the predicted low-frequency trend component with the forward-compensated high-frequency detail component can be achieved by aligning the predicted low-frequency trend component and the delayed-compensated high-frequency detail component with timestamps in a unified joint control coordinate system. The low-frequency trend component and the high-frequency detail component are then superimposed according to a preset weighting coefficient to form the fused joint target position prediction data. This fused joint target position prediction data is input to the velocity smoothing interpolation module to reduce phase conflict between prediction compensation and filtering and improve joint trajectory control accuracy.

[0077] By decomposing the trend component and detail component as described above, smoothing filtering and future position prediction are performed on the low-frequency trend component, while adaptive noise threshold suppression and forward compensation are performed on the high-frequency detail component. Furthermore, time stamp alignment and weighted fusion are performed under a unified joint control coordinate system. This reduces the phase conflict between prediction compensation and filtering, thereby preserving the necessary instantaneous change data for high-frequency detail movements while maintaining the smoothness of the low-frequency main motion. This improves the accuracy and stability of joint trajectory control.

[0078] In summary, in this embodiment, the full-body posture data of the target operator collected by the motion capture device is acquired. Then, the full-body posture data is transformed into a coordinate system to obtain a joint target position sequence corresponding to the joint coordinate system of the target robot. Redundancy motion detection is performed on the joint target position sequence, and adaptive smoothing processing is performed on the joint target position sequence based on the result of the redundancy motion detection to obtain smooth joint control commands. The smooth joint control commands are then input to the drive controller of the target robot joint, thereby controlling the target robot joint to reduce unnecessary joint drive movements and reduce energy consumption when performing capture actions.

[0079] Figure 4 This is a schematic diagram illustrating the structure of a motion capture-based motion object control system according to an example embodiment of this application. Figure 4 As shown, the motion capture-based motion object control system 300 provided in this embodiment includes:

[0080] The acquisition module 310 is used to acquire the full-body posture data of the target operator collected by the motion capture device;

[0081] Processing module 320 is used to perform coordinate system transformation on the whole body posture data to obtain a joint target position sequence corresponding to the joint coordinate system of the target robot;

[0082] The processing module 320 is used to perform redundant motion detection on the joint target position sequence, and based on the result of the redundant motion detection, to perform adaptive smoothing processing on the joint target position sequence to obtain smooth joint control commands.

[0083] The control module 330 is used to input the smooth joint control command to the drive controller of the target robot joint.

[0084] Optionally, the processing module 320 is specifically used for:

[0085] Based on frequency domain analysis, high-frequency components above a preset cutoff frequency in the joint target position sequence are determined;

[0086] The amplitude threshold is used to determine whether the change in joint position corresponding to the high-frequency component is less than a preset joint micro-motion threshold.

[0087] Optionally, the processing module 320 is specifically used for:

[0088] The joint target position sequence is subjected to adaptive Kalman filtering to dynamically adjust the ratio of process noise to measurement noise based on real-time noise estimation.

[0089] The filtered joint position data is subjected to velocity smoothing interpolation based on a third-order polynomial to limit the rate of change of joint acceleration to less than a preset acceleration threshold.

[0090] Optionally, the processing module 320 is specifically used for:

[0091] A second-order kinematic prediction model based on joint position, joint velocity, and joint acceleration is constructed to predict the future target position of the joint according to a preset delay compensation time, wherein the delay compensation time corresponds to the phase delay introduced by the interpolation window length used in the velocity smoothing interpolation.

[0092] Optionally, the processing module 320 is specifically used for:

[0093] Obtain the most recent historical sampled value of the joint position and the corresponding joint velocity and joint acceleration;

[0094] The joint velocity and joint acceleration are subjected to a second-order Taylor expansion based on the joint position to obtain the instantaneous predicted value of the future joint position;

[0095] The preset delay compensation time is input as the prediction time step into the second-order Taylor expansion calculation formula to generate the future joint target position corresponding to the preset delay compensation time.

[0096] Optionally, the processing module 320 is specifically used for:

[0097] The smooth joint control command is configured with a drive feedforward component based on the predicted joint velocity to compensate for the dynamic response lag of the target robot joint caused by inertia and frictional damping.

[0098] Optionally, the processing module 320 is specifically used for:

[0099] The target driving torque corresponding to the predicted joint velocity is determined based on the second-order kinematic prediction model.

[0100] The target driving torque is linearly superimposed with the joint position control amount of the smooth joint control command to obtain a combined control command that includes the joint position control amount and the driving feedforward component.

[0101] Optionally, the processing module 320 is specifically used for:

[0102] Obtain the instantaneous predicted angular velocity and instantaneous predicted angular acceleration of the target robot joints;

[0103] The inertial driving torque is calculated based on the equivalent rotational inertia parameters of the target robot joints;

[0104] The frictional driving torque is calculated based on the viscous friction coefficient of the target robot joint;

[0105] The target driving torque is obtained by weighted summing of the inertial driving torque and the frictional driving torque.

[0106] Optionally, the processing module 320 is specifically used for:

[0107] The joint position control quantity is converted into a position loop output torque, and the target driving torque is used as the velocity loop output torque;

[0108] The output torque of the position loop and the output torque of the velocity loop are linearly superimposed in the same joint control channel according to a preset weighting coefficient to obtain a combined control torque.

[0109] The combined control torque is converted into the combined control command and input to the drive controller of the target robot joint.

[0110] Figure 5 This is a schematic diagram of the structure of an electronic device according to an example embodiment of this application. For example... Figure 5 As shown, the electronic device 400 provided in this embodiment includes: a processor 401 and a memory 402; wherein:

[0111] Memory 402 is used to store computer programs, and the memory may also be flash memory.

[0112] Processor 401 is used to execute the execution instructions stored in the memory to implement the various steps in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0113] Alternatively, the memory 402 can be either standalone or integrated with the processor 401.

[0114] When the memory 402 is a device independent of the processor 401, the electronic device 400 may further include:

[0115] Bus 403 is used to connect the memory 402 and the processor 401.

[0116] This embodiment also provides a readable storage medium storing a computer program, which, when executed by at least one processor of an electronic device, enables the electronic device to perform the methods provided in the various embodiments described above.

[0117] This embodiment also provides a program product including a computer program stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the methods provided in the various embodiments described above.

[0118] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0119] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for controlling a moving object based on motion capture, characterized in that, include: Acquire full-body posture data of the target operator collected by the motion capture device; The whole-body posture data is transformed to obtain the object target position sequence corresponding to the target moving object coordinate system; Redundant motion detection is performed on the target position sequence of the object, and adaptive smoothing is performed on the target position sequence of the object based on the result of the redundant motion detection to obtain a smooth object control command; The smooth object control command is input to the drive controller of the target moving object; The redundant motion detection of the target position sequence of the object includes: Based on frequency domain analysis, high-frequency components above a preset cutoff frequency in the target position sequence of the object are determined; Based on the amplitude threshold, determine whether the change amplitude of the object position corresponding to the high-frequency component is less than the preset object micro-motion threshold; The adaptive smoothing process for the target position sequence of the object includes: An adaptive Kalman filter is applied to the target position sequence of the object to dynamically adjust the ratio of process noise to measurement noise based on real-time noise estimation. The filtered object position data is subjected to velocity smoothing interpolation based on a third-order polynomial to limit the rate of change of object acceleration to be less than a preset acceleration threshold. Before performing velocity smoothing interpolation based on a third-order polynomial on the filtered object position data, the method further includes: A second-order kinematic prediction model based on object position, object velocity, and object acceleration is constructed to predict the future target position of the object according to a preset delay compensation time, wherein the delay compensation time corresponds to the phase delay introduced by the interpolation window length used in the velocity smoothing interpolation. The construction of a second-order kinematic prediction model based on object position, object velocity, and object acceleration to predict the future target position of the object according to a preset delay compensation time includes: Obtain the most recent historical sample value of the object's position, as well as the corresponding object velocity and object acceleration; The object velocity and object acceleration are subjected to a second-order Taylor expansion based on the object position to obtain the instantaneous prediction value of the future object position; The preset delay compensation time is input as the prediction time step into the second-order Taylor expansion calculation formula to generate the future target position of the object corresponding to the preset delay compensation time.

2. The motion object control method based on motion capture according to claim 1, characterized in that, The step of inputting the smooth object control command to the drive controller of the target moving object includes: The smooth object control command is configured with a drive feedforward component based on the predicted object velocity to compensate for the dynamic response lag of the target moving object caused by inertia and frictional damping.

3. The motion object control method based on motion capture according to claim 2, characterized in that, Configuring a drive feedforward component based on the predicted object velocity in the smooth object control command includes: The target driving torque corresponding to the velocity of the predicted object is determined based on the second-order kinematic prediction model. The target driving torque is linearly superimposed with the object position control quantity of the smooth object control command to obtain a combined control command that includes the object position control quantity and the driving feedforward component.

4. The motion object control method based on motion capture according to claim 3, characterized in that, Determining the target driving torque corresponding to the velocity of the predicted object based on the second-order kinematic prediction model includes: Obtain the instantaneous predicted angular velocity and instantaneous predicted angular acceleration of the target moving object; Calculate the inertial driving torque based on the equivalent rotational inertia parameters of the target moving object; The frictional driving torque is calculated based on the viscous friction coefficient of the target moving object; The target driving torque is obtained by weighted summing of the inertial driving torque and the frictional driving torque.

5. The motion object control method based on motion capture according to claim 3, characterized in that, The step of linearly superimposing the target driving torque with the object position control amount of the smooth object control command includes: The object position control quantity is converted into position loop output torque, and the target driving torque is used as velocity loop output torque; The output torque of the position loop and the output torque of the speed loop are linearly superimposed in the same object control channel according to a preset weighting coefficient to obtain the combined control torque. The combined control torque is converted into the combined control command and input to the drive controller of the target moving object.

6. A motion capture-based motion object control system, characterized in that, The system is used to execute the motion capture-based motion object control method as described in any one of claims 1-5; the system includes: The acquisition module is used to acquire the full-body posture data of the target operator collected by the motion capture device; The processing module is used to perform coordinate system transformation on the whole-body posture data to obtain the object target position sequence corresponding to the target moving object coordinate system; The processing module is used to perform redundant motion detection on the target position sequence of the object, and based on the result of the redundant motion detection, to perform adaptive smoothing processing on the target position sequence of the object to obtain a smoothed object control command; The control module is used to input the smooth object control command to the drive controller of the target moving object.

Images