Robot control method and electronic device

Abstract

The application provides a robot control method and an electronic device, wherein the method comprises: obtaining a current robot body state through a generative middleware, performing geometric residual parameterization on an original motion trajectory to obtain a linear interpolation baseline, and predicting and recovering residuals by using a flow matching model. According to the deviation between the current body state and the last control cycle correction trajectory, adaptive time scale planning is performed to dynamically determine the time domain length and the re-planning interval, and then the key frame trajectory is corrected in combination with the baseline, the residual and the time parameter and is executed by a physical tracking module. By introducing the adaptive time scale planning and the generative residual correction mechanism, the dynamic response speed, the motion fluency and the control robustness of the robot in a complex environment are significantly improved, and efficient collaborative control from high-level planning to low-level execution is realized.

Description

Technical Field

[0001] This application relates to the field of robotics, and more specifically, to a robot control method and an electronic device. Background Technology

[0002] Humanoid robots need to balance precise motion execution and disturbance resistance in real and complex environments. Precise tracking and flexible recovery are core requirements for general control. Biological motion can naturally integrate task execution and disturbance recovery, and robot control needs to reproduce this dual capability to adapt to unstructured scenarios.

[0003] Current robot control methods mainly fall into two categories: tracking and generative. Tracking methods rely on reinforcement learning to imitate reference trajectories, while generative methods use diffusion and large language models to synthesize natural motion. The hybrid architecture attempts to combine tracking and generative methods, fusing and optimizing generative priors with tracking targets.

[0004] However, pure tracking is prone to rigid failure when subjected to strong disturbances, while pure generation ignores physical constraints and is difficult to implement. Hybrid architectures are loosely coupled and rely on explicit switching. Fixed planning time domains cannot adapt to the dynamic requirements of tracking and recovery, making it difficult to simultaneously guarantee accuracy and robustness. Summary of the Invention

[0005] The purpose of this application is to provide a robot control method and electronic device to address the shortcomings of the prior art, thereby solving the problem that the prior art cannot simultaneously guarantee accuracy and robustness.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows: In a first aspect, this application provides a robot control method applied to a robot control system, the robot control system comprising: a motion input module, a generative middleware, and a physical tracking module, wherein the generative middleware is located between the motion input module and the physical tracking module; the method comprises: The motion input module receives the original motion command, determines the original motion trajectory based on the original motion command, and sends the original motion trajectory to the generative middleware; The generative middleware acquires the robot's current body state and performs geometric residual parameterization based on the current body state and the original motion trajectory to obtain a linear interpolation baseline. The flow matching model then predicts the recovery residual based on the current body state, the original motion trajectory, and the linear interpolation baseline. Adaptive time-scale planning is performed based on the deviation between the current body state and the corrected keyframe trajectory from the previous control cycle to determine the time domain length and replanning interval. The keyframe trajectory in the original motion trajectory is corrected based on the linear interpolation baseline, the recovery residual, the time domain length, and the replanning interval, and the corrected keyframe trajectory is written into the reference buffer corresponding to the physical tracking module. The physical tracking module acquires the robot's current body state and discrete motion embedding features, obtains the corrected keyframe trajectory from the reference buffer, generates the target joint position based on the corrected keyframe trajectory, the current body state, and the discrete motion embedding features, and controls the robot's movement based on the target joint position.

[0007] Optionally, the step of performing geometric residual parameterization based on the current body state and the original motion trajectory to obtain a linear interpolation baseline includes: The linear interpolation baseline is determined based on the current body state, the original motion trajectory, and the interpolation weight coefficients corresponding to each keyframe.

[0008] Optionally, the step of obtaining the recovered residual by the flow matching model based on the current ontology state, the original motion trajectory, and the linear interpolation baseline includes: Using the current body state and the original motion trajectory as the state condition vector, and the prior Gaussian noise as the initial state, the velocity field at each time step is predicted based on the flow matching model. The recovered residual is obtained by integrating the velocity field at each time step.

[0009] Optionally, the velocity field is parameterized using a Transformer architecture with adaptive layer normalization modulation; The state condition vector is embedded with the stream time step and injected into each module of the Transformer architecture through an adaptive shift-scaling-gated modulation method; During the integration of the vector field function along the time dimension, a repair constraint is applied, fixing the first residual label as follows: ,in, , The zero residual anchor point is the initial state of the robot. This represents the temporal interpolation weighting coefficients, used to balance the proportion of the initial anchor point and the random prior. The variable is a Gaussian random noise variable that follows a standard Gaussian distribution. Represents the identity matrix.

[0010] Optionally, the step of performing adaptive time-scale planning based on the deviation between the current body state and the corrected keyframe trajectory of the previous control cycle, and determining the time-domain length and replanning interval, includes: The tracking deviation between the current body state and the corrected keyframe trajectory of the previous control cycle is calculated in real time, and the tracking deviation is input into a preset smoothing gating function to obtain the weighting coefficient. The time domain length is obtained by dynamically interpolating the preset minimum duration and the preset maximum duration based on the weighting coefficients. The replanning interval is obtained by converting the temporal length based on the output frame rate and the number of keyframes.

[0011] Optionally, generating the corrected keyframe trajectory based on the linear interpolation baseline, the recovery residual, the temporal length, and the replanning interval includes: The recovered residual is superimposed on the linear interpolation baseline to obtain the spatially corrected compensated baseline trajectory. The trajectory planning interval of the compensated baseline trajectory is determined based on the time domain length and the replanning interval, and time scale matching is performed to obtain the corrected keyframe trajectory.

[0012] Optionally, writing the keyframe trajectory into the reference buffer corresponding to the physical tracking module includes: The keyframe trajectory is subjected to dense interpolation processing to obtain a continuous reference trajectory; The continuous reference trajectory is written into the reference buffer corresponding to the physical tracking module.

[0013] Optionally, the physical tracking module acquires the robot's current body state and discrete motion embedding features, including: Acquire the robot's current body state, periodic motion characteristics, motion patterns, and prior action information; The periodic motion characteristics, motion patterns, and action prior information of the robot are encoded to obtain the discrete motion embedding features.

[0014] Optionally, generating the target joint position based on the corrected keyframe trajectory, the current body state, and the discrete motion embedding features includes: The corrected keyframe trajectory is analyzed to obtain the desired trajectory; The trajectory tracking deviation is determined based on the current ontology state, and motion prior constraints are generated through discrete motion embedding features. The target joint position is obtained by performing inverse kinematics operations on the desired trajectory, the trajectory tracking deviation, and the motion prior constraints.

[0015] Secondly, this application provides an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the robot control method described above.

[0016] The beneficial effects of this application are: through a layered architecture and generative middleware, closed-loop control of original trajectory reception, adaptive state correction, and physical tracking execution can be achieved within the robot control system. Geometric residual parameterization and flow matching models can balance tracking accuracy and disturbance recovery capability. Adaptive time-scale planning dynamically balances response speed and recovery stability. After the corrected trajectory is transmitted through a buffer, the physical tracking module combines the body state and motion embedding to generate joint commands, enabling the robot to maintain high-precision tracking under normal working conditions and automatically generate a natural recovery trajectory when disturbed, improving motion robustness and anthropomorphism. At the same time, it achieves decoupling between high-level intent and low-level execution, thereby adapting to complex unstructured environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of the architecture of a robot control system provided in an embodiment of this application is shown; Figure 2 A flowchart of a robot control method provided in an embodiment of this application is shown; Figure 3 This application provides a flowchart for determining a recovery residual. Figure 4 This document illustrates a flowchart of a method for determining the time-domain length and replanning interval, as provided in an embodiment of this application. Figure 5 A flowchart illustrating a corrected keyframe trajectory provided in an embodiment of this application is shown; Figure 6 This document illustrates a flowchart of writing a modified keyframe trajectory according to an embodiment of this application. Figure 7This document illustrates a flowchart of an embodiment of obtaining discrete motion embedding features. Figure 8 This document illustrates a flowchart of an embodiment of the present application for obtaining the position of a target joint; Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0020] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0022] Current robot control technologies primarily employ a layered architecture, where the upper layer plans the motion trajectory, while the lower layer's physical controller is responsible for tracking and executing it. This approach relies on precise mathematical models and preset fixed time parameters to generate the trajectory. While generative middleware can introduce a degree of intelligence, it often lacks sufficient flexibility and adaptability when facing complex dynamic environments or model errors. Physical tracking modules typically perform simple error corrections based on fixed reference trajectories, making it difficult to handle significant trajectory deviations or sudden environmental changes.

[0023] In summary, the biggest problem with existing technologies is the lack of joint adaptive capabilities in the temporal and spatial domains. First, fixed timescale planning (such as fixed trajectory duration and replanning frequency) leads to rigid system responses: when the robot deviates from its trajectory, it cannot dynamically adjust how far it should look ahead or how often to recalculate, resulting in slow or oscillating recovery actions. Second, purely physical control lacks high-level semantic guidance, making it prone to getting trapped in local optima when encountering large deviations, and unable to perform global trajectory reconstruction like generative models. This causes the robot to either move stiffly or stop abruptly when facing strong interference or complex tasks, resulting in insufficient robustness and smoothness.

[0024] Based on this, this application proposes a robot control method applicable to a robot control system comprising a motion input module, generative middleware, and a physical tracking module. For example... Figure 1 The diagram illustrates the architecture of a robot control system. The motion input module receives reference motion commands from an upstream motion planning system, motion capture system, or teleoperation system. The generative middleware incorporates a GPU (Graphics Processing Unit) for accelerated computing and deploys a flow-matching-based state-condition generative middleware model. This model generates short-time-domain keyframe trajectories based on real-time proprioceptive states and reference motion commands, and performs dense interpolation processing. The physical tracking module deploys a general physical tracking strategy network (including a motion encoder, an improved finite scalar quantizer, a reconstruction decoder, and an action decoder). This network receives the densed keyframe trajectories and proprioceptive states and outputs the target joint positions. The physical tracking module can also connect to the underlying execution module, which contains controllers and drive motors for each joint. This execution module tracks the target joint positions output by the physical tracking module at a high frequency, generating physical torques to drive the robot's motion.

[0025] This application obtains the robot's current body state through generative middleware, performs geometric residual parameterization on the original motion trajectory to obtain a linear interpolation baseline, and uses a flow matching model to predict and recover the residual. It can correct the deviation between the current body state and the trajectory from the previous control cycle by performing adaptive time-scale planning to dynamically determine the time domain length and replanning interval. Then, it combines the baseline, residual, and time parameters to correct the keyframe trajectory and hand it over to the physical tracking module for execution. This effectively solves the technical problems of existing robot control technologies lacking joint temporal and spatial adaptive capabilities in dynamic environments, and exhibiting rigid responses and insufficient robustness when facing trajectory deviations. By introducing adaptive time-scale planning and generative residual correction mechanisms, it significantly improves the robot's dynamic response speed, motion smoothness, and control robustness in complex environments, achieving efficient collaborative control from high-level planning to low-level execution.

[0026] Next, combine Figure 2The robot control method of this application is described below, and the method is applied to... Figure 1 The robot control system shown includes a motion input module, generative middleware, and a physical tracking module. The generative middleware is located between the motion input module and the physical tracking module. The generative middleware can be deployed on the robot's local high-performance computing unit or implemented through cloud-based collaborative computing. Figure 2 As shown, the method includes: S201. The motion input module receives the original motion command, determines the original motion trajectory based on the original motion command, and sends the original motion trajectory to the generative middleware.

[0027] Optionally, the original motion command can be a target motion requirement from upstream systems such as motion planning, motion capture, and remote control, including information such as joint positions, base velocities, and attitude targets. The original motion trajectory refers to the initial motion path obtained by parsing the original motion command without undergoing state adaptive correction; it can be a position sequence in Cartesian space or an angular displacement sequence in joint space.

[0028] The motion input module can parse the input raw motion commands and generate raw motion trajectories containing keyframes, postures, and velocities according to the time sequence. The generation of raw motion trajectories can be achieved through motion capture data parsing, preset command library mapping, and remote control signal conversion.

[0029] S202. The generative middleware obtains the robot's current body state and performs geometric residual parameterization based on the current body state and the original motion trajectory to obtain a linear interpolation baseline. The flow matching model predicts the recovery residual based on the current body state, the original motion trajectory, and the linear interpolation baseline. Based on the deviation between the current body state and the corrected keyframe trajectory of the previous control cycle, adaptive time-scale planning is performed to determine the time domain length and replanning interval. The keyframe trajectory in the original motion trajectory is corrected based on the linear interpolation baseline, the recovery residual, the time domain length, and the replanning interval. The corrected keyframe trajectory is written into the reference buffer corresponding to the physical tracking module.

[0030] Optionally, the current body state can be the robot's real-time perceived state, including base posture, angular velocity, joint position, joint velocity, gravity projection vector, etc. The linear interpolation baseline refers to the base trajectory obtained by linear interpolation using the current body state and the original trajectory as endpoints, according to the keyframe weights.

[0031] Geometric residual parameterization is a trajectory representation method that does not directly predict the absolute trajectory, but instead calculates the deviation relative to the baseline, simplifying model learning and preserving tracking accuracy.

[0032] Flow matching models are generative models based on continuous probabilistic flows. They learn a mapping from noise to target residuals to generate stable, natural recovered trajectories. The recovered residual refers to the amount of deviation superimposed on the linear interpolation baseline to correct the trajectory for balanced recovery. Flow matching models can use the ontological state and the original trajectory as conditions to learn physically feasible recovery deviations and output effective residuals under perturbation.

[0033] Tracking deviation refers to the spatial and attitude differences between the current state of the trajectory and the trajectory corrected in the previous cycle. Adaptive time-scale planning is a mechanism that dynamically adjusts the trajectory planning duration and update frequency according to the magnitude of the deviation. Temporal length refers to the time range covered by a single trajectory planning iteration. Replanning interval refers to the number of cycle steps for the generative middleware to update the trajectory.

[0034] Small deviations allow for a shorter time domain and increased update frequency to maintain tracking accuracy; large deviations allow for a longer time domain and sufficient recovery time, balancing response and stability. Therefore, real-time tracking deviations can be calculated, and the time domain length and replanning interval can be determined, thereby achieving dynamic time-scale planning.

[0035] The corrected keyframe trajectory refers to the executable keyframe path after spatial residual compensation and temporal scale matching. The reference buffer is a data buffer that stores the trajectory to be executed and is read by the physical tracking module.

[0036] Optionally, the generative middleware can spatially superimpose the residuals to correct the baseline trajectory and temporally adapt it to the replanning interval in the time domain to generate a physically feasible trajectory as the corrected keyframe trajectory.

[0037] S203 The physical tracking module obtains the robot's current body state and discrete motion embedding features, obtains the corrected keyframe trajectory from the reference buffer, and generates the target joint position based on the corrected keyframe trajectory, the current body state and discrete motion embedding features, and controls the robot's movement based on the target joint position.

[0038] Discrete motion embedding features refer to the compact semantic features obtained by encoding periodic motion, patterns, and priors, which are used to constrain motion execution. The physical tracking module can fuse ontology state, motion priors, and corrected trajectories to obtain discrete motion embedding features, thereby providing complete input for control command generation.

[0039] The target joint position refers to the target angle and target position that each joint of the robot needs to reach, which is the direct instruction executed at the lower level.

[0040] Optionally, the trajectory can be parsed to obtain the desired path, the tracking deviation can be calculated, discrete motion embedding features can be used as constraints, the target joint position can be obtained through inverse kinematics, and then the target joint position can be sent to the underlying execution module, which will drive the motor to execute the movement.

[0041] In this embodiment, a closed-loop control system encompassing original trajectory reception, adaptive state correction, and physical tracking execution can be achieved within the robot control system through a layered architecture and generative middleware. Geometric residual parameterization and flow matching models balance tracking accuracy and disturbance recovery capability. Adaptive time-scale planning dynamically balances response speed and recovery stability. After the corrected trajectory is transmitted via a buffer, the physical tracking module combines the robot's state and motion embedding to generate joint commands, enabling the robot to maintain high-precision tracking under normal operating conditions and automatically generate a naturally recoverable trajectory when disturbed, thus improving motion robustness and anthropomorphism. Simultaneously, it decouples high-level intent from low-level execution, thereby adapting to complex unstructured environments.

[0042] The following is a further explanation of the linear interpolation baseline obtained by parameterizing the geometric residuals based on the current ontology state and the original motion trajectory, including: The linear interpolation baseline is determined based on the current body state, the original motion trajectory, and the interpolation weight coefficients corresponding to each keyframe.

[0043] Optionally, the current body state and the original motion trajectory can be used as two endpoints, and a linear weighted sum can be performed on each keyframe with a fixed weight to obtain a basic trajectory that smoothly transitions from the current state to the original command. This baseline serves as a residual prediction benchmark. During normal tracking, the residual approaches zero, directly maintaining high-precision tracking. During disturbances, the residual is superimposed to achieve balance recovery.

[0044] In one possible implementation, the linear interpolation baseline can be calculated using the following equation (1).

[0045] (1) in, This represents the interpolation weight coefficient of the k-th keyframe, used to control the interpolation ratio between the current body state and the original motion trajectory of the current keyframe. It refers to the linear interpolation baseline state at the t-th control time and the k-th keyframe, which serves as the reference trajectory for the middleware to predict the recovery residual. It is the robot's real-time proprioceptive state at time t. It refers to the original reference motion command / trajectory at time t. Indicates the current ontology state The weights on the linear interpolation baseline, where k represents the keyframe number and K represents the total number of keyframes within a trajectory, are used to determine the number of interpolation segments and control the smoothness of the linear interpolation baseline.

[0046] when When =0 (frame 0), = The linear interpolation baseline is equal to the robot's current state, when When =1 (last frame), = The linear interpolation baseline is equal to the original motion trajectory.

[0047] In this embodiment, the baseline trajectory parameterized by geometric residuals is obtained by linear interpolation using the current body state, the original motion trajectory and the key frame interpolation weight coefficients. This provides a stable benchmark for subsequent residual prediction, ensuring approximate identity mapping under normal working conditions without loss of tracking accuracy, while also reserving space for superimposing and recovering residuals during disturbances. This simplifies the learning difficulty of generative middleware and improves trajectory correction efficiency and physical feasibility.

[0048] The following is a further explanation of the recovery residuals predicted by the flow matching model based on the current ontology state, the original motion trajectory, and the linear interpolation baseline. Figure 3 As shown, the above step S202 includes: S301. Using the current body state and the original motion trajectory as the state condition vector, and the prior Gaussian noise as the initial state, the velocity field at each time step is predicted based on the flow matching model.

[0049] Here, the state condition vector is a conditional input vector formed by concatenating the robot's current body state and the original motion trajectory. It is used to enable the flow matching model to generate an adaptive recovery residual based on the real-time state. Prior Gaussian noise refers to an initial random signal that follows a standard Gaussian distribution, serving as the starting point for the flow matching model to generate the recovery residual. The velocity field is a continuous vector field learned by the flow matching model, representing the direction and rate of change of the residual in the time dimension.

[0050] The flow matching model uses a continuous flow matching method to synthesize the multimodal recovery residual distribution. The probabilistic path is defined as follows (2): (2) in, , This represents the normalized truth residual. denoted as prior Gaussian noise, and t represents the flow matching time step.

[0051] The velocity field learning objective of the flow matching model is defined as follows (3): (3) in, Represents the state condition vector. Indicates the current state of the entity. Represents the original trajectory. This represents the velocity field predicted by the model.

[0052] S302. Integrate the velocity field at each time step to obtain the recovered residual.

[0053] During the model inference phase, the velocity field can be integraled from t=1 to t=0 to obtain the final recovery residual.

[0054] In this embodiment, the velocity field is predicted by the flow matching model and integrated to obtain the recovery residual, based on the current body state and the original motion trajectory as conditions and Gaussian noise as the starting point. This makes the residual generation have real-time state adaptability and physical feasibility, and can automatically output a reasonable compensation amount when the robot deviates from the reference trajectory. This does not destroy the tracking accuracy under normal working conditions, and can provide human-like and stable recovery correction when disturbed, thereby improving control robustness and motion naturalness.

[0055] Optionally, the velocity field is parameterized using a Transformer architecture with adaptive layer normalization modulation; The state condition vector is embedded with the stream time step and injected into each module of the Transformer architecture through an adaptive shift-scaling-gated modulation method; During the integration of the vector field function along the time dimension, a repair constraint is applied to fix the first residual label as follows (4): (4) in, , The zero residual anchor point is the initial state of the robot. This represents the temporal interpolation weighting coefficients, used to balance the proportion of the initial anchor point and the random prior. The variable is a Gaussian random noise variable that follows a standard Gaussian distribution. Represents the identity matrix.

[0056] Optionally, the first residual marker refers to the residual corresponding to the first keyframe of the trajectory, serving as the starting point for connecting to the current state. Adaptive shift-scaling-gated modulation refers to injecting conditional information and time embedding into each Transformer layer through shifting, scaling, and gating to achieve strong conditional constraints.

[0057] This application embodiment achieves velocity field parameterization through adaptive layer normalization modulation of the Transformer, and injects conditional and temporal information using a shift-scaling-gating method to improve the velocity field prediction accuracy and conditional fitting capability. During integration, a repair constraint is applied and the first residual marker is fixed to ensure that the generated trajectory is kinematically continuous with the robot's current state, without jumps or instability, while preserving the diversity of recovery actions. This ensures both the stability of high-precision tracking and makes the disturbance recovery trajectory more natural and physically feasible.

[0058] The following is a further explanation of the adaptive time-scale planning used to determine the temporal length and replanning interval for the deviation between the current ontology state and the corrected keyframe trajectory from the previous control cycle. Figure 4 As shown, the above step S202 includes: S401. Calculate the tracking deviation between the current body state and the corrected keyframe trajectory of the previous control cycle in real time, and input the tracking deviation into a preset smoothing gating function to obtain the weight coefficient.

[0059] Tracking deviation refers to the state difference between the current body state and the corrected keyframe trajectory output in the previous control cycle. A smoothing gating function is a function whose output changes continuously and smoothly between 0 and 1. It is used to smoothly map the tracking deviation into time-domain adjustment weights, obtaining weight coefficients and avoiding abrupt transitions during switching.

[0060] In one possible implementation, the process of calculating the weight coefficients of the smoothing gating function can be expressed as the following equation (5): (5) in, Indicates the weighting coefficient. Indicates the gating steepness coefficient. Indicates the deviation threshold. This indicates tracking deviation.

[0061] S402. Based on the weighting coefficients, perform dynamic interpolation calculations on the preset minimum duration and the preset maximum duration to obtain the time domain length.

[0062] Optionally, the time domain length can be calculated using the following formula (6).

[0063] (6) in, For the preset minimum length, The preset maximum duration, Indicates the length of the time domain.

[0064] In this process, in order to prevent the composite delay generated by the positive feedback loop in the cascading fault, the upper bound of the gradual expansion rate is constrained, as expressed in the following equation (7).

[0065]

[0066] in, This represents the time-domain length of the current nth replanning iteration. This represents the time-domain length of the last (n-1) replanning iteration. This indicates the maximum allowable increment in the time domain length for a single operation.

[0067] S403. The temporal length is converted based on the output frame rate and the number of keyframes to obtain the replanning interval.

[0068] Output frame rate refers to the frequency at which the corrected keyframe trajectory, after densification, is output to the physical tracking module. Keyframe number refers to the total number of trajectory keyframes in a single planning iteration.

[0069] If the tracking deviation is small, the time domain should be shortened and the update frequency increased to ensure accurate tracking. If the tracking deviation is large, the time domain should be extended and the update frequency decreased to provide sufficient recovery time. This ensures a smooth transition throughout the process, without rigid switching or motion jitter.

[0070] Optionally, the replanning interval can be obtained by conversion using the following formula (8). .

[0071] (8) in, This represents the output frame rate, and K represents the number of keyframes. This means the result is limited to between 2 and 10.

[0072] In this embodiment, the weight coefficients are dynamically calculated using the tracking deviation and smoothing gating function, and the time domain length is obtained through adaptive interpolation. The replanning interval is then calculated based on the time domain length, frame rate, and number of keyframes to achieve automatic adaptation of the tracking and recovery time scale. When the deviation is small, short-time-domain high-frequency replanning is used to ensure motion accuracy, while when the deviation is large, long-time-domain low-frequency replanning is used to provide a stable recovery margin. The entire process is smooth without rigid switching, which not only improves the response speed of accurate tracking but also enhances the balance stability and motion continuity under large disturbances.

[0073] The following is a further explanation of the modified keyframe trajectory generated above based on the linear interpolation baseline, recovered residual, temporal length, and replanning interval, such as... Figure 5 As shown, the above step S202 includes: S501. The recovered residual is superimposed onto the linear interpolation baseline to obtain the spatially corrected compensated baseline trajectory.

[0074] Optionally, the recovery residuals can be superimposed onto the linear interpolation baseline using the following equation (9).

[0075] (9) in, Indicates the baseline trajectory after compensation. Indicates the linear interpolation baseline. This indicates the restoration of the residual.

[0076] S502. Determine the trajectory planning interval of the compensated baseline trajectory based on the time domain length and replanning interval, and perform time scale matching to obtain the corrected keyframe trajectory.

[0077] Optionally, the time range covered by the trajectory can be determined by the time domain length, the trajectory update rhythm can be determined by the replanning interval, and the compensated trajectory can be resampled and aligned according to the time axis to obtain the final corrected keyframe trajectory.

[0078] In this embodiment, the recovered residual is first superimposed on the linear interpolation baseline to complete spatial correction, and then the time scale is matched by the time domain length and the replanning interval to finally generate the corrected keyframe trajectory. Under normal operating conditions, the residual approaches zero, and the trajectory maintains high-precision tracking; when disturbed, reasonable residuals are automatically superimposed and adapted to the time scale, so that the trajectory has both motion accuracy and recovery stability, with no rigid switching throughout the process and continuous and natural movements, improving the robot's robustness and anthropomorphism in complex environments.

[0079] The following is a further explanation of writing the keyframe trajectory into the reference buffer corresponding to the physical tracking module, such as... Figure 6 As shown, the above step S202 includes: S601. Perform dense interpolation processing on the corrected keyframe trajectory to obtain a continuous reference trajectory.

[0080] Optionally, linear interpolation, polynomial interpolation, third-order interpolation, or spline interpolation can be performed between keyframes to generate and fill intermediate attitude points, converting discrete keyframes into high-frequency continuous trajectories and obtaining continuous reference trajectories.

[0081] S602. Write the continuous reference trajectory into the reference buffer corresponding to the physical tracking module.

[0082] After dense interpolation, the trajectory can be smoothed to remove the jitter caused by interpolation, and the continuous reference trajectory can be written into the reference buffer corresponding to the physical tracking module.

[0083] This application embodiment converts sparse keyframe trajectories into high-frequency continuous reference trajectories through dense interpolation, and then writes them into the reference buffer. This enables the planning layer and execution layer to match frequencies and connect data smoothly, ensuring that the robot's motion is continuous and smooth, without jumps or stutters. At the same time, it reduces the amount of middleware computation and improves the system's real-time performance and operational stability.

[0084] The following is a further explanation of how the physical tracking module acquires the robot's current body state and discrete motion embedding features, such as... Figure 7 As shown, step S203 above includes: S701. Obtain the robot's current body state, periodic motion characteristics, motion patterns, and prior action information.

[0085] Periodic motion characteristics refer to the temporal periodicity, gait phase, rhythm, and other features extracted from repetitive motion patterns. Prior motion information refers to stable, natural, and physically constrained knowledge of motion patterns and posture distributions learned from large amounts of motion data.

[0086] S702. Encode the robot's periodic motion characteristics, motion patterns, and prior action information to obtain discrete motion embedding features.

[0087] Optionally, complex, high-dimensional, and continuous motion information can be compressed into compact, stable, and discrete embedded features, allowing the physical tracking module to quickly identify the current motion type, rhythm, and constraints, thereby improving tracking stability and human-likeness of the movements.

[0088] The following is a further explanation of how the target joint position is generated based on the corrected keyframe trajectory, the current ontology state, and discrete motion embedding features. Figure 8 As shown, step S203 above includes: S801. Analyze the corrected keyframe trajectory to obtain the desired trajectory.

[0089] The desired trajectory refers to the motion path of the base and limb targets that the robot needs to track, obtained from the keyframe trajectory. Information such as the base linear velocity, angular velocity, target joint position, reference height, and anchor point orientation can be extracted from the corrected keyframe trajectory to obtain the desired trajectory.

[0090] S802. Determine the trajectory tracking deviation based on the current ontology state, and generate motion prior constraints through discrete motion embedding features.

[0091] Optionally, the current body state (joint position, base posture, velocity) can be subtracted from the desired trajectory to obtain the trajectory tracking deviation, which includes position deviation, orientation error, and velocity error. The discrete motion embedding features then provide the posture range, torque limits, and motion rhythm constraints corresponding to the current action type.

[0092] S803. Perform inverse kinematics calculations on the desired trajectory, trajectory tracking deviation, and motion prior constraints to obtain the target joint position.

[0093] Optionally, the desired trajectory, tracking deviation, and prior constraints can be integrated to solve for the optimal joint angle that satisfies the following conditions: minimum tracking error, conformity to motion priors, joints not exceeding limits, and dynamic feasibility, thereby obtaining the target joint position.

[0094] In this embodiment, the desired path is obtained by parsing the keyframe trajectory, the tracking deviation is calculated by combining the ontology state, the prior constraints are provided by discrete motion embedding, and the target joint position is obtained by inverse kinematics solution. This achieves the fusion control of high-level trajectory, real-time state, and motion prior, which not only ensures tracking accuracy but also enhances disturbance stability and motion naturalness. It can also output safe, smooth, and directly executable joint commands.

[0095] Based on the same inventive concept, this application also provides a robot control system corresponding to the robot control method. Since the principle of the system in this application is similar to the robot control method described above, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be described again.

[0096] Reference Figure 1 The diagram shown is a schematic of a robot control system provided in an embodiment of this application. The system includes: a motion input module, a generative middleware, and a physical tracking module.

[0097] The motion input module receives the original motion command, determines the original motion trajectory based on the original motion command, and sends the original motion trajectory to the generative middleware. The generative middleware acquires the robot's current body state and performs geometric residual parameterization based on the current body state and the original motion trajectory to obtain a linear interpolation baseline. The flow matching model then predicts the recovery residual based on the current body state, the original motion trajectory, and the linear interpolation baseline. Based on the deviation between the current body state and the corrected keyframe trajectory from the previous control cycle, adaptive time-scale planning is performed to determine the time domain length and replanning interval. The keyframe trajectory in the original motion trajectory is corrected based on the linear interpolation baseline, the recovery residual, the time domain length, and the replanning interval, and the corrected keyframe trajectory is written into the reference buffer corresponding to the physical tracking module. The physical tracking module acquires the robot's current body state and discrete motion embedding features, obtains the corrected keyframe trajectory from the reference buffer, and generates the target joint position based on the corrected keyframe trajectory, the current body state, and the discrete motion embedding features, and controls the robot's movement based on the target joint position.

[0098] Optionally, generative middleware is specifically used for: The linear interpolation baseline is determined based on the current body state, the original motion trajectory, and the interpolation weight coefficients corresponding to each keyframe.

[0099] Optionally, generative middleware is specifically used for: Using the current body state and the original motion trajectory as the state condition vector and the prior Gaussian noise as the initial state, the velocity field at each time step is predicted based on the flow matching model. Integrating the velocity field at each time step yields the recovered residual.

[0100] Optionally, the velocity field is parameterized using a Transformer architecture with adaptive layer normalization modulation; The state condition vector is embedded with the stream time step and injected into each module of the Transformer architecture through an adaptive shift-scaling-gated modulation method; During the integration of the vector field function along the time dimension, a repair constraint is applied, fixing the first residual label as follows: ,in, , The zero residual anchor point is the initial state of the robot. This represents the temporal interpolation weighting coefficients, used to balance the proportion of the initial anchor point and the random prior. The variable is a Gaussian random noise variable that follows a standard Gaussian distribution. Represents the identity matrix.

[0101] Optionally, generative middleware is specifically used for: The tracking deviation between the current body state and the corrected keyframe trajectory of the previous control cycle is calculated in real time. The tracking deviation is input into a preset smoothing gating function to obtain the weighting coefficient. The time domain length is obtained by dynamically interpolating the preset minimum and maximum durations based on the weighting coefficients. The temporal length is converted based on the output frame rate and the number of keyframes to obtain the replanning interval.

[0102] Optionally, generative middleware is specifically used for: The recovered residuals are superimposed onto the linear interpolation baseline to obtain the spatially corrected compensated baseline trajectory. The trajectory planning interval of the compensated baseline trajectory is determined based on the time domain length and the replanning interval, and time scale matching is performed to obtain the corrected keyframe trajectory.

[0103] Optionally, generative middleware is specifically used for: Dense interpolation is performed on the keyframe trajectory to obtain a continuous reference trajectory; Write the continuous reference trajectory into the reference buffer corresponding to the physical tracking module.

[0104] Optionally, the physical tracking module is specifically used for: Acquire the robot's current body state, periodic motion characteristics, motion patterns, and prior action information; The periodic motion characteristics, motion patterns, and prior action information of the robot are encoded to obtain discrete motion embedding features.

[0105] Optionally, the physical tracking module is specifically used for: The corrected keyframe trajectory is analyzed to obtain the desired trajectory; The trajectory tracking deviation is determined based on the current ontology state, and motion prior constraints are generated through discrete motion embedding features. The inverse kinematics operation is performed on the desired trajectory, trajectory tracking deviation, and motion prior constraints to obtain the target joint position.

[0106] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0107] This application's embodiments, through a layered architecture and generative middleware, enable closed-loop control within the robot control system, encompassing original trajectory reception, adaptive state correction, and physical tracking execution. Geometric residual parameterization and flow matching models balance tracking accuracy and disturbance recovery capability. Adaptive time-scale planning dynamically balances response speed and recovery stability. After the corrected trajectory is transmitted via a buffer, the physical tracking module combines the robot's state and motion embedding to generate joint commands. This allows the robot to maintain high-precision tracking under normal operating conditions and automatically generate a naturally recoverable trajectory when disturbed, improving motion robustness and anthropomorphism. Simultaneously, it decouples high-level intent from low-level execution, thereby adapting to complex unstructured environments.

[0108] This application also provides an electronic device, such as... Figure 9 The diagram shown is a schematic representation of an electronic device structure provided in an embodiment of this application, including a processor 901, a memory 902, and a bus 903. The memory 902 stores machine-readable instructions executable by the processor 901. When the computer device is running, the processor 901 and the memory 902 communicate via the bus. When the machine-readable instructions are executed by the processor 901, the aforementioned robot control method is performed.

[0109] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the robot control method described above.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0111] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0112] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A robot control method, characterized in that, An application is made in a robot control system, the robot control system comprising: a motion input module, a generative middleware, and a physical tracking module, wherein the generative middleware is located between the motion input module and the physical tracking module; the method comprises: The motion input module receives the original motion command, determines the original motion trajectory based on the original motion command, and sends the original motion trajectory to the generative middleware; The generative middleware acquires the robot's current body state and performs geometric residual parameterization based on the current body state and the original motion trajectory to obtain a linear interpolation baseline. The flow matching model then predicts the recovery residual based on the current body state, the original motion trajectory, and the linear interpolation baseline. Adaptive time-scale planning is performed based on the deviation between the current body state and the corrected keyframe trajectory from the previous control cycle to determine the time domain length and replanning interval. The keyframe trajectory in the original motion trajectory is corrected based on the linear interpolation baseline, the recovery residual, the time domain length, and the replanning interval, and the corrected keyframe trajectory is written into the reference buffer corresponding to the physical tracking module. The physical tracking module acquires the robot's current body state and discrete motion embedding features, obtains the corrected keyframe trajectory from the reference buffer, generates the target joint position based on the corrected keyframe trajectory, the current body state, and the discrete motion embedding features, and controls the robot's movement based on the target joint position.

2. The method according to claim 1, characterized in that, The step of performing geometric residual parameterization based on the current body state and the original motion trajectory to obtain a linear interpolation baseline includes: The linear interpolation baseline is determined based on the current body state, the original motion trajectory, and the interpolation weight coefficients corresponding to each keyframe.

3. The method according to claim 1, characterized in that, The process of obtaining the recovered residuals by the flow matching model based on the current ontology state, the original motion trajectory, and the linear interpolation baseline includes: Using the current body state and the original motion trajectory as the state condition vector, and the prior Gaussian noise as the initial state, the velocity field at each time step is predicted based on the flow matching model. The recovered residual is obtained by integrating the velocity field at each time step.

4. The method according to claim 3, characterized in that, The velocity field is parameterized using an adaptive layer normalized modulation Transformer architecture; The state condition vector is embedded with the stream time step and injected into each module of the Transformer architecture through an adaptive shift-scaling-gated modulation method; During the integration of the vector field function along the time dimension, a repair constraint is applied, fixing the first residual label as follows: in, , The zero residual anchor point is the initial state of the robot. This represents the temporal interpolation weighting coefficients, used to balance the proportion of the initial anchor point and the random prior. The variable is a Gaussian random noise variable that follows a standard Gaussian distribution. Represents the identity matrix.

5. The method according to claim 1, characterized in that, The adaptive time-scale planning based on the deviation between the current body state and the corrected keyframe trajectory of the previous control cycle, determining the time domain length and replanning interval, includes: The tracking deviation between the current body state and the corrected keyframe trajectory of the previous control cycle is calculated in real time, and the tracking deviation is input into a preset smoothing gating function to obtain the weighting coefficient. The time domain length is obtained by dynamically interpolating the preset minimum duration and the preset maximum duration based on the weighting coefficients. The replanning interval is obtained by converting the temporal length based on the output frame rate and the number of keyframes.

6. The method according to claim 1, characterized in that, The step of generating the corrected keyframe trajectory based on the linear interpolation baseline, the recovered residual, the temporal length, and the replanning interval includes: The recovered residual is superimposed on the linear interpolation baseline to obtain the spatially corrected compensated baseline trajectory. The trajectory planning interval of the compensated baseline trajectory is determined based on the time domain length and the replanning interval, and time scale matching is performed to obtain the corrected keyframe trajectory.

7. The method according to claim 1, characterized in that, The step of writing the corrected keyframe trajectory into the reference buffer corresponding to the physical tracking module includes: The corrected keyframe trajectory is subjected to dense interpolation processing to obtain a continuous reference trajectory; The continuous reference trajectory is written into the reference buffer corresponding to the physical tracking module.

8. The method according to claim 1, characterized in that, The physical tracking module acquires the robot's current body state and discrete motion embedding features, including: Acquire the robot's current body state, periodic motion characteristics, motion patterns, and prior action information; The periodic motion characteristics, motion patterns, and action prior information of the robot are encoded to obtain the discrete motion embedding features.

9. The method according to claim 1, characterized in that, The step of generating the target joint position based on the corrected keyframe trajectory, the current body state, and the discrete motion embedding features includes: The corrected keyframe trajectory is analyzed to obtain the desired trajectory; The trajectory tracking deviation is determined based on the current ontology state, and motion prior constraints are generated through discrete motion embedding features. The target joint position is obtained by performing inverse kinematics operations on the desired trajectory, the trajectory tracking deviation, and the motion prior constraints.

10. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the robot control method as described in any one of claims 1-9.

Images