Motion tracking method, apparatus, and electronic device

Abstract

The application provides a motion tracking method and device and electronic equipment, and relates to the technical field of robots. The method comprises the following steps: determining a motion increment for performing a tracking action based on a tracking action and a motion parameter of a target robot through a trained motion strategy, wherein the motion strategy is obtained by training based on distribution perception and failure-priority resampling; and controlling the target robot to perform motion tracking based on the motion increment and the tracking action. Through the distribution perception and failure-priority resampling, the motion strategy can converge faster, the training efficiency is improved, the drift and cumulative error are reduced by combining the motion increment with the focus on dynamics compensation, and more stable long-term high-dynamic tracking can be achieved.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to a motion tracking method, apparatus, and electronic device. Background Technology

[0002] In existing technologies, robot motion imitation or tracking mostly rely on direct prediction of absolute joint commands, ignoring the systematic differences between the robot's actual dynamics and the reference motion. This leads to problems such as long-term drift and error accumulation, long data tails and inefficient learning, limitations of phase scalars, and gaps between simulation and reality. Therefore, an effective solution is urgently needed to address at least one of these problems. Summary of the Invention

[0003] To address the above problems, the present invention provides a motion tracking method, apparatus, and electronic device.

[0004] This invention provides a motion tracking method, comprising:

[0005] Using a trained motion strategy, the motion increment for performing the tracking action is determined based on the tracking action and the motion parameters of the target robot. The motion strategy is trained based on distributed perception and failure-first resampling.

[0006] Based on the motion increment and the tracking action, the target robot is controlled to perform motion tracking.

[0007] According to a motion tracking method provided by the present invention, the training process of the motion strategy includes:

[0008] The reference action sequence is subjected to distribution-aware resampling and failure-priority dynamic sampling to obtain a hybrid sampling distribution;

[0009] Based on the aforementioned mixed sampling distribution, training segments are selected;

[0010] The untrained movement strategy is trained based on the training segments to obtain the trained movement strategy.

[0011] According to the action tracking method provided by the present invention, the step of performing distribution-aware resampling and failure-priority dynamic sampling on a reference action sequence to obtain a hybrid sampling distribution includes:

[0012] The reference action sequence is divided into at least two segments according to chronological order;

[0013] Using the occupancy vector of the key joint degrees of freedom, a distribution-aware equalization sampling is performed on all the segments to obtain a distribution matrix, which contains the sampling distribution of the grid occupied by each segment;

[0014] For each segment, hard cases are sampled first based on the failure rate of the segment to obtain the sampling probability of the segment;

[0015] Based on the distribution matrix and the sampling probability corresponding to each segment, the mixed sampling distribution of each segment is determined.

[0016] According to a motion tracking method provided by the present invention, the step of performing distribution-aware equalization sampling on all segments using the occupancy vectors of key joint degrees of freedom to obtain a distribution matrix includes:

[0017] The target segments corresponding to the key joint degrees of freedom are selected and discretized into a mesh, and the occupancy vector of each segment in the mesh is calculated.

[0018] The distribution matrix is ​​obtained by taking the average of the occupancy vectors as the objective.

[0019] According to an action tracking method provided by the present invention, the step of performing hard-case priority sampling based on the failure rate of the segment to obtain the sampling probability of the segment includes:

[0020] Obtain the local failure rate of the segment in the current batch training and the global failure rate of the segment in the overall training;

[0021] The global failure rate is updated based on the local failure rate.

[0022] The sampling probability of the segment is determined based on the updated global failure rate.

[0023] According to a motion tracking method provided by the present invention, the step of training an untrained motion strategy based on the training segment to obtain the trained motion strategy includes:

[0024] Based on the interaction of each training segment, an observation balance condition is constructed. The observation balance condition represents the balance between overall observation and perception action. The perception action includes the body perception of the target robot and the reference action corresponding to each training segment.

[0025] Based on the overall observation, the reference increment corresponding to the reference action is determined;

[0026] Based on the reference increment, an action tracking target is constructed, wherein the action tracking target represents the target action as the sum of the reference action and the reference increment;

[0027] Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joints of the target robot to update the motion strategy and obtain the trained motion strategy.

[0028] According to a motion tracking method provided by the present invention, the step of performing feasible domain projection and joint actuator execution on the target joint of the target robot based on the motion tracking target to update the motion strategy includes:

[0029] Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joint of the target robot to obtain the change in state of the target robot after interaction with the environment.

[0030] Calculate the reward based on the change;

[0031] Based on the reward, the parameters of the motion strategy and the parameters of the overall strategy are updated, and the overall strategy is used to determine the state value.

[0032] According to a motion tracking method provided by the present invention, the method further includes:

[0033] During the training of the motion strategy, the weights and thresholds in the tracking environment are adjusted according to the training progress; when the motion tracking is reset, the physical parameters of the tracking environment are randomized.

[0034] The present invention also provides a motion tracking device, comprising:

[0035] The determination module is configured to determine the motion increment for performing the tracking action based on the tracking action and the motion parameters of the target robot using a trained motion strategy, wherein the motion strategy is trained based on distribution perception and failure-first resampling.

[0036] The control module is configured to control the target robot to perform motion tracking based on the motion increment and the tracking action.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the motion tracking methods described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the motion tracking method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described motion tracking methods.

[0040] The motion tracking method, apparatus, and electronic device provided by this invention determine the motion increment for executing the tracking action based on the tracking action and the motion parameters of the target robot using a trained motion strategy. The motion strategy is trained based on distributed sensing and failure-first resampling. Based on the motion increment and the tracking action, the target robot is controlled to perform motion tracking. This invention enables faster convergence of the motion strategy through distributed sensing and failure-first resampling, improving training efficiency. Combined with motion increment-focused dynamic compensation, it reduces drift and accumulated errors, achieving more stable long-term high-dynamic tracking. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the motion tracking method provided by the present invention.

[0043] Figure 2 This is a schematic diagram of the motion tracking device provided by the present invention.

[0044] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0046] First, a brief description of the relevant content involved in this invention will be given.

[0047] MDP stands for Markov Decision Process, and POMDP stands for Partially Observable Markov Decision Process.

[0048] s represents the system state, which cannot be directly observed; O represents the overall observation; p represents the robot's proprioception; G represents the reference motion / motion library. In this invention, O=Ψ(P,G) represents the fused observation of P and G.

[0049] Residual / Delta Action is the increment 'a' of the motion policy output relative to the reference joint / motion (qref) command, with the final joint target qtar = qref + a.

[0050] Goal-conditioned RL is a type of reinforcement learning that encodes a reference fragment / local future as the "target" g, and outputs an action with a motion policy π(a|O,g).

[0051] A PD (Proportion Differential) controller, also known as a joint actuator or joint space proportional-differential controller, is used to convert qtar into an actuating torque τ, where τ = PD(qtar, q, q ， ), where q represents the joint position, q ， For reference position.

[0052] Feasible domain projection (ΠC) is used to limit and project at least one of the position, velocity, and torque of a joint to ensure hardware safety.

[0053] Distribution-aware resampling is used to balance sampling by joint occupancy histogram, improving long-tail pose coverage.

[0054] Failure-aware priority is used to dynamically increase the sampling probability by applying an exponential moving average failure rate based on the difficulty of the case.

[0055] Curriculum learning is used to progressively increase the difficulty of a schedule from "relaxed" to "rigorous".

[0056] Domain randomization is used to randomize physical parameters such as friction, inertia, gain, and delay to enhance robustness and portability.

[0057] Selective residualization is used to output residuals for critical / sensitive degrees of freedom, while other joint degrees of freedom (DOF) pass-through references.

[0058] Related technologies suffer from the following problems because they ignore the systematic differences between the robot's actual dynamics and the reference motion: long-term drift and error accumulation, such as in long sequences (e.g., dance), where small deviations are gradually amplified; long data tails and inefficient learning, such as common pose oversampling and rare undersampling of "key poses"; limitations of phase scalars, such as using a single "phase" to encode progress, which is not robust to rhythm changes, time distortion, and segment splicing; and the gap between simulation and reality (sim2real), such as deviations in friction, inertia, and delay, which cause tracking instability and exacerbate jitter.

[0059] To address the problems of drift in long-term motion tracking, inefficiency in long-tail data learning, and mismatch with dynamics, this invention provides a motion tracking method, device, and electronic device.

[0060] The following is combined with Figures 1-3 The present invention describes the motion tracking method, apparatus, and electronic device.

[0061] Figure 1 This is a flowchart illustrating the motion tracking method provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 101 and 102.

[0062] Step 101: Using a trained motion strategy, based on the tracking action and the motion parameters of the target robot, determine the motion increment for performing the tracking action. The motion strategy is trained based on distribution perception and failure-first resampling.

[0063] Specifically, the tracking action is the action that the target robot needs to track. Motion increments can be residual actions, used to concentrate capacity on dynamic compensation, resulting in greater stability over long time. Motion strategies are used to focus on compensating for dynamic differences using residual actions.

[0064] In practical applications, the tracking action can be acquired first, and then the tracking action and the motion parameters of the target robot can be processed through a trained motion strategy to output the increment 'a' of the tracking action, i.e., the motion increment.

[0065] In particular, before using the trained motion strategy, the motion strategy can be trained by resampling with distribution / failure awareness to improve training efficiency.

[0066] Step 102: Based on the motion increment and the tracking action, control the target robot to perform motion tracking.

[0067] In practical applications, based on the obtained motion increment, the motion increment and the tracking action are superimposed to obtain the target action. Furthermore, the target robot is controlled to execute the target action in order to perform motion tracking.

[0068] The motion tracking method provided by this invention determines the motion increment for executing the tracking action based on the tracking action and the motion parameters of the target robot, using a trained motion strategy. The motion strategy is trained based on distributed sensing and failure-first resampling. Based on the motion increment and the tracking action, the method controls the target robot to perform motion tracking. This invention enables faster convergence of the motion strategy through distributed sensing and failure-first resampling, improving training efficiency. Combined with motion increment-focused dynamic compensation, it reduces drift and accumulated errors, achieving more stable long-term high-dynamic tracking.

[0069] In one or more optional embodiments of the present invention, the training process of the motion strategy includes:

[0070] The reference action sequence is subjected to distribution-aware resampling and failure-priority dynamic sampling to obtain a hybrid sampling distribution;

[0071] Based on the aforementioned mixed sampling distribution, training segments are selected;

[0072] The untrained movement strategy is trained based on the training segments to obtain the trained movement strategy.

[0073] In practical applications, a distribution-aware resampling method can be used to evenly sample the occupancy histogram of key joint degrees of freedom in the reference action sequence. A failure-aware dynamic sampling method can be used to construct a hard case priority distribution based on the exponential moving average (EMA) failure rate corresponding to the reference action sequence, thus obtaining a mixed sampling distribution. Here, hard cases refer to cases with high failure rates, and the hard case priority distribution means that cases with higher failure rates are given priority.

[0074] Furthermore, after obtaining the mixed sampling distribution, the segments with higher mixed sampling distributions can be selected from the segments corresponding to the reference action sequence as training segments to train the motion model, or the segments can be arranged in order of high to low mixed sampling distributions to form training segments for training the motion model, thereby obtaining a trained motion strategy.

[0075] In this embodiment of the invention, distribution-aware resampling can improve long-tail pose coverage and reduce training bias; failure-first dynamic sampling can adaptively focus on difficult segments of the reference action sequence in the continuous reference action sequence, thereby accelerating convergence.

[0076] In one or more optional embodiments of the present invention, the step of performing distribution-aware resampling and failure-priority dynamic sampling on the reference action sequence to obtain a hybrid sampling distribution includes:

[0077] The reference action sequence is divided into at least two segments according to chronological order;

[0078] Using the occupancy vector of the key joint degrees of freedom, a distribution-aware equalization sampling is performed on all the segments to obtain a distribution matrix, which contains the sampling distribution of the grid occupied by each segment;

[0079] For each segment, hard cases are sampled first based on the failure rate of the segment to obtain the sampling probability of the segment;

[0080] Based on the distribution matrix and the sampling probability corresponding to each segment, the mixed sampling distribution of each segment is determined.

[0081] In practical applications, segmentation and histogram estimation can be performed first: the reference action sequence is divided into S segments according to time, where S is a positive integer greater than 1. Then, histogram equalization sampling, i.e., distribution-aware equalization sampling, is performed using the occupancy vector of the key joint DOF to obtain the distribution matrix ω, thereby improving long-tail pose coverage and reducing training bias.

[0082] Furthermore, the hardest-case priority distribution is constructed using the EMA failure rate to obtain the sampling probability P of the s-th segment. s This approach adaptively focuses on persistently difficult segments to accelerate convergence. Here, s is an integer greater than or equal to 1 and less than or equal to S.

[0083] Then, the mixed sampling distribution P of the s-th segment is calculated according to the following formula. mix .

[0084] P mix =(1-λ)softmax(ω)+λsoftmax(P s )

[0085] Where λ is the annealing parameter, which can be from 0 to 0.5.

[0086] In this embodiment of the invention, the mixed sampling distribution is determined based on the distribution matrix and the sampling probability corresponding to each segment, thereby training the motion strategy and helping to improve training efficiency.

[0087] In one or more optional embodiments of the present invention, the step of performing distribution-aware equalization sampling on all said segments using the occupancy vectors of key joint degrees of freedom to obtain a distribution matrix includes:

[0088] The target segments corresponding to the key joint degrees of freedom are selected and discretized into a mesh, and the occupancy vector of each segment in the mesh is calculated.

[0089] The distribution matrix is ​​obtained by taking the average of the occupancy vectors as the objective.

[0090] In practical applications, histogram estimation can be performed: select the target segments corresponding to key joint DOFs (such as hip / knee pitch) and discretize them into a grid, and calculate the occupancy vector (occ) of each segment relative to the grid. s As an optimization problem, we aim to average the occupancy vectors, i.e., Pω = u. Here, ω is the distribution matrix formed by the distribution of occupancy in each segment of the grid; P is the probability matrix formed by the sampling probabilities of each segment; and u is the target of the average distribution. This ensures both the accuracy and efficiency of obtaining the distribution matrix.

[0091] For example, the distribution-aware weights (probabilities) can be solved statically, i.e., by solving the following formula, to obtain the distribution matrix ω.

[0092]

[0093] Where ω is the distribution matrix; P is the probability matrix; and u is the target with a uniform distribution.

[0094] Alternatively, the inverse frequency weighting (probability) method can be used to determine the distribution matrix ω, that is, the distribution ω of the s-th segment can be determined according to the following formula. s Then, the distribution of each segment is arranged in matrix form to obtain the distribution matrix ω.

[0095] ω s ∝1 / (occ) s +ε1)

[0096] Here, ε1 is the local minimum value to prevent the denominator from being zero.

[0097] In one or more optional embodiments of the present invention, the step of performing hard-case priority sampling based on the failure rate of the fragment to obtain the sampling probability of the fragment includes:

[0098] Obtain the local failure rate of the segment in the current batch training and the global failure rate of the segment in the overall training;

[0099] The global failure rate is updated based on the local failure rate.

[0100] The sampling probability of the segment is determined based on the updated global failure rate.

[0101] In practical applications, the failure rate can be calculated online, and the global failure rate can be updated after each batch of training. The formula for updating the global failure rate is as follows:

[0102] r s ←αf s +(1-α)r s

[0103] Where, rs f is the global failure rate corresponding to the s-th segment. s The local failure rate corresponding to the s-th segment, where α is a parameter based on the moving average.

[0104] After updating the global failure rate, the updated global failure rate is processed according to the following formula to obtain the sampling probability P of the s-th segment. s This is used to prioritize difficult cases.

[0105] P s ∝(r s +ε2) β

[0106] Where β is the sampling temperature and ε2 is the basic sampling probability (minimum value).

[0107] In this embodiment of the invention, updating the global failure rate by the local failure rate and prioritizing difficult cases based on the updated global failure rate can improve the accuracy of sampling probability.

[0108] In one or more optional embodiments of the present invention, the step of training the untrained motion strategy based on the training segment to obtain the trained motion strategy includes:

[0109] Based on the interaction of each training segment, an observation balance condition is constructed. The observation balance condition represents the balance between overall observation and perception action. The perception action includes the body perception of the target robot and the reference action corresponding to each training segment.

[0110] Based on the overall observation, the reference increment corresponding to the reference action is determined;

[0111] Based on the reference increment, an action tracking target is constructed, wherein the action tracking target represents the target action as the sum of the reference action and the reference increment;

[0112] Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joints of the target robot to update the motion strategy and obtain the trained motion strategy.

[0113] In practical applications, rolling and policy updates can be performed. Specifically, the sampling fragments are expanded interactively to construct the observation equilibrium condition O. t =[p t ;G t ], where t represents the current training batch, O represents the overall observation, p represents the ontology perception, and G represents the reference action.

[0114] Then, selective residualization and safe projection are performed: based on the overall observation, the reference increment corresponding to the reference action is determined through a neural network (NN) (built into the robot or the motion strategy), i.e., a = NN(O t NN represents the neural network; based on the reference increment, the action tracking target qtar = qref + a is constructed, where qref is the reference action and qtar is the target action. Thus, by replacing a single phase with a phase-independent reference condition, i.e., qref, robustness to rhythm changes / segment splicing can be improved.

[0115] Then, based on the motion tracking target, projection and smoothing regularization are performed through ΠC (feasible region projection) and PD (joint actuator) to suppress jitter and out-of-bounds, so as to update the motion strategy.

[0116] In one or more optional embodiments of the present invention, the step of performing feasible domain projection and joint actuator execution on the target joint of the target robot based on the motion tracking target to update the motion strategy includes:

[0117] Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joint of the target robot to obtain the change in state of the target robot after interaction with the environment.

[0118] Calculate the reward based on the change;

[0119] Based on the reward, the parameters of the motion strategy and the parameters of the overall strategy are updated, and the overall strategy is used to determine the state value.

[0120] In practical applications, based on motion tracking of the target robot, and through ΠC (feasible region projection) and PD (articular actuator) execution, the change in state after the target robot interacts with the environment can be obtained. Then, the reward R is calculated based on this change (including Rimit and Rphys). See the following formula:

[0121] R=wimit·Rimit + wphys·Rphys.

[0122] Where wimit is the imitation reward weight; Rimit is the joint / velocity / root pose / endpoint / contact timing; wphys is the reward weight of physical constraints; Rphys is at least one of torque, joint limit, foot slip and Δa, where Δa is the change in reference increment.

[0123] Furthermore, proximal policy optimization (PPO), soft actor-critic (SAC), twin-delayed deep deterministic policy gradient (TD3), or important weighted actor-learner architectures (IMPALA) can be used to update πθ and Vϕ based on rewards, where πθ is the parameter of the motion policy and Vϕ is the parameter of the overall policy.

[0124] In this embodiment of the invention, robustness can be improved by calculating the reward based on the change and updating the parameters based on the reward.

[0125] In one or more optional embodiments of the present invention, the method further includes:

[0126] During the training of the motion strategy, the weights and thresholds in the tracking environment are adjusted according to the training progress; when the motion tracking is reset, the physical parameters of the tracking environment are randomized.

[0127] In practical applications, courses / randomization can be performed, that is, adjusting weights and thresholds according to the training progress; and randomizing physical parameters when resetting motion tracking (episode). In this way, the progression of difficulty and randomization of physical parameters can balance accuracy, stability and transferability.

[0128] It should be noted that the motion space can be joint position residual, joint torque / velocity residual, or end-effector space residual.

[0129] The conditional input for overall observation can include the addition of a short look-ahead window. Or form embedding (mass, inertia, joint constraints, etc.).

[0130] Furthermore, the phase signals of the overall observation O and the reference motion / action library G can be used to provide phase in parallel with the reference joints in specific rhythmic tasks (as a redundancy robustness). Meta-learning / adaptive gains can be introduced in online adaptation, or the sampling temperature β and mixing coefficient λ can be updated online.

[0131] For example, the motion tracking method is applied to a motion tracking system, which includes:

[0132] Data / Reference Library Module (G): Stores reference joint sequences {qref}, fragment indexes, and a small number of look-ahead windows;

[0133] Body perception module (P): Collects robot joint position / velocity, root velocity / angular velocity, contact markers, historical residuals, etc.

[0134] Observation fusion module (O): Ot=Ψ(Pt,Gt) (supports splicing or encoder fusion);

[0135] Resampling module: Two-stage sampler consisting of distributed sensing (static) and failure-first (online);

[0136] Residual strategy and training module: πθ(a t |O t G t Output residual (reference increment) a t Optimizers such as PPO update parameters, where t represents the current training batch as t;

[0137] Safety and Control Execution Module: qtar=qref+ a → ΠC → Joint Actuator Output τ;

[0138] Course scheduling and domain randomization module: weight / threshold annealing; friction / inertia / gain / delay randomization;

[0139] Log and Evaluation Module: Record at least one metric such as Root Mean Square Error (RMSE), slip time, Root Mean Square Torque (RMS), and success rate.

[0140] The pseudocode for the motion tracking method is as follows:

[0141] Input: Reference library G, control frequency fc, PPO parameters (γ, λGAE, clip);

[0142] Output: Policy parameters θ.

[0143] Initialize w ← balance_by_hist(P), r s ← 0

[0144] for iter = 1..T:

[0145] Calculate P s ∝(r s +ε2) β , P mix =(1-λ)softmax(ω)+λsoftmax(P s )

[0146] Sampling fragments s ~ P mix Trajectory collection: O=[P;G], a~πθ(a|o,g), qtar=qref+a

[0147] (qtar,τ) ← Π_C(qtar, PD(qtar,q,q , )); Calculate returns and advantages

[0148] PPO update(θ, Vϕ); update r s EMA failure rate

[0149] Course / Randomized Scheduling (Weighted and Threshold Annealing; Physical Parameter Randomization)

[0150] For example (Unitree G1): Referencing a dance movement library, the frequency is controlled at 50 Hz; selective residuals are used (12 DOF output residuals for both legs); long-term stable tracking of dance segments is achieved after training for 100,000 steps.

[0151] For example (H1-2): cross-platform migration is achieved by using the same reward model and various hyperparameters, reward functions and observation space settings.

[0152] It should be noted that the hardware implementation of the motion tracking method includes: NV Orin AGX; the joint actuator supports position / velocity / torque modes and PD; the software implementation of the motion tracking method includes: Linux + Python / C++; RL framework (PyTorch), simulation (MuJoCo / IsaacGym), and real-time execution (LCM kit).

[0153] The motion tracking method provided by this invention focuses on dynamic compensation through residual learning to reduce drift and accumulated errors, thus achieving long-term stability; it accelerates the coverage of key poses and improves sample efficiency through distribution balancing and hard example priority, thus achieving high training efficiency; it balances rhythm changes and parameter uncertainties through phase-independent references and domain randomization, thus achieving robust generalization; it suppresses high-frequency jitter and hardware overshoot through selective residualization and feasible region projection, thus achieving safety and controllability; and it can be transferred across platforms by reusing the same motion strategy on platforms such as G1 / H1-2 / PND Adam or adapting it with small samples.

[0154] The motion tracking device provided by the present invention is described below. The motion tracking device described below and the motion tracking method described above can be referred to in correspondence.

[0155] Figure 2 This is a schematic diagram of the motion tracking device provided by the present invention, as shown below. Figure 2 As shown, the device includes:

[0156] The determination module 201 is configured to determine the motion increment for performing the tracking action based on the tracking action and the motion parameters of the target robot using a trained motion strategy. The motion strategy is trained based on distribution perception and failure-first resampling.

[0157] The control module 202 is configured to control the target robot to perform motion tracking based on the motion increment and the tracking action.

[0158] The motion tracking device provided by this invention determines the motion increment for executing the tracking action based on the tracking action and the motion parameters of the target robot, using a trained motion strategy. The motion strategy is trained based on distributed sensing and failure-first resampling. Based on the motion increment and the tracking action, the device controls the target robot to perform motion tracking. This invention enables faster convergence of the motion strategy through distributed sensing and failure-first resampling, improving training efficiency. Combined with motion increment-focused dynamic compensation, it reduces drift and accumulated errors, achieving more stable long-term high-dynamic tracking.

[0159] In one or more optional embodiments of the present invention, the apparatus further includes a training module configured to:

[0160] The reference action sequence is subjected to distribution-aware resampling and failure-priority dynamic sampling to obtain a hybrid sampling distribution;

[0161] Based on the aforementioned mixed sampling distribution, training segments are selected;

[0162] The untrained movement strategy is trained based on the training segments to obtain the trained movement strategy.

[0163] In one or more optional embodiments of the present invention, the training module is further configured to:

[0164] The reference action sequence is divided into at least two segments according to chronological order;

[0165] Using the occupancy vector of the key joint degrees of freedom, a distribution-aware equalization sampling is performed on all the segments to obtain a distribution matrix, which contains the sampling distribution of the grid occupied by each segment;

[0166] For each segment, hard cases are sampled first based on the failure rate of the segment to obtain the sampling probability of the segment;

[0167] Based on the distribution matrix and the sampling probability corresponding to each segment, the mixed sampling distribution of each segment is determined.

[0168] In one or more optional embodiments of the present invention, the training module is further configured to:

[0169] The target segments corresponding to the key joint degrees of freedom are selected and discretized into a mesh, and the occupancy vector of each segment in the mesh is calculated.

[0170] The distribution matrix is ​​obtained by taking the average of the occupancy vectors as the objective.

[0171] In one or more optional embodiments of the present invention, the training module is further configured to:

[0172] Obtain the local failure rate of the segment in the current batch training and the global failure rate of the segment in the overall training;

[0173] The global failure rate is updated based on the local failure rate.

[0174] The sampling probability of the segment is determined based on the updated global failure rate.

[0175] In one or more optional embodiments of the present invention, the training module is further configured to:

[0176] Based on the interaction of each training segment, an observation balance condition is constructed. The observation balance condition represents the balance between overall observation and perception action. The perception action includes the body perception of the target robot and the reference action corresponding to each training segment.

[0177] Based on the overall observation, the reference increment corresponding to the reference action is determined;

[0178] Based on the reference increment, an action tracking target is constructed, wherein the action tracking target represents the target action as the sum of the reference action and the reference increment;

[0179] Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joints of the target robot to update the motion strategy and obtain the trained motion strategy.

[0180] In one or more optional embodiments of the present invention, the training module is further configured to:

[0181] Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joint of the target robot to obtain the change in state of the target robot after interaction with the environment.

[0182] Calculate the reward based on the change;

[0183] Based on the reward, the parameters of the motion strategy and the parameters of the overall strategy are updated, and the overall strategy is used to determine the state value.

[0184] In one or more optional embodiments of the present invention, the training module is further configured to:

[0185] During the training of the motion strategy, the weights and thresholds in the tracking environment are adjusted according to the training progress; when the motion tracking is reset, the physical parameters of the tracking environment are randomized.

[0186] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 3 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a motion tracking method, which includes: determining the motion increment for performing the tracking action based on the tracking action and the motion parameters of the target robot using a trained motion strategy, wherein the motion strategy is trained based on distributed perception and failure-first resampling; and controlling the target robot to perform motion tracking based on the motion increment and the tracking action.

[0187] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0188] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the motion tracking methods provided by the above methods. The method includes: determining the motion increment for performing the tracking action based on the tracking action and the motion parameters of the target robot using a trained motion strategy, wherein the motion strategy is trained based on distributed perception and failure-first resampling; and controlling the target robot to perform motion tracking based on the motion increment and the tracking action.

[0189] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the motion tracking method provided by the above methods. The method includes: determining a motion increment for performing the tracking action based on a trained motion strategy and motion parameters of a target robot, wherein the motion strategy is trained based on distributed perception and failure-first resampling; and controlling the target robot to perform motion tracking based on the motion increment and the tracking action.

[0190] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0192] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A motion tracking method, characterized in that, include: Using a trained motion strategy, the motion increment for performing the tracking action is determined based on the tracking action and the motion parameters of the target robot. The motion strategy is trained based on distributed perception and failure-first resampling. Based on the motion increment and the tracking action, control the target robot to perform motion tracking; The training process for the aforementioned movement strategy includes: The reference action sequence is subjected to distribution-aware resampling and failure-priority dynamic sampling to obtain a hybrid sampling distribution; Based on the aforementioned mixed sampling distribution, training segments are selected; The untrained movement strategy is trained based on the training segments to obtain the trained movement strategy.

2. The motion tracking method according to claim 1, characterized in that, The process of performing distribution-aware resampling and failure-priority dynamic sampling on the reference action sequence to obtain a hybrid sampling distribution includes: The reference action sequence is divided into at least two segments according to chronological order; Using the occupancy vector of the key joint degrees of freedom, a distribution-aware equalization sampling is performed on all the segments to obtain a distribution matrix, which contains the sampling distribution of the grid occupied by each segment; For each segment, hard cases are sampled first based on the failure rate of the segment to obtain the sampling probability of the segment; Based on the distribution matrix and the sampling probability corresponding to each segment, the mixed sampling distribution of each segment is determined.

3. The motion tracking method according to claim 2, characterized in that, The distribution matrix is ​​obtained by performing distribution-aware equalization sampling on all segments using the occupancy vectors of key joint degrees of freedom, including: The target segments corresponding to the key joint degrees of freedom are selected and discretized into a mesh, and the occupancy vector of each segment in the mesh is calculated. The distribution matrix is ​​obtained by taking the average of the occupancy vectors as the objective.

4. The motion tracking method according to claim 2, characterized in that, The step of performing hard-case priority sampling based on the failure rate of the fragment to obtain the sampling probability of the fragment includes: Obtain the local failure rate of the segment in the current batch training and the global failure rate of the segment in the overall training; The global failure rate is updated based on the local failure rate. The sampling probability of the segment is determined based on the updated global failure rate.

5. The motion tracking method according to any one of claims 1-4, characterized in that, The step of training an untrained motion strategy based on the training fragment to obtain the trained motion strategy includes: Based on the interaction of each training segment, an observation balance condition is constructed. The observation balance condition represents the balance between overall observation and perception action. The perception action includes the body perception of the target robot and the reference action corresponding to each training segment. Based on the overall observation, the reference increment corresponding to the reference action is determined; Based on the reference increment, an action tracking target is constructed, wherein the action tracking target represents the target action as the sum of the reference action and the reference increment; Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joints of the target robot to update the motion strategy and obtain the trained motion strategy.

6. The motion tracking method according to claim 5, characterized in that, The step of updating the motion strategy by projecting feasible regions and executing joint actuators on the target joints of the target robot based on the motion tracking target includes: Based on the motion tracking target, feasible domain projection and joint actuator execution are performed on the target joint of the target robot to obtain the change in state of the target robot after interaction with the environment. Calculate the reward based on the change; Based on the reward, the parameters of the motion strategy and the parameters of the overall strategy are updated, and the overall strategy is used to determine the state value.

7. The motion tracking method according to claim 1, characterized in that, The method further includes: During the training of the motion strategy, the weights and thresholds in the tracking environment are adjusted according to the training progress; when the motion tracking is reset, the physical parameters of the tracking environment are randomized.

8. A motion tracking device, characterized in that, include: The determination module is configured to determine the motion increment for performing the tracking action based on the tracking action and the motion parameters of the target robot using a trained motion strategy, wherein the motion strategy is trained based on distribution perception and failure-first resampling. The control module is configured to control the target robot to perform motion tracking based on the motion increment and the tracking action; The training module is configured to perform distribution-aware resampling and failure-priority dynamic sampling on the reference action sequence to obtain a hybrid sampling distribution; and to select training segments based on the hybrid sampling distribution. The untrained movement strategy is trained based on the training segments to obtain the trained movement strategy.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the motion tracking method as described in any one of claims 1 to 7.

Images