Adaptive resampling methods, devices, electronic equipment, and storage media for robot motion control training

Abstract

This invention provides an adaptive resampling method, device, electronic equipment, and storage medium for robot motion control training, relating to the field of embodied intelligence technology. The method includes: performing motion segment-level sampling based on a pre-constructed robot motion dataset to determine a target motion segment from the dataset according to a preset first sampling distribution; performing time-segment-level sampling on the target motion segment to determine a training start time segment from the target motion segment according to a second sampling distribution dynamically updated based on historical failure information; and initializing the robot's execution state based on the reference robot state corresponding to the training start time segment to start or reset the training task. The method provided by this invention significantly improves the coverage and learning efficiency of key and difficult motion segments during training, thereby accelerating policy convergence and enhancing the robustness and stability of the robot when performing long-duration, complex motion tasks.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligence technology, and in particular to an adaptive resampling method, apparatus, electronic device, and storage medium for robot motion control training. Background Technology

[0002] Current humanoid robot motion control training heavily relies on large-scale motion data, but this data usually comes from various heterogeneous sources such as optical motion capture, IMU sensors, open-source human motion libraries and generative models, resulting in significant differences in data quality, length, initial state and distribution.

[0003] Existing training methods typically involve simply mixing and randomly sampling these data, or employing generic data augmentation and playback mechanisms, lacking sophisticated design tailored to the specific characteristics of motion control tasks. Current technologies suffer from significant drawbacks when processing multi-source heterogeneous motion data, including low data utilization efficiency, insufficient learning of difficult examples, and limited engineering performance. Therefore, a method is urgently needed to address these issues. Summary of the Invention

[0004] This invention provides an adaptive resampling method, apparatus, electronic device, and storage medium for robot motion control training, in order to overcome the deficiencies existing in the prior art.

[0005] This invention provides an adaptive resampling method for robot motion control training, comprising the following steps: Based on a pre-built robot motion dataset, motion segment-level sampling is performed to determine the target motion segment from the motion dataset according to a preset first sampling distribution; For the target action segment, time-segment sampling is performed to determine the training start time segment from the target action segment based on a second sampling distribution that is dynamically updated based on historical failure information; Based on the reference robot state corresponding to the training start time period, the robot's state is initialized to start or reset the training task.

[0006] According to an adaptive resampling method for robot motion control training provided by the present invention, the process of constructing the first sampling distribution includes: Obtain historical failure information for each action segment in the robot motion dataset; Based on the historical failure information of each action segment, calculate the first failure sampling probability corresponding to each action segment; Based on the historical frequency of each action segment being selected in historical training, the first novelty sampling probability corresponding to each action segment is calculated. The first sampling distribution is obtained by weighting and mixing the first failure sampling probability, the first novelty sampling probability, and the uniform sampling probability; wherein, the uniform sampling probability is defined as the probability that each action segment is selected for training is equal.

[0007] An adaptive resampling method for robot motion control training according to the present invention, wherein the first sampling distribution is obtained by weighted mixing of the first failure sampling probability, the first novelty sampling probability, and the uniform sampling probability, includes: In the initial training phase, the first sampling distribution is configured to contain only the uniform sampling probability; During the training transition phase, the weights of the first failure sampling probability and the first novelty sampling probability are gradually increased from zero to a preset target weight, and then weighted and mixed with the uniform sampling probability to obtain the first sampling distribution.

[0008] An adaptive resampling method for robot motion control training according to the present invention includes performing time-segment-level sampling on the target motion segment to determine the training start time segment from the target motion segment based on a second sampling distribution dynamically updated based on historical failure information, comprising: The target action segment is divided into multiple consecutive time periods according to the time sequence; Based on the failure events that occurred during the historical training over the multiple consecutive time periods, historical failure information for the multiple consecutive time periods is generated and dynamically updated. The second sampling distribution is generated based on the updated historical failure information; The training start time period is determined from the plurality of consecutive time periods based on the second sampling distribution.

[0009] According to an adaptive resampling method for robot motion control training provided by the present invention, after determining the training start time period from the plurality of consecutive time periods based on the second sampling distribution, the method further includes: Determine the target time point within the training start time period; The robot state corresponding to the target time point in the robot motion dataset is used as the reference robot state.

[0010] According to the adaptive resampling method for robot motion control training provided by the present invention, before performing motion segment-level sampling based on a pre-built robot motion dataset, the method further includes: If the total number of action segments in the action dataset exceeds the total number of parallel training environments, a subset of actions is determined from the action dataset as an active action pool according to the first sampling distribution; wherein the size of the action subset is smaller than the total number of action segments in the action dataset but larger than the total number of parallel training environments. The step of determining the target action segment from the action dataset according to a preset first sampling distribution includes: The target action segment is determined from the active action pool.

[0011] According to the adaptive resampling method for robot motion control training provided by the present invention, after initializing the robot state based on the reference robot state corresponding to the training start time period to start or reset the training task, the method further includes: In the event of a training task failure, record the failed action segment that caused the failure and the corresponding training start time. Based on the failed action segments and their corresponding training start time periods, update the historical failure information used to calculate the first sampling distribution and the historical failure information used to calculate the second sampling distribution.

[0012] The present invention also provides an adaptive resampling device for robot motion control training, comprising the following modules: The first sampling module is used to perform motion segment-level sampling based on a pre-built robot motion dataset, so as to determine the target motion segment from the motion dataset according to a preset first sampling distribution; The second sampling module is used to perform time-segment-level sampling on the target action segment in order to determine the training start time segment from the target action segment according to the second sampling distribution that is dynamically updated based on historical failure information. The training module is used to perform state initialization on the robot based on the reference robot state corresponding to the training start time period, so as to start or reset the training task.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an adaptive resampling method for robot motion control training as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an adaptive resampling method for robot motion control training as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements an adaptive resampling method for robot motion control training as described above.

[0016] This invention provides an adaptive resampling method, apparatus, electronic device, and storage medium for robot motion control training. It performs action segment-level sampling based on a pre-built robot action dataset to determine target action segments from the dataset according to a preset first sampling distribution. For the target action segments, it performs time-segment-level sampling to determine the training start time segment from the target action segments according to a second sampling distribution dynamically updated based on historical failure information. Based on the reference robot state corresponding to the training start time segment, it performs state initialization on the robot to start or reset the training task. Therefore, this invention effectively overcomes the problems of low training efficiency and insufficient stability caused by direct random sampling after mixing multi-source action data. Through progressive adaptive resampling from macroscopic action segments to microscopic time segments, combined with precise state initialization, it significantly improves the coverage and learning efficiency of key and difficult motion segments in the training process, thereby accelerating policy convergence and enhancing the robustness and stability of the robot when performing long-term, complex motion tasks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the adaptive resampling method for robot motion control training provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the framework of the adaptive resampling method for robot motion control training provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the adaptive resampling device for robot motion control training provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] Before describing the present invention, the terms and concepts involved in the present invention will be explained illustratively.

[0024] MoCap: Motion Capture (Optical Motion Capture / Inertial Motion Capture).

[0025] Multi-source motion corpus: A collection of multi-source motion data, including self-collected optical / IMU motion capture, open-source human motion libraries, and generated motion and periodic augmentation data.

[0026] Robot-space motion: A unified representation of robot joint space motion, including joint position / velocity and the pose and velocity of a selected rigid body (key body part).

[0027] Motion preloading: Preloads and concatenates multiple motion files into a continuous tensor, supporting batch indexing and high-speed sampling.

[0028] Motion-level sampling: Sampling probability is obtained by mixing difficulty (failure rate), novelty (low-frequency coverage) and uniformity.

[0029] Bin-level adaptive sampling: The starting frame is sampled in time segments (bin) within a single action, and the failure distribution is updated by exponential moving average (EMA) and stabilized by kernel smoothing.

[0030] Active motion pool: When the total number of motions is much greater than the number of parallel environments, a limited "active motion pool" is sampled first and then allocated to the environments to improve coverage and training efficiency.

[0031] Sharding across GPUs: During multi-GPU training, action subsets are loaded in rank-based shards to reduce repetition and memory usage.

[0032] RSI (Reference-State Initialization): Initializes from any phase to a reference action state, and can be augmented with small perturbations to improve robustness.

[0033] The following description, in conjunction with the accompanying drawings, describes an adaptive resampling method, apparatus, electronic device, and storage medium for robot motion control training according to the present invention.

[0034] Figure 1 This is a flowchart illustrating the adaptive resampling method for robot motion control training provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: Step 100: Based on the pre-built robot motion dataset, perform motion segment-level sampling to determine the target motion segment from the motion dataset according to the preset first sampling distribution.

[0035] Specifically, a robot motion dataset refers to a collection of multi-source motion trajectories that have been pre-collected, processed, and standardized. The data sources may include high-precision optical motion capture systems, easily deployable inertial measurement unit sensors, open-source human motion databases, and motion fragments synthesized through generative models. These heterogeneous data need to undergo processing steps such as skeleton retargeting, frame rate alignment, and coordinate system unification, and be uniformly converted into robot state sequences characterized by dimensions such as robot joint angles, joint velocities, key rigid body poses, and velocities, ultimately forming a standardized data file that can be directly indexed by training programs.

[0036] Action segment level sampling refers to selecting complete motion trajectory segments (i.e. target action segments) for the next training session from a macroscopic level of the entire dataset, based on a first sampling distribution that comprehensively considers historical performance and coverage. This distribution can be a preset static distribution, but is preferably an adaptive distribution that can be dynamically adjusted according to training feedback.

[0037] Step 200: Perform time-segment sampling on the target action segment to determine the training start time segment from the target action segment based on the second sampling distribution that is dynamically updated based on historical failure information.

[0038] Specifically, time-segment sampling involves selecting a specific action segment and then further refining the selection on its timeline. This is based on a second sampling distribution, which is dynamically generated based on the frequency of failures (i.e., historical failure information) that occurred in different time segments (i.e., time periods) of the segment during past training. This allows for the precise location of error-prone critical moments in the movement.

[0039] Step 300: Based on the reference robot state corresponding to the training start time period, perform state initialization on the robot to start or reset the training task.

[0040] It should be noted that the process for determining the robot's state includes: Determine the target time point within the training start time period; The robot state corresponding to the target time point in the robot motion dataset is used as the reference robot state.

[0041] Specifically, after determining a training start time period through the second sampling distribution, the system further determines a target time point within that training start time period. After determining the target time point, the system then indexes and extracts the complete robot state data corresponding to that precise time point from the pre-built and loaded robot motion dataset. This set of data is defined as the reference robot state for this training round, which specifically includes the actual recorded values ​​of physical quantities such as the angular position of all robot joints at that moment, the instantaneous angular velocity of the joints, the position and orientation of the robot's body root in space, and the linear and angular velocities of the root.

[0042] Furthermore, state initialization refers to directly setting the state of the simulation environment or physical robot (including all joint positions, velocities, and body root poses, etc.) to the reference robot state corresponding to the sampled training start time period, thereby ensuring that each training round can start from a physically real and reasonable motion state, avoiding the distribution mismatch problem caused by initialization from a fixed static posture.

[0043] The above describes the steps of the adaptive resampling method for robot motion control training provided by this invention. As can be seen from the above description, the adaptive resampling method for robot motion control training provided by this invention performs action segment-level sampling based on a pre-built robot action dataset to determine target action segments from the action dataset according to a preset first sampling distribution; for the target action segments, performs time-segment-level sampling to determine the training start time segment from the target action segments according to a second sampling distribution dynamically updated based on historical failure information; and initializes the robot's state based on the reference robot state corresponding to the training start time segment to start or reset the training task. Therefore, this invention effectively overcomes the problems of low training efficiency and insufficient stability caused by direct random sampling after mixing multi-source action data. Through progressive adaptive resampling from macroscopic action segments to microscopic time segments, combined with precise state initialization, it significantly improves the coverage and learning efficiency of key and difficult motion segments in the training process, thereby accelerating policy convergence and enhancing the robustness and stability of the robot when performing long-term, complex motion tasks.

[0044] Based on the above embodiments, in this embodiment, the process of constructing the first sampling distribution includes: Obtain historical failure information for each action segment in the robot motion dataset; Based on the historical failure information of each action segment, calculate the first failure sampling probability corresponding to each action segment; Based on the historical frequency of each action segment being selected in historical training, the first novelty sampling probability corresponding to each action segment is calculated. The first sampling distribution is obtained by weighting and mixing the first failure sampling probability, the first novelty sampling probability, and the uniform sampling probability; wherein, the uniform sampling probability is defined as the probability that each action segment is selected for training is equal.

[0045] Specifically, historical failure information is a quantitative record of failure events that led to task termination in past training for each action segment. This can be represented as the cumulative number of failures, failure rate, or a smoothed failure popularity value for that segment, directly reflecting its relative difficulty. The first failure sampling probability is calculated based on this information. Its core idea is to give segments with high failure rates a higher chance of being selected in subsequent training, thereby achieving targeted reinforcement learning for difficult examples. Historical frequency records the total number of times each segment is assigned to the training environment, reflecting its exposure. The first novelty sampling probability is calculated based on this, typically assigning higher probabilities to segments with low exposure. Its calculation can be represented as an inverse proportionality to the number of selections or an inverse square root function. This aims to encourage exploration and prevent training from becoming overly focused on a few popular segments while neglecting the "long tail" of the dataset. The uniform sampling probability, as a fundamental term, provides each segment with an equal chance of being selected, serving as an important baseline to ensure broad coverage of the sampling distribution and avoid early local bias due to inaccurate failure information. Finally, the first sampling distribution is obtained by weighting and mixing these three probabilities with adjustable weights.

[0046] Further, the step of weighting and mixing the first failure sampling probability, the first novelty sampling probability, and the uniform sampling probability to obtain the first sampling distribution includes: In the initial training phase, the first sampling distribution is configured to contain only the uniform sampling probability; During the training transition phase, the weights of the first failure sampling probability and the first novelty sampling probability are gradually increased from zero to a preset target weight, and then weighted and mixed with the uniform sampling probability to obtain the first sampling distribution.

[0047] Specifically, the initial training phase refers to the first few rounds or a certain time window of model training. At this time, due to the limited number of samplings, the failure count of each action segment is not yet representative. If a sampling strategy based on failure rate is adopted rashly, it is easy to over-amplify or ignore certain segments due to a few accidental failures or successes, leading to deviations in the training direction. Therefore, in this phase, the system adopts a conservative strategy, configuring the first sampling distribution to include only uniform sampling probability, that is, all action segments have an equal chance of being selected. This ensures that the action dataset can be explored extensively and evenly in the early stage of training, accumulating reliable and comprehensive initial data for subsequent difficulty assessment.

[0048] Subsequently, the system enters the training transition phase. During this phase, as the exploration deepens, the failure statistics gradually become more stable and reliable. The system then begins to introduce an intelligent sampling mechanism based on the learning effect in a controllable manner. Through a predefined scheduling strategy (such as linear growth), the weight coefficients corresponding to the first failure sampling probability and the first novelty sampling probability are gradually increased from the initial value of zero to the pre-set target weights. In this process, the weight of the uniform sampling probability is correspondingly reduced from 1, and finally, the three form a stable mixing ratio under the target weights.

[0049] The adaptive resampling method for robot motion control training provided in this embodiment significantly improves the relevance and efficiency of training data utilization by constructing a hybrid sampling distribution that integrates failure rate, novelty and uniform exploration, and by adopting a dynamic weight scheduling strategy that smoothly transitions from uniform exploration to adaptive sampling during the training process.

[0050] Based on the above embodiments, in this embodiment, step 200 performs time-segment-level sampling on the target action segment to determine the training start time segment from the target action segment according to the second sampling distribution dynamically updated based on historical failure information, including: Step 210: Divide the target action segment into multiple consecutive time periods according to the time sequence.

[0051] Step 220: Based on the failure events that occurred in the historical training over the multiple consecutive time periods, generate and dynamically update the historical failure information for the multiple consecutive time periods.

[0052] Step 230: Generate the second sampling distribution based on the updated historical failure information.

[0053] Step 240: Determine the training start time period from the plurality of consecutive time periods according to the second sampling distribution.

[0054] Specifically, once a target action segment is selected, the entire timeline of the segment is first divided into multiple consecutive time periods (often referred to as "bins" in technical implementation) according to a fixed time window or based on the action control cycle. Historical failure information specifically refers to the records of the specific time periods to which failure events leading to task termination occurred when the robot executed the action segment in past training. For example, if the robot falls due to loss of balance in the aforementioned "sudden stop and turn" segment, and the fall occurs within a 0.5-second interval in the middle of the segment, the failure count for that interval will increase. The system continuously and dynamically updates this information, using algorithms such as exponential moving averages to smooth noise and track the latest trends, thereby obtaining the current failure frequency for each time period. Based on these dynamically updated failure frequencies, the system generates a second sampling distribution, the core principle of which is: the more frequently a time period fails, the higher the probability that it will be selected as the starting time period for subsequent training. This means that training will no longer simply begin from the start of the action or a random position, but will have a high probability of starting from time intervals that have been verified as error-prone points or high-difficulty phases.

[0055] The adaptive resampling method for robot motion control training provided in this embodiment concentrates training resources on the critical moments within the action that most need to be learned and corrected. This enables precise strengthening and efficient overcoming of weak links in long actions, greatly improving the efficiency and depth of learning from difficult examples.

[0056] Based on the above embodiments, in this embodiment, before step 100 performs motion fragment-level sampling based on a pre-built robot motion dataset, the method further includes: If the total number of action segments in the action dataset exceeds the total number of parallel training environments, a subset of actions is determined from the action dataset as an active action pool based on the first sampling distribution; wherein the size of the action subset is smaller than the total number of action segments in the action dataset but larger than the total number of parallel training environments.

[0057] Step 100 determines the target action segment from the action dataset according to a preset first sampling distribution, including: The target action segment is determined from the active action pool.

[0058] Specifically, in robot motion control training, especially in distributed parallel training scenarios, there are often dozens or even hundreds of parallel training environments (referring to independently running simulation instances or training threads) working simultaneously. When the pre-built robot motion dataset is extremely large, containing far more motion segments (e.g., tens of thousands) than the total number of parallel training environments (e.g., dozens), sampling from the entire dataset each time, while theoretically providing the broadest coverage, presents a significant efficiency bottleneck in engineering. On the one hand, frequent full-set indexing and probability calculations are costly; on the other hand, because the number of environments is far less than the amount of data, a large amount of data has no chance of being selected in the short term, while the few selected data may be repeatedly sampled in multiple environments, leading to uneven resource utilization and potential overfitting risks. To address this, this embodiment proposes periodically or on-demand determining a subset of actions from the motion dataset as an active motion pool based on a first sampling distribution. This first sampling distribution is the aforementioned intelligent distribution that balances difficulty, novelty, and uniformity, ensuring that the selected subset itself is of high quality and representativeness. The size of this action subset is designed to be smaller than the total number of datasets but larger than the total number of parallel environments. For example, 500 actions are selected from 10,000 data points as the active pool, while there are 50 parallel environments. This design ensures that the pool is large enough to provide ample, non-repeating selection space for all environments, while also being small enough to be efficiently loaded into memory and quickly shared and accessed by all training processes.

[0059] Subsequently, within each sampling period, when each parallel training environment needs a target action fragment, it is instead determined from the active action pool. Since the active pool is of moderate size and already resident in memory, this operation is extremely fast, effectively avoiding conflicts where different environments select the same fragment, and maximizing data utilization.

[0060] The adaptive resampling method for robot motion control training provided in this embodiment significantly optimizes the system performance of large-scale distributed training by intelligently pre-selecting and caching massive motion datasets while maintaining the core advantages of adaptive resampling.

[0061] Based on the above embodiments, in this embodiment, after step 300 initializes the robot's state based on the reference robot state corresponding to the training start time period to start or reset the training task, the method further includes: In the event of a training task failure, record the failed action segment that caused the failure and the corresponding training start time. Based on the failed action segments and their corresponding training start time periods, update the historical failure information used to calculate the first sampling distribution and the historical failure information used to calculate the second sampling distribution.

[0062] Specifically, each training task may terminate with successful completion or failure. In robot motion control training, failure typically has clear criteria, such as the robot's root height falling below a threshold (falling), joint angles exceeding safety limits, or the task objective not being achieved for an extended period. Once a task is deemed a failure, the system immediately initiates a feedback recording process: First, it precisely locates and records the failed action segment, which refers to the unique identifier of the specific data segment used as the motion blueprint in this training session. Simultaneously, it records the training start time segment from which the failed round began (i.e., the "corresponding training start time segment"). This combination of information precisely locates the failure event in both the data and time dimensions.

[0063] Subsequently, the system updates the historical failure information used to calculate the first sampling distribution and the historical failure information used to calculate the second sampling distribution based on these records. For the first sampling distribution, the system increases the overall failure count of the failed action segment, which may increase its probability of being selected as the first failed sample in subsequent iterations, making training more inclined to revisit this proven difficult overall action. For the second sampling distribution, the system further identifies which specific "bin" (time period) within the action segment the failure occurred in and increases the failure count of that specific "bin," dynamically updating its failure heat through methods such as exponential moving averages. This ensures that when the action segment is selected again in the future, the high-risk time period within it that led to the failure will have a higher probability of being selected as the starting time period for the next training iteration.

[0064] The adaptive resampling method for robot motion control training provided in this embodiment accurately attributes each training failure to a specific action segment and time period, and dynamically updates the historical failure information used to guide sampling, which constitutes the core driver for the self-optimization of the entire adaptive resampling method.

[0065] Figure 2 This is a schematic diagram of the framework of the adaptive resampling method for robot motion control training provided by the present invention. The following is a combination of... Figure 2 The adaptive resampling method for robot motion control training provided by this invention will be fully described.

[0066] 1. The entire process begins with the data source, which aggregates heterogeneous data from multiple sources, including high-precision optical motion capture (Vicon), scalable inertial motion capture (IMU), open-source human motion library (AMASS), and generated model data (Gemmo / FLD).

[0067] 2. Next, the unified processing stage begins. Through skeleton relocation, coordinate system normalization, frame rate alignment, and necessary filtering and interpolation, all data is uniformly converted into a standardized format (i.e., "Robot-space") representing robot joint angles, velocities, and key rigid body poses / velocities.

[0068] 3. The processed data enters the preloading and indexing stage. All action fragments are concatenated into a continuous tensor in memory, and the length and offset index (motion_lengths / offsets) are maintained. During distributed training, the data is also split by GPU to reduce repeated loading and memory usage.

[0069] 4. Next is the core adaptive sampling loop: The Motion-level sampling module uses the failure rate (difficulty) of the action segment, the reciprocal of the selection frequency (novelty), and a uniform term to ensure coverage. The sampling distribution is formed by weighting and mixing these three factors. The warmup + ramp strategy is used to smoothly transition in the early stage of training. When the amount of data is extremely large, the Activepool will be used for pre-selection to improve efficiency.

[0070] Based on this, the Bin-level sampling module divides each action into multiple time periods (bins) according to time sequence. Based on the failure events that occur in each time period, it updates the failure statistics with an exponential moving average, and then uses a convolutional kernel to smooth the data after adding a uniform base, thereby generating a fine-grained sampling distribution that focuses on the high-incidence moments of failure.

[0071] 5. After sampling is completed, perform RSI reset to initialize the robot state to the reference state corresponding to the sampling point, and superimpose attitude, velocity and joint perturbations of limited amplitude to enhance robustness.

[0072] 6. Finally, the initialized state is sent to the training phase, where reinforcement learning or imitation learning algorithms such as PPO and BC perform policy optimization in a simulation environment (such as a 50Hz control frequency), thereby completing the end-to-end efficient training from multi-source data to a high-performance motion control policy.

[0073] To facilitate reproduction, this embodiment provides the definition and calculation formula for the core two-layer resampling and preloading index. Let the action library contain M action segments, with indices m∈{0,…,M-1}, and the length of the m-th action segment be L. m (Frame count). The training control period is Δt (1 / 50 s in this embodiment).

[0074] (1) Preloading of continuous tensors and global index mapping.

[0075] Concatenate all actions into a continuous tensor along the time dimension and maintain the following index information: ; ; Total frames .

[0076] Given (motion_id=m, frame=t), its global index g(m,t) in a continuous tensor is defined as: .

[0077] Therefore, batch gathering: X[g(m,t)] can be completed in O(1) time, avoiding file I / O during training. During distributed training, a deterministic shuffle of motion_paths can be performed before loading, and chunk / stride slicing can be performed according to (rank, world_size) to form non-overlapping motion subsets, thereby reducing the resampling in multi-GPU training.

[0078] (2) Motion-level mixed sampling distribution. Maintain three counts: Number of samples ; Number of times assigned ; Number of failures .

[0079] Action segment level sampling module: maintenance ;according to Generates motion sampling distribution, supporting warmup+ramp and cap_beta limiting; optional max_active_motions active pool.

[0080] Action-level bin sampling module: Divide the action into bins according to frames_per_bin=round(1 / Δt); when termination fails, update current_bin_failed and perform EMA with C←α·current+(1-α)·C; add uniform floor and smooth with convolution kernel to obtain p(b|m), thereby sampling time_step and using it for RSI.

[0081] make (If the denominator is 0, then set it to 0); The novelty term is defined as follows: and normalization Simultaneously define a uniform distribution .

[0082] The final motion-level sampling distribution is a weighted mixture of the three: ,in .

[0083] To reduce the "cold start bias" in the early stages of training (where almost all initial failures lead to distorted difficulty estimation), this implementation uses a warmup+ramp approach: during the warmup period, let... (Completely uniform), then within the ramp period, linear / cosine scheduling will be applied ( , Smoothly increase from 0 to the configured target value.

[0084] When the total number of actions M is much larger than the number of parallel environments N, we can first sample K actions without replacement from p(m) to form an active action pool (max_active_motions), and then evenly map the active pool to N environments to balance coverage and throughput.

[0085] (3) Prioritize sampling for failures within the action (bin-level). For each action m, divide the timeline into... bin: .

[0086] When an environment fails to terminate at time step t of action m, calculate the bin for failure attribution: .

[0087] And on Add 1. After each simulation step, update the cumulative failure heatmap using an exponential moving average: and clear to zero .

[0088] Input: Action set {m}; Number of parallel environments N; Control period Δt; Parameters weight_fail, weight_novel, cap_beta; warmup_s, ramp_s, schedule; EMA coefficient α; Bin kernel k(λ,K); Remapping period P (seconds).

[0089] Output: (motion_id, t0) obtained after each reset and the reference window r_{t0:t0+H-1} with horizon=H.

[0090] Initialization: The preloading action is a continuous tensor and computation. ;Set ( , , )=0; set =0; initialize the motion-level distribution to uniform; sample motion for each environment and set time_steps=0.

[0091] Loop (per simulation step): (i) time_steps++; if time_steps≥ Then call ResampleCommand; (ii) If the termination_manager instruction fails, attribute the failure to bin b and increment current_bin_failed. (iii) Update with EMA And clear current_bin_failed; (iv) After each P / Δt step, reassign motion (available active pool) to all environments by p(m), and sample t0 by p(b|m) within the action, and perform RSI write to the reference state and perturbation.

[0092] ,in (i=0…K-1) and normalize.

[0093] Normalization yields the bin distribution and sampling Then, within bin, use a random offset. Generate continuous time positions and map them back to frame indices: and cut off to .

[0094] (4) RSI Reset and Perturbation Enhancement. Write the robot root state and joint states to the reference values ​​(root, q, dq) at time t0: .

[0095] And apply finite perturbations to improve robustness: root location / Euler angle increment Root velocity increment Joint position increment Then, soft limit clipping is applied to Δq, and finally the result is written into the simulator / robot controller.

[0096] The adaptive resampling method for robot motion control training provided in this invention significantly reduces I / O overhead and improves the efficiency of large-scale training by preloading continuous tensors and batch indexing; motion-level (difficulty / novelty / uniformity) and bin-level (failure priority) dual-layer sampling simultaneously ensures diversity and focus on difficult examples; warmup+ramp gradually introduces difficulty weights from uniformity to reduce bias caused by statistical noise in early failures; continuous revisiting of high-failure segments accelerates the repair of key shortcomings and reduces stumbling / falling; slice loading and active action pool reduce repetition and memory pressure and adapt to massive action libraries.

[0097] The adaptive resampling device for robot motion control training provided by the present invention will be described below. The adaptive resampling device for robot motion control training described below can be referred to in correspondence with the adaptive resampling method for robot motion control training described above.

[0098] Figure 3 This is a schematic diagram of the adaptive resampling device for robot motion control training provided by the present invention, as shown below. Figure 3 As shown, the adaptive resampling device for robot motion control training provided by the present invention includes: The first sampling module 301 is used to perform motion segment-level sampling based on a pre-built robot motion dataset to determine the target motion segment from the motion dataset according to a preset first sampling distribution. The second sampling module 302 is used to perform time-segment-level sampling on the target action segment to determine the training start time segment from the target action segment based on the second sampling distribution that is dynamically updated based on historical failure information. The training module 303 is used to perform state initialization on the robot based on the reference robot state corresponding to the training start time period, so as to start or reset the training task.

[0099] The adaptive resampling device for robot motion control training provided by this invention performs action segment-level sampling based on a pre-built robot action dataset to determine target action segments from the dataset according to a preset first sampling distribution. For the target action segments, time-segment-level sampling is performed to determine the training start time segment from the target action segments according to a second sampling distribution dynamically updated based on historical failure information. Based on the reference robot state corresponding to the training start time segment, the robot's state is initialized to start or reset the training task. Therefore, this invention effectively overcomes the problems of low training efficiency and insufficient stability caused by direct random sampling after mixing multi-source action data. Through progressive adaptive resampling from macroscopic action segments to microscopic time segments, combined with precise state initialization, it significantly improves the coverage and learning efficiency of key and difficult motion segments in the training process, thereby accelerating policy convergence and enhancing the robustness and stability of the robot when performing long-term, complex motion tasks.

[0100] Based on the above embodiments, in this embodiment, the device further includes a construction module, specifically used for: Obtain historical failure information for each action segment in the robot motion dataset; Based on the historical failure information of each action segment, calculate the first failure sampling probability corresponding to each action segment; Based on the historical frequency of each action segment being selected in historical training, the first novelty sampling probability corresponding to each action segment is calculated. The first sampling distribution is obtained by weighting and mixing the first failure sampling probability, the first novelty sampling probability, and the uniform sampling probability; wherein, the uniform sampling probability is defined as the probability that each action segment is selected for training is equal.

[0101] Based on the above embodiments, in this embodiment, the device further includes a mixing module, specifically used for: In the initial training phase, the first sampling distribution is configured to contain only the uniform sampling probability; During the training transition phase, the weights of the first failure sampling probability and the first novelty sampling probability are gradually increased from zero to a preset target weight, and then weighted and mixed with the uniform sampling probability to obtain the first sampling distribution.

[0102] Based on the above embodiments, in this embodiment, the second sampling module 302 is specifically used for: The target action segment is divided into multiple consecutive time periods according to the time sequence; Based on the failure events that occurred during the historical training over the multiple consecutive time periods, historical failure information for the multiple consecutive time periods is generated and dynamically updated. The second sampling distribution is generated based on the updated historical failure information; The training start time period is determined from the plurality of consecutive time periods based on the second sampling distribution.

[0103] Based on the above embodiments, in this embodiment, the device further includes a first determining module, specifically used for: After determining the training start time period from the plurality of consecutive time periods according to the second sampling distribution, Determine the target time point within the training start time period; The robot state corresponding to the target time point in the robot motion dataset is used as the reference robot state.

[0104] Based on the above embodiments, in this embodiment, the device further includes a second determining module, specifically used for: Before performing motion fragment-level sampling based on the pre-built robot motion dataset, If the total number of action segments in the action dataset exceeds the total number of parallel training environments, a subset of actions is determined from the action dataset as an active action pool according to the first sampling distribution; wherein the size of the action subset is smaller than the total number of action segments in the action dataset but larger than the total number of parallel training environments. The first sampling module 301 is specifically used for: The target action segment is determined from the active action pool.

[0105] Based on the above embodiments, in this embodiment, the device further includes an update module, specifically used for: The robot's state is initialized based on the reference robot state corresponding to the training start time period, in order to start or reset the training task. In the event of a training task failure, record the failed action segment that caused the failure and the corresponding training start time. Based on the failed action segments and their corresponding training start time periods, update the historical failure information used to calculate the first sampling distribution and the historical failure information used to calculate the second sampling distribution.

[0106] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device can be a robot or other electronic device, and may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions from the memory 430 to execute an adaptive resampling method for robot motion control training, including: Based on a pre-built robot motion dataset, motion segment-level sampling is performed to determine the target motion segment from the motion dataset according to a preset first sampling distribution; For the target action segment, time-segment sampling is performed to determine the training start time segment from the target action segment based on a second sampling distribution that is dynamically updated based on historical failure information; Based on the reference robot state corresponding to the training start time period, the robot's state is initialized to start or reset the training task.

[0107] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the adaptive resampling method for robot motion control training provided by the above methods, including: Based on a pre-built robot motion dataset, motion segment-level sampling is performed to determine the target motion segment from the motion dataset according to a preset first sampling distribution; For the target action segment, time-segment sampling is performed to determine the training start time segment from the target action segment based on a second sampling distribution that is dynamically updated based on historical failure information; Based on the reference robot state corresponding to the training start time period, the robot's state is initialized to start or reset the training task.

[0109] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an adaptive resampling method for robot motion control training provided by the methods described above, comprising: Based on a pre-built robot motion dataset, motion segment-level sampling is performed to determine the target motion segment from the motion dataset according to a preset first sampling distribution; For the target action segment, time-segment sampling is performed to determine the training start time segment from the target action segment based on a second sampling distribution that is dynamically updated based on historical failure information; Based on the reference robot state corresponding to the training start time period, the robot's state is initialized to start or reset the training task.

[0110] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0111] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive resampling method for robot motion control training, characterized in that, include: Based on a pre-built robot motion dataset, motion segment-level sampling is performed to determine the target motion segment from the motion dataset according to a preset first sampling distribution; For the target action segment, time-segment sampling is performed to determine the training start time segment from the target action segment based on a second sampling distribution that is dynamically updated based on historical failure information; Based on the reference robot state corresponding to the training start time period, the robot's state is initialized to start or reset the training task.

2. The adaptive resampling method for robot motion control training according to claim 1, characterized in that, The process of constructing the first sampling distribution includes: Obtain historical failure information for each action segment in the robot motion dataset; Based on the historical failure information of each action segment, calculate the first failure sampling probability corresponding to each action segment; Based on the historical frequency of each action segment being selected in historical training, the first novelty sampling probability corresponding to each action segment is calculated. The first sampling distribution is obtained by weighting and mixing the first failure sampling probability, the first novelty sampling probability, and the uniform sampling probability; wherein, the uniform sampling probability is defined as the probability that each action segment is selected for training is equal.

3. The adaptive resampling method for robot motion control training according to claim 2, characterized in that, The step of weighting and mixing the first failure sampling probability, the first novelty sampling probability, and the uniform sampling probability to obtain the first sampling distribution includes: In the initial training phase, the first sampling distribution is configured to contain only the uniform sampling probability; During the training transition phase, the weights of the first failure sampling probability and the first novelty sampling probability are gradually increased from zero to a preset target weight, and then weighted and mixed with the uniform sampling probability to obtain the first sampling distribution.

4. The adaptive resampling method for robot motion control training according to claim 1, characterized in that, The step of performing time-segment sampling on the target action segment to determine the training start time segment from the target action segment based on a second sampling distribution dynamically updated based on historical failure information includes: The target action segment is divided into multiple consecutive time periods according to the time sequence; Based on the failure events that occurred during the historical training over the multiple consecutive time periods, historical failure information for the multiple consecutive time periods is generated and dynamically updated. The second sampling distribution is generated based on the updated historical failure information; The training start time period is determined from the plurality of consecutive time periods based on the second sampling distribution.

5. The adaptive resampling method for robot motion control training according to claim 4, characterized in that, After determining the training start time period from the plurality of consecutive time periods according to the second sampling distribution, the method further includes: Determine the target time point within the training start time period; The robot state corresponding to the target time point in the robot motion dataset is used as the reference robot state.

6. The adaptive resampling method for robot motion control training according to claim 1, characterized in that, Before performing motion fragment-level sampling based on the pre-built robot motion dataset, the method further includes: If the total number of action segments in the action dataset exceeds the total number of parallel training environments, a subset of actions is determined from the action dataset as an active action pool according to the first sampling distribution; wherein the size of the action subset is smaller than the total number of action segments in the action dataset but larger than the total number of parallel training environments. The step of determining the target action segment from the action dataset according to a preset first sampling distribution includes: The target action segment is determined from the active action pool.

7. The adaptive resampling method for robot motion control training according to any one of claims 1-6, characterized in that, After initializing the robot's state based on the reference robot state corresponding to the training start time period to start or reset the training task, the method further includes: In the event of a training task failure, record the failed action segment that caused the failure and the corresponding training start time. Based on the failed action segments and their corresponding training start time periods, update the historical failure information used to calculate the first sampling distribution and the historical failure information used to calculate the second sampling distribution.

8. An adaptive resampling device for robot motion control training, characterized in that, include: The first sampling module is used to perform motion segment-level sampling based on a pre-built robot motion dataset, so as to determine the target motion segment from the motion dataset according to a preset first sampling distribution; The second sampling module is used to perform time-segment-level sampling on the target action segment in order to determine the training start time segment from the target action segment according to the second sampling distribution that is dynamically updated based on historical failure information. The training module is used to perform state initialization on the robot based on the reference robot state corresponding to the training start time period, so as to start or reset the training task.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the adaptive resampling method for robot motion control training as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the adaptive resampling method for robot motion control training as described in any one of claims 1 to 7.

Images