Robot imitation learning method based on stage-aware visual motion representation learning

Abstract

The application discloses a robot imitation learning method based on stage-aware visual motion representation learning, and relates to the technical field of robot intelligent control and artificial intelligence.The application realizes the collaborative modeling of visual information and task structure information by organically combining task key stage information, task progress information and visual motion representation learning process, improves the perception ability of the robot to the task key state and semantic structure, and thus enhances the decision accuracy of the strategy model and the stability of task execution.The application can effectively improve the utilization efficiency of task-related information in visual features, realize the collaborative optimization of task structure information and visual representation learning, and continuously obtain performance improvement with the increase of the training data size, and thus embodies good data utilization efficiency and expansion capability.

Description

Technical Field

[0001] This invention relates to the fields of robot intelligent control and artificial intelligence technology, specifically to a robot imitation learning method based on stage-based perceptual visual motion representation learning. Background Technology

[0002] With the development of robotics and artificial intelligence, visual perception-based robot imitation learning methods have been widely applied to robotic arm grasping, assembly, handling, and other embodied intelligent maneuvering tasks. These methods typically enable robots to complete complex tasks by learning the state-action mapping relationship from expert demonstration data. In recent years, constructing visual motion control strategies using large-scale pre-trained visual models has become an important research direction in the field of robot learning. Compared to end-to-end control methods trained from scratch, pre-trained visual models can provide rich visual prior knowledge, improve data utilization efficiency, and reduce the teaching cost required for robot policy learning. However, existing robot imitation learning methods based on pre-trained visual representations generally achieve good control performance in the training environment, but often exhibit significant performance degradation when the testing environment changes. For example, when the lighting conditions, background texture, appearance of the target object, or distribution of interference in the scene change, the robot strategy is prone to making incorrect decisions, leading to task failure.

[0003] The inventors discovered the following shortcomings in the existing technology:

[0004] (1) Most robot imitation learning methods mainly learn the direct mapping relationship between observation information and control actions, lack the ability to model the internal stage structure and semantic information in the task execution process, and are prone to relying on accidental visual features in the training data, thereby reducing the ability to adapt to unknown environments.

[0005] (2) Existing visual motion control methods usually directly use high-dimensional visual features for policy learning, which makes it difficult to effectively distinguish key areas related to task execution from irrelevant background information, resulting in the policy model being affected by environmental changes.

[0006] (3) Some methods enhance the performance of long time-series tasks by introducing complex temporal modeling networks, but they often require high computational overhead and it is difficult to establish an effective correlation between the task progress process and visual representation, which limits the generalization ability of the model.

[0007] Therefore, how to improve the robot's visual motion control strategy's ability to perceive information at key stages of the task, enhance the visual representation's ability to express task-related information, and improve the robot's generalization performance and control robustness in distributed offset environments without significantly increasing model complexity has become a pressing technical problem in this field. Summary of the Invention

[0008] The purpose of this invention is to provide a robot imitation learning method based on stage-aware visual motion representation learning, so as to solve the technical problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a robot imitation learning method based on stage-aware visual motion representation learning, comprising the following steps:

[0010] S1: Construct a robot teaching data acquisition and preprocessing module to uniformly organize multi-view visual observations, ontological state information and expert action data during robot operation, and establish a time synchronization relationship between visual observations and action labels to provide basic data support for subsequent keyframe extraction, visual representation learning and policy training.

[0011] S2: Construct a keyframe-enhanced supervision module based on visual change perception. By calculating the degree of visual change between adjacent time step observation images, it automatically identifies key states in the trajectory and constructs a keyframe reweighted behavior cloning objective function. This enables the model to pay more attention to task phase transitions, contact state changes, and key decision nodes during training, thereby improving the policy's ability to learn key action segments.

[0012] S3: Construct a visual representation enhancement module CAP (Cross-AttentionPooling) based on cross-attention pooling. It establishes a cross-attention relationship between the learnable latent variable query vector and the high-dimensional visual token output by the visual encoder. Through an adaptive feature aggregation mechanism, it extracts key visual information related to task execution and forms a compact and discriminative global visual representation, thereby reducing the impact of background interference and environmental changes on policy learning.

[0013] S4: Construct a self-supervised task progress prediction module, which automatically generates normalized task progress labels through trajectory time and location, and uses shared visual representations to predict the current task execution progress. This makes the visual feature learning process constrained by task stage information, thereby establishing a structured correspondence between visual observation and task execution process, and improving the model's ability to perceive the overall temporal structure of the task.

[0014] S5: Construct a joint optimization mechanism based on keyframe-enhanced behavior cloning loss and task progress prediction loss. Use an end-to-end training method to collaboratively optimize the visual representation enhancement module, task progress prediction module and robot policy network. Drive visual feature aggregation weights, task progress prediction parameters and action generation parameters to be updated together through loss feedback, forming a closed-loop optimization process that includes visual perception, task structure modeling and action decision-making.

[0015] S6: Input the real-time collected robot observation information into the trained visual motion strategy model, and output robot control commands through visual representation extraction, task stage perception and action generation process to realize robot grasping, handling, insertion and assembly operations.

[0016] Preferably, step S1 involves collecting expert teaching trajectories during the robot's execution of operational tasks and constructing a teaching dataset:

[0017]

[0018] in: Indicates time step Corresponding observation information; This indicates the corresponding expert action; Indicates the length of the trajectory;

[0019] The observation information includes multi-view RGB images as well as robot joint positions, joint velocities, and gripper status;

[0020] For multi-view input scenarios, images captured by multiple cameras are organized in a preset order and synchronized with corresponding action data in time to form a unified training sample.

[0021] Preferably, step S2 includes at least the following steps:

[0022] First, read the continuous image observation data from the robot's teaching trajectory and traverse the trajectory in chronological order;

[0023] For each time step, the degree of visual change between the current observed image and the observed image at the previous time step is calculated to measure whether the robot's state has changed significantly.

[0024] In multi-view scenarios:

[0025] First, images captured by multiple cameras are stitched together in a preset order, and then the visual differences between adjacent time steps are calculated.

[0026] The visual changes in the entire trajectory were then sorted, and a percentile threshold strategy was used to automatically select the time steps with the largest changes as keyframes.

[0027] To ensure that the starting and ending states of the trajectory can be effectively preserved, the first and last frames of the trajectory are forcibly set as keyframes.

[0028] After keyframe selection is completed, keyframe identification information is established for each time step, and different weights are assigned according to whether it belongs to a keyframe during training.

[0029] For keyframe samples, increase their contribution to the loss function; for ordinary samples, maintain the default weights.

[0030] This approach enables the strategy model to focus more on key decision states such as approaching the target, grasping and contacting, releasing the object, and inserting and assembling during the training process, thereby improving the robot's ability to learn key action segments.

[0031] The keyframe identification information is generated and saved offline before training begins, and is directly read and used during model training. It does not participate in the network parameter update process, nor does it introduce additional trainable parameters.

[0032] Preferably, step S3 includes the following steps:

[0033] First, the multi-view images collected by the robot are input into a pre-trained visual encoder to extract high-dimensional visual features from the images.

[0034] The feature map output by the visual encoder is then unfolded into multiple visual feature units, and a set of learnable latent variable query vectors is constructed as information aggregation carriers.

[0035] The correlation between latent variable query vectors and visual feature units is calculated by using a cross-attention mechanism, enabling different latent variables to actively focus on information in different regions of the image.

[0036] During the attention calculation process, regions related to robot task execution (including target object region, end effector region of robotic arm, and contact region) will receive higher attention weights, while background regions and regions unrelated to the task will receive lower weights.

[0037] Then, attention weights are used to weight and aggregate the visual features to form multiple compact latent variable representations;

[0038] Finally, the aggregated latent variable representation is pooled to obtain a global visual representation, which is then simultaneously input into the task progress prediction module and the robot policy network.

[0039] Through the above methods, the present invention can achieve adaptive filtering and aggregation of task-related visual information under the condition of freezing the visual encoder, and reduce the impact of background changes and lighting disturbances on the policy learning process.

[0040] Preferably, the visual encoder includes a ResNet pre-trained model, a MoCo pre-trained model, or other basic visual models, and the visual encoder keeps its parameters frozen during training.

[0041] Preferably, step S4 includes the following steps:

[0042] For each expert teaching trajectory, a task progress label is automatically generated based on the relative position of the current time step in the entire trajectory. The task progress label reflects the current task stage of the robot and can be obtained automatically without manual annotation.

[0043] During training, the global visual features output by the visual representation enhancement module are input into the lightweight process prediction network, which predicts the task execution progress corresponding to the current observation. The process prediction network adopts a multilayer perceptron structure and the prediction results are limited to a preset range by normalizing the output.

[0044] The prediction results are then compared with the automatically generated task progress labels, the prediction error is calculated, and the error is fed back to the shared visual representation layer as an auxiliary supervision signal. Since there is a correspondence between the task progress labels and visual observations, the model needs to learn the task stage features from visual information, rather than just relying on action imitation for learning.

[0045] By continuously optimizing the task progress prediction error, the shared visual representation gradually encodes information about the stages of approaching the target, contacting the target, performing operations, and completing the task, thereby establishing a structured correlation between visual observation and the task execution process. This auxiliary supervision mechanism is only enabled during the training phase and requires no additional inference overhead during the actual deployment of the robot.

[0046] Preferably, step S5 includes the following steps:

[0047] After completing the construction of the keyframe enhancement supervision module, visual representation enhancement module, and task progress prediction module, the three are integrated into the robot visual motion strategy learning framework for joint training.

[0048] During training:

[0049] First, the robot's observation data is input into the frozen visual encoder to extract basic visual features;

[0050] Then, the visual features are adaptively aggregated using the CAP visual representation enhancement module to obtain a global visual representation containing task-related information.

[0051] The global visual representation is input into the robot policy network to generate action prediction results on the one hand, and into the task progress prediction module to generate task stage prediction results on the other hand.

[0052] For the action prediction branch, the behavior cloning loss is calculated based on the difference between the expert-taught action and the model-predicted action. Combined with the keyframe weight information generated by the keyframe enhancement supervision module, the training samples corresponding to key states are given higher loss weights, so that the model pays more attention to key decision states during parameter update.

[0053] For the task process prediction branch, the task process prediction loss is calculated based on the difference between the automatically generated task process label and the model prediction result, and this loss is used as an auxiliary supervision signal to be applied to the shared visual representation layer.

[0054] Subsequently, the behavior cloning loss and the task progress prediction loss are weighted and fused according to a preset ratio to form a unified training objective;

[0055] The gradient information of the loss function with respect to the network parameters is calculated using the backpropagation algorithm, and the model parameters are updated using the gradient descent optimization strategy.

[0056] During the parameter update process, the policy network parameters, the attention parameters in the CAP module, and the task progress prediction module parameters are updated synchronously, enabling the visual feature aggregation process, the task stage perception process, and the action generation process to be optimized in a coordinated manner.

[0057] Through joint training, keyframe augmentation supervision is responsible for highlighting the importance of key decision states, task progress prediction is responsible for providing global task stage constraints, and the CAP module is responsible for filtering task-related visual information. The three work together to share the visual representation learning process, thereby establishing a unified representation space between visual information and task structure.

[0058] During training, the model performance is periodically evaluated using the validation set, and the parameters of the model with the best validation performance are saved as the final robot control model.

[0059] Preferably, step S6 includes at least the following steps:

[0060] When the robot actually performs a task, the optimal model parameters obtained during the training phase are loaded, and an online inference system for the robot is established. During system operation, one or more visual sensors installed in the robot's workspace acquire environmental images in real time, while simultaneously obtaining robot joint states, end effector states, and other proprioceptive information. The acquired observation data is preprocessed and then input into the trained visual motion strategy model. First, a pre-trained visual encoder is used to extract visual features from the input images, and then the CAP visual representation enhancement module adaptively aggregates the visual features to extract key visual information relevant to the current task from the complex scene.

[0061] Subsequently, the shared visual representations learned during the training phase are used to complete task state understanding, and then input into the policy network for action reasoning. The policy network outputs robot control commands based on the current observation state. These robot control commands include end-effector pose control, joint control, gripper opening and closing control, or other robot actions. The robot controller drives the robot to complete the corresponding operation task based on the output robot control commands.

[0062] As the robot continuously receives new environmental observation data, the aforementioned processes of visual perception, feature aggregation, action reasoning, and control execution are cyclically performed, forming a real-time closed-loop control system. When the robot is in an environment with distributed shifts such as changes in lighting, background, target appearance, or scene disturbances, the model, having established a structured correlation between visual information and task-stage information during the training phase, can more accurately identify critical task states and maintain stable action decision-making capabilities. This improves the robot's task success rate and control robustness in unknown environments. This invention can be applied to robot grasping, handling, stacking, insertion, assembly, and other embodied intelligent operation tasks, and can be deployed on various robot platforms such as industrial robotic arms, service robots, collaborative robots, and mobile operation robots.

[0063] Compared with the prior art, the beneficial effects of the present invention are:

[0064] 1. This invention organically combines key task stage information, task progress information, and visual motion representation learning process to achieve collaborative modeling of visual information and task structure information, thereby improving the robot's ability to perceive key task states and semantic structures, and thus enhancing the decision-making accuracy of the strategy model and the stability of task execution.

[0065] 2. This invention can effectively improve the utilization efficiency of task-related information in visual features, realize the synergistic optimization of task structure information and visual representation learning, and continuously improve performance as the scale of training data increases, demonstrating good data utilization efficiency and scalability.

[0066] 3. This invention achieves high performance without retraining the pre-trained visual encoder, reducing training and deployment costs while ensuring computational efficiency, and has good prospects for engineering applications. Attached Figure Description

[0067] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a schematic diagram of the overall model framework of the present invention;

[0069] Figure 2 This is a schematic diagram of the keyframe detection reweighting process of the present invention;

[0070] Figure 3 This is a schematic diagram of the KF-BC loss+Prog loss auxiliary loss control of the present invention;

[0071] Figure 4 This is a schematic diagram of the learnable cross-attention pooling module CAP of the present invention;

[0072] Figure 5 This is a schematic diagram illustrating the initial state of datasets inside and outside different task domains in this invention;

[0073] Figure 6 This is a visualization of the attention stages in the task process outside different task domains according to the present invention. Detailed Implementation

[0074] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0075] To address the shortcomings of existing robot imitation learning methods, such as insufficient generalization ability in distributed offset environments, difficulty in effectively utilizing task stage information, and the presence of a large amount of task-irrelevant interference information in visual representations, this invention provides a robot imitation learning method based on stage-aware visual motion representation learning.

[0076] Specifically as follows:

[0077] Please see Figure 1 A robot imitation learning method based on stage-aware visual motion representation learning includes the following steps:

[0078] S1: Construct a robot teaching data acquisition and preprocessing module to uniformly organize multi-view visual observations, ontological state information and expert action data during robot operation, and establish a time synchronization relationship between visual observations and action labels to provide basic data support for subsequent keyframe extraction, visual representation learning and policy training.

[0079] S2: Construct a keyframe-enhanced supervision module based on visual change perception. By calculating the degree of visual change between adjacent time step observation images, it automatically identifies key states in the trajectory and constructs a keyframe reweighted behavior cloning objective function. This enables the model to pay more attention to task phase transitions, contact state changes, and key decision nodes during training, thereby improving the policy's ability to learn key action segments.

[0080] S3: Construct a visual representation enhancement module CAP (Cross-AttentionPooling) based on cross-attention pooling. It establishes a cross-attention relationship between the learnable latent variable query vector and the high-dimensional visual token output by the visual encoder. Through an adaptive feature aggregation mechanism, it extracts key visual information related to task execution and forms a compact and discriminative global visual representation, thereby reducing the impact of background interference and environmental changes on policy learning.

[0081] S4: Construct a self-supervised task progress prediction module, which automatically generates normalized task progress labels through trajectory time and location, and uses shared visual representations to predict the current task execution progress. This makes the visual feature learning process constrained by task stage information, thereby establishing a structured correspondence between visual observation and task execution process, and improving the model's ability to perceive the overall temporal structure of the task.

[0082] S5: Construct a joint optimization mechanism based on keyframe-enhanced behavior cloning loss and task progress prediction loss. Use an end-to-end training method to collaboratively optimize the visual representation enhancement module, task progress prediction module and robot policy network. Drive visual feature aggregation weights, task progress prediction parameters and action generation parameters to be updated together through loss feedback, forming a closed-loop optimization process that includes visual perception, task structure modeling and action decision-making.

[0083] S6: Input the real-time collected robot observation information into the trained visual motion strategy model, and output robot control commands through visual representation extraction, task stage perception and action generation process to realize robot grasping, handling, insertion and assembly operations.

[0084] Based on the above steps, the following substantive settings are proposed:

[0085] The overall network framework of this invention is as follows: Figure 1 As shown, this paper addresses the problems of insufficient generalization ability of existing robot imitation learning methods in distributed offset environments, difficulty in effectively utilizing task stage information, and the presence of a large amount of task-irrelevant interference information in visual representations.

[0086] Specifically, this invention designs a structured vision-motor strategy learning framework that jointly models visual information and temporal signals at the representation layer.

[0087] Unlike methods that rely on complex long-term time-series modeling, this invention introduces lightweight but structured time-series signals, including keyframe enhancements such as... Figure 2 The task progress modeling and control method shown is as follows: Figure 3 As shown, this guides the feature learning process. Simultaneously, a learnable attention-based feature pooling CAP module is designed, such as... Figure 4As shown, task-related visual information is adaptively aggregated. The method proposed in this invention can effectively capture the inherent structure of the task, thereby improving the robustness and generalization ability of the strategy.

[0088] Imitation learning behavior cloning

[0089] Suppose there exists an expert policy π* that outputs robot actions a∈A based on proprioceptive input p∈P and visual input o∈O. Executing this policy yields a series of expert demonstration datasets. Each of the N trajectories contains T time steps of observations and their corresponding actions. The learner's goal is to utilize a parameterized strategy. Approximating π*. This invention employs the classic behavioral cloning method: training on expert demonstration data. Minimize the mean squared error between expert actions and policy-predicted actions:

[0090] (1)

[0091] in, This represents the behavioral cloning loss function, which measures the gap between the current policy model and the expert demonstration data. The smaller the loss, the more the model tends to output expert actions under the same observations. This represents the trainable parameters of the policy model. This represents the expert demonstration dataset. Indicates from dataset Sample a time step sample. Indicates by parameters Control strategy model. This indicates that the model sees the observation At that time, output expert actions The conditional probability.

[0092] In continuous motion control, we incorporate the robot's body state. As another part of the observations, using a fixed-variance Gaussian strategy, the negative log-likelihood can be reduced to a form that minimizes the squared error:

[0093] (2)

[0094] Among them, the function This represents a fixed pre-trained visual representation, whose function is to transform the original visual stream... Convert to a global feature vector. and Indicates the first Trajectory The robot's current state, expert actions, and RGB image observations are based on visual representation. and the robot's body state Predict actions and make the predicted actions as close as possible to the expert's actions. Following general practice, this invention will Implemented as a lightweight multilayer perceptron (MLP), it consists of a 4-layer network. ReLU activation is used between layers, and the output layer uses tanh activation to constrain the output range. Policy prediction has a fixed variance σ. 2 The mean μ of the Gaussian distribution, and from the truncated Gaussian distribution N T (μ, σ 2 Sampling action in )

[0095] Keyframe-aware behavior cloning

[0096] Because a large number of consecutive frames in robot operation tasks correspond only to smooth motion processes with small information increments, and success or failure is often dominated by a few "critical decision states" (such as alignment, grasping / releasing, contact, and insertion moments), standard behavior cloning is easily dominated by a large number of redundant frames during optimization, leading to underfitting of critical decision states. To automatically extract a set of key frames that can characterize significant state changes from the trajectory (…),… Figure 2 This invention proposes a keyframe-aware reweighting strategy that reveals the effect of improving key states on optimization during training.

[0097] Keyframe detection and mask construction: for trajectory Assign a binary indicator variable to each time step: This is used to mark whether a time step is a keyframe. First, define the change fraction between adjacent frames:

[0098] (3)

[0099] in, Distance metrics based on RGB differences (e.g.) distance), and For two adjacent RGB observations. Keyframe selection strategies include:

[0100] 1) Percentile strategy: Select the percentiles with the highest percentage changes. The frame;

[0101] 2) Peak detection strategy: In Local maxima are detected in the sequence, and minimum margin constraints are applied;

[0102] 3) To ensure trajectory coverage, the first and last frames are always marked as keyframes.

[0103] Keyframe reweighting objective: In standard behavior cloning, all samples are treated equally. To highlight the importance of keyframes, this invention defines a new weighting function:

[0104] (4)

[0105] in Indicates time step Training weights corresponding to the samples, This represents the additional amplification factor for the keyframe loss. When When =0, =1 This sample is trained using the ordinary behavioral cloning loss; when When =1, Keyframe samples receive higher weights during training. A weighted behavior clone target is then constructed.

[0106] (5)

[0107] This method employs a data-dependent importance weighting mechanism, enabling the model to focus more on critical moments of significant state changes during training. It introduces no additional supervision signals, does not alter the model structure, and can be seamlessly integrated into standard behavior cloning frameworks, exhibiting good versatility and scalability.

[0108] Self-supervised task progress prediction

[0109] The goal of this invention is not to model long-sequence temporal dependencies, but rather to establish a behavior clone resistant to environmental interference. While keyframe reweighting can enhance a policy's ability to model local decision states, standard behavior clones still lack explicit modeling of the overall temporal structure of the task. Specifically, behavior clones only perform action regression step-by-step, ignoring the stage semantic differences at different time steps during task execution. This makes it difficult for the policy to develop a consistent stage awareness in long-sequence tasks. To overcome these shortcomings, this invention introduces a self-supervised auxiliary task: task progress prediction, used to learn continuous representations related to task stages, thereby improving the policy's temporal understanding.

[0110] Task process definition and supervision signal construction: Given an expert demonstration dataset:

[0111]

[0112] The task progress is defined as the normalized position of the time step in the trajectory:

[0113] (6)

[0114] in, This indicates the preset maximum trajectory length or maximum number of task steps; This represents the truncation function, used to restrict the normalization result to the interval ([0,1]).

[0115] This process label does not require manual annotation and can be directly generated from the trajectory, thus belonging to self-supervised signals. Intuitively, this definition encourages the model to learn the representational differences between different stages (such as approaching the target, contact, completion of the operation, etc.).

[0116] Process prediction head: Introducing a lightweight process prediction head on top of the shared visual representation of the policy network. .

[0117] Let the output features of the visual encoder be The process prediction is as follows:

[0118] (7)

[0119] in, It is a multilayer perceptron, and the output is constrained to [0,1] using the Sigmoid function.

[0120] In its implementation, this invention adopts a structure of "LayerNorm + two-layer nonlinear mapping + linear regression" to minimize the introduction of additional parameters while ensuring that the auxiliary task effectively constrains the backbone representation. This design ensures that the model must learn task stage information from visual observations, rather than relying on explicit temporal signals for "shortcut learning".

[0121] Joint optimization objective: Process prediction uses mean squared error loss.

[0122] (8)

[0123] Finally, the multi-task loss function for constructing the training objective is:

[0124] (9)

[0125] in, This is a weighting coefficient used to balance the influence between auxiliary tasks and the main task.

[0126] Visual representation enhancement based on learnable feature attention pooling (CAP)

[0127] In robotic manipulation tasks, different regions in an image often exhibit significantly different importance for decision-making. For example, the contact area between the end effector and the target object often contains more critical task information than the background area. However, standard visual encoders typically generate feature representations through global average pooling, which struggles to distinguish the importance of different spatial regions, thus weakening their ability to model key visual cues. To address this, this invention introduces a CAP feature pooling module on top of the feature map output by the visual encoder. By learning spatial attention weights, it weights and aggregates features from different regions to generate a more discriminative global representation.

[0128] CAP feature pooling: Assume the visual encoder outputs a feature map as follows: .

[0129] First, attention weights are generated for spatial location. Normalize it using softmax as follows:

[0130] (10)

[0131] in, This represents a lightweight attention network.

[0132] Then, the global feature representation is obtained through weighted pooling:

[0133] (11)

[0134] in, Indicates position Attention weights are used, a process that is an adaptive feature selection mechanism in the spatial dimension, enabling the model to focus on regions that are highly relevant to task decisions.

[0135] To verify the effectiveness of the robot operation method proposed in this invention, which is based on multi-view visual representation enhancement and key stage perception, in-distribution (ID) and out-of-distribution (OOD) experiments were conducted on the MetaWorld multi-task operation benchmark, and compared with typical imitation learning methods such as ACT, Diffusion Policy (DP), and TE-PVR.

[0136] The experiment selected seven representative robotic arm operation tasks: Bin-Picking, Assembly, Box-Close, Button-Press, Coffee-Pull, Disassemble, and Drawer-Open, using the average success rate of these tasks as the evaluation metric. Experimental results show that, under the in-domain test environment, the method of this invention achieved an average success rate of 86.7%, significantly better than ACT's 47.3%, DP's 63.9%, and TE-PVR's 78.4%. Specifically, it achieved 100% success rate on the Button-Press and Drawer-Open tasks, and also achieved optimal performance on the Bin-Picking, Coffee-Pull, and Disassemble tasks, demonstrating that this invention can effectively enhance the robot's perception of key visual information and task status, improving the accuracy and stability of the operation strategy. In an off-domain testing environment, the generalization ability of the model was evaluated by introducing unseen environmental changes such as illumination variations, background perturbations, and target position shifts. The method of this invention achieved an average success rate of 53.6%, which is significantly higher than ACT's 1.9%, DP's 14.9%, and TE-PVR's 34.9%. The success rates on the Drawer-Open and Button-Press tasks reached 91.8% and 89.1%, respectively, indicating that the invention can learn more robust visual representations and effectively reduce the impact of environmental distribution changes on policy performance.

[0137] Furthermore, by comparing the experimental results within and outside the domain, it was found that the performance of ACT, DP, and TE-PVR decreased by 45.4 percentage points, 49.0 percentage points, and 43.5 percentage points respectively in the outside environment, while the performance of the present invention decreased by only 33.4 percentage points, showing the smallest performance degradation and verifying the significant advantage of the present invention in cross-scenario generalization.

[0138] To verify the ability of this invention to utilize out-of-domain samples, out-of-domain data containing illumination perturbations, background changes, and target position changes were further introduced during the training phase. The results showed that the average success rates of ACT, DP, TE-PVR, and the method of this invention improved by 0.1 percentage points, 1.7 percentage points, 2.5 percentage points, and 3.6 percentage points, respectively. Among these, the method of this invention achieved the largest performance gain, increasing from 86.7% to 90.3%, indicating that it can more fully utilize diverse training data in complex environments, improving the model's environmental adaptability and generalization performance. Furthermore, to analyze the contribution of each component module to the overall performance, ablation experiments were conducted on the feature pooling module CAP, the keyframe selection module KF, and the progress prediction module Progress.

[0139] When all three modules were removed, the model's average success rate was only 57.3% in in-domain and 15.4% in out-of-domain environments. Introducing only the CAP module improved the success rates to 82.7% and 51.1% in in-domain and out-of-domain environments, respectively; introducing only the KF module improved them to 62.7% and 38.7%; and introducing only the Progress module improved them to 64.4% and 39.8%. When CAP and KF, and CAP and Progress were used in combination, the model performance was further improved. The model achieved its best performance when all three modules were enabled simultaneously, with an average success rate of 86.7% in in-domain and 53.6% out-of-domain.

[0140] The above experimental results fully demonstrate that the feature pooling module, keyframe selection module, and progress prediction module proposed in this invention can effectively improve the robot's operational performance, and the three have a good synergistic gain effect, which together improves the robot's ability to complete tasks in complex environments and its cross-scene generalization ability.

[0141] Please refer to Tables 1-5 below for details:

[0142] Table 1. Test results of different methods in each task domain

[0143]

[0144] Table 2 Ablation experiments using this method

[0145]

[0146]

[0147] Table 3. Out-of-domain experimental results for different methods

[0148]

[0149] Table 4 Comparison of the decrease in average success rate inside and outside the domain for different methods

[0150] Method ID ↑ OOD ↑ Drop ↓ ACT 0.473 0.019 0.454 DP 0.639 0.149 0.490 TE-PVR 0.784 0.349 0.435 Ours 0.867 0.533 0.334

[0151] Table 5. Experimental results after training with different methods for adding out-of-domain data.

[0152]

[0153] In summary:

[0154] The verification results of the embodiments show that the average success rate of the present invention in multiple typical robot operation tasks reaches 86.7%, which is up to 39.4% higher than the existing mainstream imitation learning methods; it still maintains a task success rate of 53.3% in the distributed offset environment, which is 18.4% higher than the existing best comparison method, and the performance degradation is significantly reduced, demonstrating excellent generalization ability and environmental adaptability.

[0155] This invention aims to enhance the robot's ability to perceive key stages and semantic structures during task execution, improve the ability of visual representation to express task-related information, and reduce the impact of environmental changes on strategy decisions. This, in turn, improves the robot's task execution success rate and control robustness in scenarios with varying lighting, background changes, and the presence of distractions. Furthermore, this invention achieves effective modeling of key visual information and task progress information without relying on additional sensors or adding complex long-term modeling structures, thereby improving the data utilization efficiency and cross-scenario generalization ability of robot imitation learning methods.

[0156] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A robot imitation learning method based on stage-based perceptual visual motion representation learning, characterized in that: Includes the following steps: S1: Construct a robot teaching data acquisition and preprocessing module to uniformly organize multi-view visual observations, ontological state information and expert action data during robot operation, and establish a time synchronization relationship between visual observations and action labels to provide basic data support for subsequent keyframe extraction, visual representation learning and policy training. S2: Construct a keyframe-enhanced supervision module based on visual change perception. By calculating the degree of visual change between adjacent time step observation images, it automatically identifies key states in the trajectory and constructs a keyframe reweighted behavior cloning objective function. This enables the model to pay more attention to task phase transitions, contact state changes, and key decision nodes during training, thereby improving the policy's ability to learn key action segments. S3: Construct a visual representation enhancement module (CAP) based on cross-attention pooling. Establish a cross-attention relationship between the learnable latent variable query vector and the high-dimensional visual token output by the visual encoder. Extract key visual information related to task execution through an adaptive feature aggregation mechanism to form a compact and discriminative global visual representation, thereby reducing the impact of background interference and environmental changes on policy learning. S4: Construct a self-supervised task progress prediction module, which automatically generates normalized task progress labels through trajectory time and location, and uses shared visual representations to predict the current task execution progress. This makes the visual feature learning process constrained by task stage information, thereby establishing a structured correspondence between visual observation and task execution process, and improving the model's ability to perceive the overall temporal structure of the task. S5: Construct a joint optimization mechanism based on keyframe-enhanced behavior cloning loss and task progress prediction loss. Use an end-to-end training method to collaboratively optimize the visual representation enhancement module, task progress prediction module and robot policy network. Drive visual feature aggregation weights, task progress prediction parameters and action generation parameters to be updated together through loss feedback, forming a closed-loop optimization process that includes visual perception, task structure modeling and action decision-making. S6: Input the real-time collected robot observation information into the trained visual motion strategy model, and output robot control commands through visual representation extraction, task stage perception and action generation process to realize robot grasping, handling, insertion and assembly operations.

2. The robot imitation learning method based on stage-aware visual motion representation learning according to claim 1, characterized in that: S1 involves collecting expert teaching trajectories during the robot's execution of operational tasks and constructing a teaching dataset: in: Indicates time step Corresponding observation information; This indicates the corresponding expert action; Indicates the length of the trajectory; The observation information includes multi-view RGB images as well as robot joint positions, joint velocities, and gripper status; For multi-view input scenarios, images captured by multiple cameras are organized in a preset order and synchronized with corresponding action data in time to form a unified training sample.

3. The robot imitation learning method based on stage-aware visual motion representation learning according to claim 1, characterized in that: S2 includes at least the following steps: First, read the continuous image observation data from the robot's teaching trajectory and traverse the trajectory in chronological order; For each time step, the degree of visual change between the current observed image and the observed image at the previous time step is calculated to measure whether the robot's state has changed significantly. In multi-view scenarios: First, images captured by multiple cameras are stitched together in a preset order, and then the visual differences between adjacent time steps are calculated. The visual changes in the entire trajectory were then sorted, and a percentile threshold strategy was used to automatically select the time steps with the largest changes as keyframes. To ensure that the starting and ending states of the trajectory can be effectively preserved, the first and last frames of the trajectory are forcibly set as keyframes. After keyframe selection is completed, keyframe identification information is established for each time step, and different weights are assigned according to whether it belongs to a keyframe during training. For keyframe samples, increase their contribution to the loss function; for ordinary samples, maintain the default weights. This approach enables the strategy model to focus more on key decision states such as approaching the target, grasping and contacting, releasing the object, and inserting and assembling during the training process, thereby improving the robot's ability to learn key action segments. The keyframe identification information is generated and saved offline before training begins, and is directly read and used during model training. It does not participate in the network parameter update process, nor does it introduce additional trainable parameters.

4. The robot imitation learning method based on stage-aware visual motion representation learning according to claim 3, characterized in that: S3 includes the following steps: First, the multi-view images collected by the robot are input into a pre-trained visual encoder to extract high-dimensional visual features from the images. The feature map output by the visual encoder is then unfolded into multiple visual feature units, and a set of learnable latent variable query vectors is constructed as information aggregation carriers. The correlation between latent variable query vectors and visual feature units is calculated by using a cross-attention mechanism, enabling different latent variables to actively focus on information in different regions of the image. During attention calculation, regions related to the robot's task execution will receive higher attention weights, while background regions and regions unrelated to the task will receive lower weights. Then, attention weights are used to weight and aggregate the visual features to form multiple compact latent variable representations; Finally, the aggregated latent variable representations are pooled to obtain a global visual representation, which is then simultaneously input into the task progress prediction module and the robot policy network.

5. The robot imitation learning method based on stage-aware visual motion representation learning according to claim 4, characterized in that: The visual encoder includes a ResNet pre-trained model, a MoCo pre-trained model, or other basic visual models, and the parameters of the visual encoder are kept frozen during training.

6. The robot imitation learning method based on stage-aware visual motion representation learning according to claim 4, characterized in that: S4 includes the following steps: For each expert teaching trajectory, a task progress label is automatically generated based on the relative position of the current time step in the entire trajectory. The task progress label reflects the current task stage of the robot and can be obtained automatically without manual annotation. During training, the global visual features output by the visual representation enhancement module are input into the lightweight process prediction network, which predicts the task execution progress corresponding to the current observation. The process prediction network adopts a multilayer perceptron structure and the prediction results are limited to a preset range by normalizing the output. The prediction results are then compared with the automatically generated task progress labels, the prediction error is calculated, and the error is fed back to the shared visual representation layer as an auxiliary supervision signal. Since there is a correspondence between the task progress labels and visual observations, the model needs to learn the task stage features from visual information, rather than just relying on action imitation for learning. By continuously optimizing the task progress prediction error, the shared visual representation gradually encodes stage information, thereby establishing a structured relationship between visual observation and the task execution process.

7. The robot imitation learning method based on stage-aware visual motion representation learning according to claim 6, characterized in that: S5 includes the following steps: After completing the construction of the keyframe enhancement supervision module, visual representation enhancement module, and task progress prediction module, the three are integrated into the robot visual motion strategy learning framework for joint training. During training: First, the robot's observation data is input into the frozen visual encoder to extract basic visual features; Then, the visual features are adaptively aggregated using the CAP visual representation enhancement module to obtain a global visual representation containing task-related information. The global visual representation is input into the robot policy network to generate action prediction results on the one hand, and into the task progress prediction module to generate task stage prediction results on the other hand. For the action prediction branch, the behavior cloning loss is calculated based on the difference between the expert-taught action and the model-predicted action. Combined with the keyframe weight information generated by the keyframe enhancement supervision module, the training samples corresponding to key states are given higher loss weights, so that the model pays more attention to key decision states during parameter update. For the task process prediction branch, the task process prediction loss is calculated based on the difference between the automatically generated task process label and the model prediction result, and this loss is used as an auxiliary supervision signal to be applied to the shared visual representation layer. Subsequently, the behavior cloning loss and the task progress prediction loss are weighted and fused according to a preset ratio to form a unified training objective; The gradient information of the loss function with respect to the network parameters is calculated using the backpropagation algorithm, and the model parameters are updated using the gradient descent optimization strategy. During the parameter update process, the policy network parameters, the attention parameters in the CAP module, and the task progress prediction module parameters are updated synchronously, enabling the visual feature aggregation process, the task stage perception process, and the action generation process to be optimized in a coordinated manner. During training, the model performance is periodically evaluated using the validation set, and the parameters of the model with the best validation performance are saved as the final robot control model.

8. The robot imitation learning method based on stage-aware visual motion representation learning according to claim 7, characterized in that: S6 includes at least the following steps: First, a pre-trained visual encoder is used to extract visual features from the input image, and then the visual features are adaptively aggregated by the CAP visual representation enhancement module to extract key visual information related to the current task from the complex scene. Subsequently, the shared visual representations learned during the training phase are used to complete task state understanding, and then input into the policy network for action reasoning. The policy network outputs robot control commands based on the current observation state, and the robot controller drives the robot to complete the corresponding operation task according to the output robot control commands.

Images