A robot imitation learning method based on intention understanding and subconscious execution

Abstract

The application provides a robot imitation learning method based on intention understanding and subconscious execution, comprising the following steps: collecting a plurality of robot demonstration trajectory data to form a data set, calculating an intention influence measure according to the robot demonstration trajectory data; smoothing the intention influence measure by using a smoothing filter, calculating a dynamic down-sampling frequency, and down-sampling the robot demonstration trajectory data to generate an awareness trajectory data set; establishing a deep learning-based action prediction model, and training the deep learning-based action prediction model by using the awareness trajectory data set; inputting current observation data, sensor data and action data into the trained action prediction model to output a predicted future action sequence, and predicting an execution action at a next moment through a cognitive unloading mechanism and a weighted cumulative prediction strategy; feeding back the predicted execution action at the next moment to a robot executor to obtain new observation data, and repeating the above steps until a target task is completed.

Description

Technical Field

[0001] This invention belongs to the field of robot imitation learning technology, and relates to a robot imitation learning method based on intention understanding and subconscious execution. Background Technology

[0002] In the field of robot control, robot imitation learning is a key technology that automatically acquires task control strategies by learning from expert demonstrations, and it is widely used in industrial production, service robots, and other fields. In complex operational tasks, robots can complete tasks such as grasping, assembly, and precision manufacturing by imitating human operation demonstrations. However, existing imitation learning methods still face many challenges in practical applications, especially in terms of execution efficiency, real-time performance, and accuracy.

[0003] Traditional robot imitation learning methods typically rely on extensive computation and multi-step prediction, especially in high-dimensional state and action spaces. The high computational resource consumption and multi-step prediction errors severely impact the efficiency and accuracy of robot task execution. Furthermore, these methods fail to effectively address the problem of redundant data, leading to the processing of large amounts of irrelevant data during task execution, further increasing the computational burden and affecting the robot's real-time response capabilities.

[0004] Currently, although action prediction models based on deep learning and transformer architectures exist and can improve prediction accuracy to some extent, these methods still face challenges such as redundant data processing and real-time computation efficiency. Balancing task accuracy and execution efficiency in dynamically changing environments remains a pressing problem to be solved.

[0005] Subconscious imitation learning, as a learning mechanism that simulates subconscious human operations, has been proposed as a solution to improve performance efficiency by reducing unnecessary computation through cognitive offloading. However, existing subconscious imitation learning methods have not yet fully addressed issues such as how to effectively reduce redundant computation and avoid the cumulative error of multi-step predictions while ensuring task accuracy. Summary of the Invention

[0006] To address the aforementioned technical problems, the purpose of this invention is to provide a robot imitation learning method based on intent understanding and subconscious execution, aiming to improve the efficiency and accuracy of robot task execution.

[0007] The present invention provides a robot imitation learning method based on intention understanding and subconscious execution, comprising:

[0008] Step 1: Collect multiple robot demonstration trajectory data in a real-world environment to form a dataset, and calculate the intention impact metric based on the robot demonstration trajectory data;

[0009] Step 2: Use a smoothing filter to smooth the intention influence metric, calculate the dynamic downsampling frequency, and downsample the robot demonstration trajectory data based on the dynamic downsampling frequency to generate an intention trajectory dataset;

[0010] Step 3: Establish a deep learning-based action prediction model and train the deep learning action prediction model using a consciousness trajectory dataset;

[0011] Step 4: During the robot's action execution phase, the current observation data, sensor data, and action data are processed through steps 1 and 2 and then input into the trained action prediction model. The model outputs the predicted future action sequence and uses a cognitive unloading mechanism and a weighted cumulative prediction strategy to predict the action to be executed at the next moment.

[0012] Step 5: Feed back the predicted action to the robot actuator, obtain new observation data, and repeat steps 4 and 5 until the target task is completed.

[0013] Furthermore, step 1 specifically includes:

[0014] Step 1.1: Collect multiple robot demonstration trajectory data in a real-world environment to form a dataset D = {τ0, τ1, ..., τ}. N}, each trajectory τ n The n = 0, 1, ..., N consists of time steps t and contains the robot's visual observation data O. (n,t) Sensor data S (n,t) and motion data A (n,t) ;

[0015] The visual observation data is image data of the robot's working environment; the sensor data includes joint position qpos. (n,t) Joint velocity qvel (n,t) With end effector torque eeft (n,t) Motion data includes target joint position α pos(n,t) and target joint velocity α qvel(n,t) ;

[0016] Step 1.2: Calculate the Intentional Influence Measurement I according to the following formula. t :

[0017]

[0018] in, It is a normalization function, and ω is a weight vector used to determine the relative importance of joints.

[0019] Furthermore, step 2 specifically includes:

[0020] Step 2.1: Smooth the intention influence measure using a Butterworth filter, as shown in the following formula:

[0021]

[0022] Where s is a complex frequency variable used to describe the frequency characteristics of the signal and the dynamic behavior of the system; m is the order of the Butterworth filter, which determines the steepness of the filter; ω c The cutoff frequency;

[0023] Step 2.2: Calculate the dynamic downsampling frequency based on the smoothed intention influence metric and the sampling frequency of the robot demonstration trajectory data:

[0024]

[0025] Among them, f ds (t) is the dynamic downsampling frequency, f d To demonstrate the raw sampling frequency of trajectory data for the robot, This is a floor operation, where M represents the time scaling factor, and f... m The target minimum sampling frequency;

[0026] Step 2.3: Downsample the original robot demonstration trajectory data according to the dynamic downsampling frequency to generate the consciousness trajectory dataset D. ds ={τ ds(0) ,τ ds(1) ,…,τ ds(N) This data is used for training subsequent action prediction models.

[0027] Furthermore, step 3 specifically includes:

[0028] Step 3.1: Calculate the consciousness trajectory τ generated by each downsampling. ds(n) Visual observation data O (n,t) The input is fed into a pre-trained ResNet34 convolutional neural network to extract visual features F. I Then visual feature F I and trajectory τ ds(n) Sensor data S (n,t) The features are concatenated to form a unified feature space; then, a multilayer perceptron (MLP) is used to further process the concatenated features to generate a fused feature vector.

[0029] X (n,k) =MLP(concat(F I ,S (n,t) ))

[0030] Among them, concat(F I ,S (n,t)) indicates that F I and S (n,t) Vertical concatenation is performed. MLP is a feedforward neural network composed of multiple fully connected layers. It can learn complex nonlinear mapping relationships. MLP maps multimodal features to a unified feature space through the nonlinear activation function ReLU.

[0031] Step 3.2: For the feature sequence A positional encoding PosEmb(k) is added to preserve the temporal order information of the sequence. The positional encoding is generated using sine and cosine functions, with the following formula:

[0032]

[0033] Where, d model T is the dimension of the feature vector, i is the dimension index, and T is the dimension index. ds It is the length of the consciousness trajectory generated by downsampling;

[0034] Step 3.3: Input the feature sequence with added positional encoding into the Transformer encoder. The encoder consists of multiple self-attention layers and feedforward neural network layers. The self-attention mechanism captures global contextual information by calculating the relationship between each time step in the feature sequence. The feature sequence processed by the encoder is calculated using the following formula:

[0035] H (n) =Transformer Encoder (X (n,k) +PosEmb(k))

[0036] Step 3.4: Feature sequence H after encoder processing (n) and historical action data A (n,k-1) The input is fed into a Transformer decoder, which consists of multiple self-attention layers, cross-attention layers, and feedforward neural network layers. The self-attention layers are used to capture the temporal dependencies in the historical action data, and the cross-attention layers are used to process the H... (n) With A (n,k-1) Perform alignment to generate target action A. (n,k) Using an autoregressive approach, future action sequences are predicted step by step, with each step generating action data A. (n,k) As input for the next step, used to predict the next action A (n,k+1) The predicted target action for the future time step can be represented as:

[0037] A (n,k) =Transformer Eecoder (H (n) A (n,k-1) )

[0038] Step 3.5: Calculate the mean squared error (MSE) between the predicted action and the actual action. The loss function is expressed as:

[0039]

[0040] The model parameters are updated using the backpropagation algorithm, and the loss function L is minimized. action Using the optimizer, Adam adjusts the model weights to ensure that the model can converge quickly.

[0041] Furthermore, step 4 specifically involves:

[0042] Step 4.1: Acquire current observation data, sensor data, and motion data, process them using the methods in Steps 1 and 2, and then input them into the trained motion prediction model to predict future motion sequences.

[0043] a T ={a T [T+1],a T [T+2],…,a T [T+K]}

[0044] Where K is the length of the action block; the next action to be executed is A. T+1 If a number has been predicted K times in the past, its predicted set is represented as:

[0045] As T+1 ={a (T-1) (T+1),a (T-2) (T+1),…,a (T-K+1) (T+1)}

[0046] Step 4.2: Execute action A at the next time step using a cognitive unloading mechanism and a weighted cumulative prediction strategy. T+1 Prediction:

[0047]

[0048] Among them, e -mk This represents the time-weighted factor, where m is the weight; COR T+1 This is a cognitive unloading mechanism used to calculate the degree of joint position overlap.

[0049]

[0050] Among them, As (T+1,；j) μ represents the predicted sequence of motion for the j-th joint. j and σ j They are As (T+1,；j) The mean and standard deviation of the value, where n is the number of robot joints, std(·) represents calculating the standard deviation, and COR is the set threshold.

[0051] A robot imitation learning method based on intention understanding and subconscious execution has the following beneficial effects:

[0052] 1. In the data collection and processing stage, the method of the present invention analyzes trajectory data through the intention influence metric, reduces redundant information, and optimizes the data processing process.

[0053] 2. In the trajectory downsampling stage, the method of the present invention dynamically calculates the downsampling frequency, prioritizes the retention of key task data, and reduces unnecessary calculations and data storage.

[0054] 3. In the action prediction stage, the method of the present invention adopts an action prediction model based on a transformer architecture, combined with a subconscious imitation learning mechanism, which effectively improves the accuracy and speed of task execution.

[0055] 4. In the action execution phase, the method of the present invention reduces unnecessary reasoning calculations through a cognitive offloading mechanism, thereby improving the real-time performance and execution efficiency of the system.

[0056] This invention significantly improves the efficiency of robots in complex tasks, solves the problems of robots getting stuck and excessive computational burden, and has broad application prospects, especially suitable for industrial and service robots with high real-time requirements. Attached Figure Description

[0057] Figure 1 This is a flowchart of a robot imitation learning method based on intention understanding and subconscious execution according to the present invention. Detailed Implementation

[0058] like Figure 1 As shown, the present invention provides a robot imitation learning method based on intention understanding and subconscious execution, comprising:

[0059] Step 1: Collect multiple robot demonstration trajectory data in a real-world environment to form a dataset, and calculate the intent influence metric based on the robot demonstration trajectory data, specifically:

[0060] Step 1.1: Collect multiple robot demonstration trajectory data in a real-world environment to form a dataset D = {τ0, τ1, ..., τ}. N}, each trajectory τ n The n = 0, 1, ..., N consists of time steps t and contains the robot's visual observation data O. (n,t) Sensor data S (n,t) and motion data A (n,t) .

[0061] The visual observation data is image data of the robot's working environment; the sensor data includes joint position qpos. (n,t)Joint velocity qvel (n,t) With end effector torque eeft (n,t) Motion data includes target joint position α pos(n,t) and target joint velocity α qvel(n,t) .

[0062] Step 1.2: Calculate the Intentional Influence Measurement I according to the following formula. t :

[0063]

[0064] in, It is a normalization function, and ω is a weight vector used to determine the relative importance of joints.

[0065] For example, in practical applications, ω can be set to [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.3, 0.1]. This metric can quantify and prioritize the retention of pattern information that is crucial to task execution, thereby ensuring the effectiveness of subsequent data processing.

[0066] Step 2: Smooth the intention influence metric using a smoothing filter, calculate the dynamic downsampling frequency, and downsample the robot demonstration trajectory data based on the dynamic downsampling frequency to generate an intention trajectory dataset, specifically:

[0067] Step 2.1: Smooth the intention influence measure using a Butterworth filter, as shown in the following formula:

[0068]

[0069] Where 's' is a complex frequency variable used to describe the frequency characteristics of the signal and the dynamic behavior of the system; 'm' is the order of the Butterworth filter, determining the steepness of the filter's transition band. A higher order results in a steeper transition band and better filtering performance, but also higher computational complexity; it is typically set to 4 or 6. ω c This is the cutoff frequency.

[0070] If the sampling frequency is 100Hz and the cutoff frequency is 10Hz, then the normalized cutoff frequency is:

[0071]

[0072] Step 2.2: Calculate the dynamic downsampling frequency based on the smoothed intention influence metric and the sampling frequency of the robot demonstration trajectory data:

[0073]

[0074] Among them, f ds (t) is the dynamic downsampling frequency, f dTo demonstrate the raw sampling frequency of trajectory data for the robot, This is a floor operation, where M represents the time scaling factor, and f... m The target minimum sampling frequency.

[0075] Step 2.3: Downsample the original robot demonstration trajectory data according to the dynamic downsampling frequency to generate the consciousness trajectory dataset D. ds ={τ ds(0) ,τ ds(1) ,…,τ ds(N) This data is used for training subsequent action prediction models.

[0076] Step 3: Establish a deep learning-based action prediction model, training it using a consciousness trajectory dataset. The action prediction model utilizes the encoder and decoder in the Transformer architecture to predict future actions. The model accurately predicts the target action at future time steps through pattern recognition of historical trajectory data. Specifically:

[0077] Step 3.1: Calculate the consciousness trajectory τ generated by each downsampling. ds(n) Visual observation data O (n,t) The input is fed into a pre-trained ResNet34 convolutional neural network to extract visual features F. I Then visual feature F I and trajectory τ ds(n) Sensor data S (n,t) The features are concatenated to form a unified feature space; then, a multilayer perceptron (MLP) is used to further process the concatenated features to generate a fused feature vector.

[0078] X (n,k) =MLP(concat(F I ,S (n,t) ))

[0079] Among them, concat(F I ,S (n,t) ) indicates that F I and S (n,t) Vertical concatenation is performed. MLP is a feedforward neural network composed of multiple fully connected layers. It can learn complex nonlinear mapping relationships. MLP uses the nonlinear activation function ReLU to map multimodal features to a unified feature space.

[0080] Step 3.2: For the feature sequence A positional encoding PosEmb(k) is added to preserve the temporal order information of the sequence. The positional encoding is generated using sine and cosine functions, with the following formula:

[0081]

[0082] Where, d model T is the dimension of the feature vector, i is the dimension index, and T is the dimension index. ds It is the length of the consciousness trajectory generated by downsampling.

[0083] Step 3.3: Input the feature sequence with added positional encoding into the Transformer encoder. The encoder consists of multiple self-attention layers and feedforward neural network layers. The self-attention mechanism captures global contextual information by calculating the relationship between each time step in the feature sequence. The feature sequence processed by the encoder is calculated using the following formula:

[0084] H (n) =Transformer Encoder (X (n,k) +PosEmb(k))

[0085] Step 3.4: Feature sequence H after encoder processing (n) and historical action data A (n,k-1) The input is fed into a Transformer decoder, which consists of multiple self-attention layers, cross-attention layers, and feedforward neural network layers. The self-attention layers are used to capture the temporal dependencies in the historical action data, and the cross-attention layers are used to process the H... (n) With A (n,k-1) Perform alignment to generate target action A. (n,k) Using an autoregressive approach, future action sequences are predicted step by step, with each step generating action data A. (n,k) As input for the next step, used to predict the next action A (n,k+1) The predicted target action for the next future time step is represented as:

[0086] A (n,k) =Transformer Eecoder (H (n) A (n,k-1) )

[0087] Step 3.5: Calculate the mean squared error (MSE) between the predicted action and the actual action. The loss function is expressed as:

[0088]

[0089] The model parameters are updated using the backpropagation algorithm, and the loss function L is minimized. action Using the optimizer, Adam adjusts the model weights to ensure that the model can converge quickly.

[0090] Step 4: During the robot's action execution phase, the current observation data, sensor data, and action data, after being processed in steps 1 and 2, are input into the trained action prediction model. The model outputs a predicted sequence of future actions. Through a cognitive unloading mechanism and a weighted cumulative prediction strategy, the next action to be executed is predicted. Specifically:

[0091] Step 4.1: Acquire current observation data, sensor data, and motion data, process them using the methods in Steps 1 and 2, and then input them into the trained motion prediction model to predict future motion sequences.

[0092] a T ={a T [T+1],a T [T+2],…,a T [T+K]}

[0093] Where K is the length of the action block; the next action to be executed is A. T+1 If a number has been predicted K times in the past, its predicted set is represented as:

[0094] As T+1 ={a (T-1) (T+1),a (T-2) (T+1),…,a (T-K+1) (T+1)}

[0095] Step 4.2: Execute action A at the next time step using a cognitive unloading mechanism and a weighted cumulative prediction strategy. T+1 Prediction:

[0096]

[0097] Among them, e -mk This represents the time-weighted factor, where m is the weight; COR T+1 This is a cognitive unloading mechanism used to calculate the degree of joint position overlap.

[0098]

[0099] Among them, As (T+1,；j) μ represents the predicted sequence of motion for the j-th joint. j and σ j They are As (T+1,；j) The mean and standard deviation of the value, where n is the number of robot joints, std(·) represents calculating the standard deviation, and COR is the set threshold.

[0100] In step 4.2 above, the overlapping joint position information of the historical trajectory is first evaluated to determine whether a skip step can be performed, and then the prediction A of the current action is directly generated. T+1 =a (T-1)(T+1). If the historically predicted joint position exceeds the set threshold, i.e., COR... T+1 If the COR is greater than 0, the subsequent weighted cumulative prediction process can be skipped, reducing the computational burden.

[0101] Step 5: Feed back the predicted action to the robot actuator, obtain new observation data, and repeat steps 4 and 5 until the target task is completed.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the ideas of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A robot imitation learning method based on intention understanding and subconscious execution, characterized by, The application relates to a method for predicting robot action based on cognitive unloading mechanism and weighted cumulative prediction strategy. The method comprises the following steps: Step 1: collecting a plurality of robot demonstration trajectory data in an actual environment to form a data set, and calculating an intention influence measure according to the robot demonstration trajectory data; Step 2: smoothing the intention influence measure by using a smoothing filter, calculating a dynamic down-sampling frequency, and down-sampling the robot demonstration trajectory data based on the dynamic down-sampling frequency to generate an awareness trajectory data set; Step 3: establishing a deep learning-based action prediction model, and training the deep learning-based action prediction model by using the awareness trajectory data set; Step 4: in the robot action execution stage, inputting current observation data, sensor data and action data into the trained action prediction model after processing by steps 1 and 2, outputting a predicted future action sequence, and predicting an execution action at the next moment by a cognitive unloading mechanism and a weighted cumulative prediction strategy; 2.The robot imitation learning method based on intention understanding and subconscious execution of claim 1, wherein, Step 5: feeding back the predicted execution action at the next moment to a robot executor, obtaining new observation data, and repeating steps 4 and 5 until a target task is completed. Step 1.1: Collect multiple robot demonstration trajectory data in real-world environment to form dataset D = {τ0, τ1, …, τN}, each trajectory τn=0,1,…,N consists of time steps t, containing robot's vision observation data O (n,t) , sensor data S (n,t) , and action data A (n,t) ; N n (n,t) (n,t) (n,t) The visual observation data is image data of a robot working environment; the sensor data includes joint positions qpos (n,t) , joint velocities qvel (n,t) , and end effector torques eeft (n,t) ; the action data includes target joint positions a pos(n,t) and target joint velocities a qvel(n,t) ​ Step 1.2: Calculate the intended impact measure I according to the formula t : wherein, is a normalization function, and ω is a weight vector used to determine the relative importance of the joints. 3.The robot imitation learning method based on intention understanding and subconscious execution of claim 1, wherein, The step 1 is specifically characterized in that: The step 2 is specifically characterized in that: where s is a complex frequency variable used to describe the frequency characteristics of the signal and the dynamic behavior of the system; m is the order of the Butterworth filter, which determines the steepness of the filter; ω c is the cutoff frequency; Step 2.1: smoothing the intention influence measure by using a Butterworth filter, and the formula is as follows: wherein f ds (t) is the dynamic down-sampling frequency, f d is the original sampling frequency of the robot demonstration trajectory data, is a floor operation, M represents a time scaling factor, f m is the target minimum sampling frequency; Step 2.3: Downsample the original robot demonstration trajectory data according to the dynamic downsample frequency to generate the awareness trajectory dataset D ds = {τ ds(0) ,τ ds(1) ,…,τ ds(N)} for subsequent action prediction model training. 4.The robot imitation learning method based on intention understanding and subconscious execution of claim 1, wherein, Step 2.2: calculating a dynamic down-sampling frequency based on the smoothed intention influence measure and a sampling frequency of the robot demonstration trajectory data: Step 3.1: input the visual observation data O in each down-sampling generated consciousness trajectory τ ds(n) (n,t) into a pre-trained ResNet34 convolutional neural network to extract visual features F I I and the sensor data S in the trajectory τ ds(n) (n,t) are spliced to form a unified feature space; then the spliced features are further processed by a multi-layer perceptron (MLP) to generate a fused feature vector:​​​ X (n,k) = MLP(concat(F I , S (n,t) )) Among them, concat(F I ,S (n,t) ) indicates that F I and S (n,t) Vertical concatenation is performed. MLP is a feedforward neural network composed of multiple fully connected layers. It can learn complex nonlinear mapping relationships. MLP maps multimodal features to a unified feature space through the nonlinear activation function ReLU. Step 3.2: Feature sequence A position encoding PosEmb(k) is added to preserve the temporal order information of the sequence, which is generated by sine and cosine functions with the formula: where d model is the dimension of the feature vector, i is the dimension index, T ds is the length of the down-sampled generated awareness trajectory; The step 3 is specifically characterized in that: H (n) = Transformer Encoder (X (n,k) + PosEmb(k)) Step 3.4: Feature sequence H processed by encoder (n) and history action data A (n,k-1) into a Transformer decoder, which consists of multiple self-attention layers, cross-attention layers and feed-forward neural network layers. The self-attention layers are used to capture the temporal dependencies in the history action data, and the cross-attention layers are used to align H (n) with A (n,k-1) to generate the target action A (n,k) . In an autoregressive manner, the future action sequence is predicted step by step, and the action data A (n,k) generated at each step is used as the input for the next step to predict the next action A (n,k+1) . The predicted target action at the future time step can be represented as: A (n,k) = Transformer Eecoder (H (n) , A (n,k-1) ) Step 3.3: inputting the feature sequence added with position encoding into a Transformer encoder, and the encoder is composed of multiple self-attention layers and feed-forward neural network layers; the self-attention mechanism calculates the relationship between each time step in the feature sequence to capture global context information; the formula for processing the feature sequence by the encoder is as follows: The model parameters are updated by backpropagation algorithm to minimize the loss function L action Adam is used to adjust the model weights using an optimizer, ensuring that the model converges quickly. 5.The robot imitation learning method based on intention understanding and subconscious execution of claim 1, wherein, Step 3.5: calculating a mean square error (MSE) between a predicted action and a real action, and the loss function is represented as: The step 4 is specifically characterized in that: a T = {a T [T+1],a T [T+2],…,a T [T+K]} Step 4.1: obtaining current observation data, sensor data and action data, and inputting the data into the trained action prediction model after processing by steps 1 and 2 to predict a future action sequence: Wherein, K is the length of the action block; next execute action A T+1 K has been predicted in the past K times, which is represented by the set of predictions: As T+1 = {a (T-1) (T+1),a (T-2) (T+1),…,a (T-K+1) (T+1)} Step 4.2: Perform the action A at the next time instant by the cognitive offloading mechanism and the weighted cumulative prediction strategy T+1 of the prediction: wherein e -mk represents a time weighting factor, m is a weight; COR T+1 is a cognitive offloading mechanism for calculating the degree of joint position overlap; where, As (T+1,；j) represents the predicted sequence of the jth joint motion, μ j and σ j are the mean and standard deviation of As (T+1,；j) , respectively, n is the number of joints of the robot, std(·) represents the standard deviation; COR is a set threshold.

Images