Robot continuous trajectory generation method based on behavior cloning

Abstract

The present invention relates to the field of robotics technology involving behavioral cloning, specifically a method for generating continuous robot trajectories using behavioral cloning. The method comprises the following steps: S1, data acquisition and preprocessing: data acquisition using multiple sensors, followed by data cleaning and noise reduction, and finally data annotation and optimization; S2, model construction and training: selecting a corresponding model architecture based on task information, training multiple models with different structures or parameter initializations, and finally training the models using a training strategy combining transfer learning and reinforcement learning. This invention effectively improves data quality through multi-sensor fusion acquisition, data cleaning and noise reduction, and annotation optimization.

Description

Technical Field

[0001] The present invention relates to the technical field of robot technology of behavior cloning, and in particular to a method for generating continuous trajectory of a robot of behavior cloning. Background Art

[0002] Behavior cloning robots are robots trained using behavioral cloning techniques from imitation learning, enabling them to mimic the behaviors and actions of human experts. Behavior cloning is a supervised learning method that trains a robot's policy network by learning from expert demonstration data, making its decision-making similar to that of human experts.

[0003] Currently, the following problems exist in the process of generating continuous robot trajectories based on behavior cloning:

[0004] 1. Raw data noise and incompleteness: When collecting human demonstration data, the raw data often contains noise due to factors such as sensor errors and environmental interference. For example, when using lidar to collect data on the robot's motion environment, factors such as external light reflection and the surface material of the object may cause deviations in the collected distance data. In addition, the data may also be incomplete. For example, when recording the robot's grasping action, the motion data of some key hand joints may be lost due to camera angle issues. This noisy and incomplete data can mislead model learning, causing the robot to deviate when generating trajectories and fail to accurately imitate human behavior.

[0005] 2. Accumulated imitation errors: Behavioral cloning aims to teach robots to learn human behavior patterns. However, in actual execution, due to differences in the robot's own dynamic characteristics and the influence of environmental factors, each action will produce certain errors. For example, when a robot performs continuous path planning and movement, the first movement may deviate from the target trajectory by 1 centimeter. Each subsequent movement is based on the previous state. These small errors accumulate, and over time, the robot's actual movement trajectory will deviate significantly from the expected trajectory, leading to mission failure.

[0006] 3. Insufficient model generalization: Traditional behavior cloning models are typically trained based on human demonstration data in specific scenarios and conditions. When robots are faced with new environments, task requirements, or interference factors, the models often fail to adapt. For example, a robot carrying model trained indoors, when applied to complex outdoor environments with slopes and uneven surfaces, cannot accurately generate appropriate trajectories, making it difficult to complete the carrying task, as the training data does not include such conditions.

[0007] 4. Data scarcity and labeling difficulties: Obtaining high-quality human demonstration data is costly, especially for complex robotic tasks such as delicate surgical operations and demanding industrial assembly tasks. This requires not only professional demonstrations, but also the time-consuming and labor-intensive process of collecting and labeling data. For example, when training robots for cardiac surgery, there is limited data from professional doctors demonstrating their actions, and accurately labeling each movement requires specialized medical knowledge and a significant amount of time. The scarcity of data and the difficulty of labeling limit the effectiveness of model training and affect the quality of the robot's continuous trajectory generation.

[0008] Based on the above, a method for generating continuous trajectories of a robot based on behavior cloning is invented. Summary of the Invention

[0009] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0010] The method for generating continuous trajectories of a robot based on behavior cloning includes the following specific steps:

[0011] S1, data collection and preprocessing: First, use multiple sensors to collect data, then clean and reduce noise, and then optimize the data annotation;

[0012] S2, model construction and training: First, select the corresponding model architecture based on the task information, then train multiple models with different structures or parameter initializations, and then use a training strategy that combines transfer learning and reinforcement learning to train the model;

[0013] S3, trajectory generation and optimization: First, a trajectory is generated based on the preprocessed data and the constructed model. Then, a smoothing algorithm is used to process the original trajectory generated by the model. After that, an optimization algorithm is used to adjust the trajectory online. At the same time, the deviation between the actual motion trajectory of the robot and the expected trajectory is analyzed.

[0014] S4, Experiment and Verification: First, conduct simulation tests on the generated robot continuous trajectory, and then conduct actual scene verification on the generated robot continuous trajectory.

[0015] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S1 are as follows:

[0016] S11, Multi-sensor Fusion Acquisition: Data collection is performed using multiple sensors, including cameras, lidar, and inertial measurement units, to obtain more comprehensive and accurate human demonstration behavior data and reduce missing and incomplete data.

[0017] S12, data cleaning and noise reduction: The collected data is processed using filtering algorithms to remove noise interference. For data points with outliers, statistical methods are used to identify and correct them. At the same time, missing data is supplemented through data interpolation methods to ensure data integrity and accuracy. The filtering algorithms include Kalman filtering and Gaussian filtering;

[0018] S13, data labeling optimization: Use semi-automatic labeling tools combined with manual verification to label data. First, use the algorithm to perform preliminary labeling on the data, and then have personnel check and correct it to improve labeling efficiency and accuracy.

[0019] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S2 are as follows:

[0020] S21, Model Architecture Selection: Based on the complexity of the task and the characteristics of the data, select the corresponding deep learning model architecture, which includes recurrent neural networks, variant long short-term memory networks, gated recurrent units, convolutional neural networks, and combined models of recurrent neural networks;

[0021] S22, model integration: First, train multiple models with different structures or parameter initializations, and then fuse the model outputs based on the integration strategy;

[0022] S23, Improved Training Strategy: First, pre-train on a large-scale general dataset to learn basic behavioral patterns and feature representations. Then, fine-tune the model based on task-specific data to improve its adaptability to specific tasks. At the same time, a reinforcement learning mechanism is introduced to guide the robot to continuously optimize its trajectory during actual execution through a reward function, thereby reducing the accumulation of imitation errors.

[0023] S24, Model Evaluation and Optimization: Establish reasonable model evaluation indicators to regularly evaluate the model during the training process, and adjust model parameters and training strategies based on the evaluation results.

[0024] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S23 are as follows:

[0025] S231, Transfer Learning:

[0026] S2311, select a general dataset: First, select a large-scale general dataset that is relevant to the robot's target task, and then check the dataset's completeness, annotation accuracy, and data diversity;

[0027] S2312, Pre-training Model: First, set training parameters based on the fused model. Then, during the training process, use a cross-validation method to divide the common dataset into a training set and a validation set. Regularly evaluate model performance to prevent overfitting. The training parameters include learning rate, batch size, and number of iterations. Then, input the common dataset into the model for training, so that the model learns the common behavior patterns and feature representations in the data.

[0028] S2313, Model Fine-tuning: First, divide the data collected for the target task into a training set and a validation set. Then, based on the pre-trained model, freeze the network layers and keep their parameters unchanged. Then, based on the task data, unfreeze the network layers related to the target task to retrain the parameters and adapt the model to the needs of the specific task. Then, reduce the learning rate and the amplitude of parameter updates during the fine-tuning process to avoid overfitting to the specific task data. Increase the number of iterations to allow the model time to learn the characteristics of the specific task.

[0029] S232, Reinforcement Learning:

[0030] S2321, define state, action, and reward function: First, determine the state representation of the robot during the task, including the robot's own state information and environmental information. Then, define the set of actions that the robot can perform and discretize each action so that the model can process and output it. Then, design a reasonable reward function to guide the robot to learn the optimal behavior. The reward function is designed based on the task objectives.

[0031] S2322, select a reinforcement learning algorithm: select a corresponding reinforcement learning algorithm based on the complexity of the task and the characteristics of the data, the reinforcement learning algorithm includes a deep Q network, a policy gradient algorithm, and a deep deterministic policy gradient algorithm;

[0032] S2323, training process: First, initialize the policy network for generating actions and the value network for evaluating state values. The policy network receives the state as input and outputs the probability distribution of actions or specific action values. The value network receives the state as input and outputs the value estimate of the state. Then, let the robot perform actions in the environment and select actions based on the current state. Observe the new state and reward obtained after performing the action. Then, store the state, action, reward, and new state information in the experience replay buffer. At this time, data is randomly sampled from the experience replay buffer periodically to update the parameters of the policy network and value network. Based on this, through continuous interaction and learning with the environment, the robot gradually optimizes its own action strategy and generates a better trajectory.

[0033] S233, Combining Transfer Learning with Reinforcement Learning:

[0034] S2331, Strategy Fusion: Using the pre-trained model obtained through transfer learning as the initial policy network for reinforcement learning, and leveraging the general knowledge and task-specific knowledge learned from the pre-trained model to provide an initial strategy for reinforcement learning;

[0035] S2332, Joint Training: During the reinforcement learning training process, newly acquired experience data is regularly integrated with specific task data to re-fine-tune the model. At the same time, based on the training effect of reinforcement learning, the parameters of the transfer learning model are dynamically adjusted to achieve coordinated optimization of the two.

[0036] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S3 are as follows:

[0037] S31, model-based trajectory generation: pre-processed data is fed into a trained model, which then outputs a continuous trajectory for the robot. When generating the trajectory, the robot's dynamic constraints and environmental information are considered to ensure that the generated trajectory is feasible in practice.

[0038] S32, trajectory smoothing: using a smoothing algorithm to process the original trajectory generated by the model;

[0039] S33, online trajectory optimization: During the robot's trajectory execution, environmental information and robot status data are collected in real time, and the trajectory is adjusted online using an optimization algorithm;

[0040] S34, Error compensation and correction: Establish an error compensation mechanism to analyze the deviation between the robot's actual motion trajectory and the expected trajectory, adjust the robot's control parameters in real time, and reduce error accumulation.

[0041] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S33 are as follows:

[0042] S331, Environment and Task Analysis: First, build an environment model based on the data, then evaluate the feasibility of the current trajectory based on the robot's current task goals and its own state;

[0043] S332, trajectory replanning algorithm selection and execution: First, select the corresponding trajectory replanning algorithm based on the environment complexity, task requirements and robot performance, and then generate a new trajectory based on the environment model and the selected algorithm;

[0044] S333, trajectory execution and control: First, the re-planned trajectory is sent to the robot's motion control system, and then the feedback control algorithm is used to adjust the robot's motion parameters in real time.

[0045] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S331 are as follows:

[0046] S3311, Environmental Modeling: Based on preprocessed data, the robot's local environment model is constructed or updated in real time. For dynamic environments, the location and state changes of obstacles in the map are updated promptly.

[0047] S3312, Mission Objective Evaluation: Based on the robot's current mission objective and its own state, the feasibility of the current trajectory is evaluated. If the current trajectory is found to be infeasible or there is room for optimization, trajectory replanning is triggered.

[0048] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S332 are as follows:

[0049] S3321, Algorithm Selection: Select a corresponding trajectory replanning algorithm based on the environment complexity, task requirements, and robot performance. The trajectory replanning algorithm includes a fast search random tree algorithm and its variants, a graph search-based algorithm, and a trajectory planning algorithm based on deep reinforcement learning.

[0050] S3322, perform replanning: use the current robot position as the starting point of the new trajectory and the target position as the end point, combine the updated environment model, and use the selected algorithm to generate a new trajectory.

[0051] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S333 are as follows:

[0052] S3331, trajectory distribution: Convert the replanned trajectory into a command format acceptable to the robot controller and distribute it to the robot's motion control system;

[0053] S3332, Real-time Control: As the robot executes a new trajectory, sensors monitor the robot's actual motion in real time and compare it with the desired trajectory. A feedback control algorithm is then used to adjust the robot's motion parameters in real time based on the deviation, ensuring the robot can accurately track the new trajectory.

[0054] As a preferred solution of the method for generating continuous trajectories of a robot by behavior cloning according to the present invention, the specific steps of S4 are as follows:

[0055] S41, Simulation Experiment: Build a model similar to the actual scenario in a simulation environment, test the generated robot continuous trajectory, and evaluate the performance and generalization ability of the model by changing the environmental parameters and task requirements;

[0056] S42, Actual Scenario Experiment: Conduct robot trajectory generation experiments in actual scenarios to further verify the effectiveness of the solution. During the experiment, record the robot's operating data and task completion status, compare and analyze them with the simulation experiment results, summarize the experience and lessons, and optimize and improve the solution.

[0057] Compared with existing technologies:

[0058] 1. Advantages in addressing raw data noise and incompleteness: Data quality is effectively improved through multi-sensor fusion acquisition, data cleaning and noise reduction, and annotation optimization. The multi-dimensional information acquired through multi-sensor fusion avoids data loss caused by viewing angles and environmental factors in a single sensor. For example, in robotic grasping tasks, combining camera visual information with IMU posture data can fully capture hand joint motion. After removing noise using algorithms such as Kalman filtering, the data is more accurate, enabling the model to learn more realistic and effective human behavior patterns, reducing trajectory generation errors caused by data bias, and significantly improving the reliability of model input data, laying the foundation for subsequent accurate trajectory generation.

[0059] 2. Advantages in addressing the problem of imitation error accumulation: A combination of transfer learning and reinforcement learning is used in the improved training strategy. Transfer learning allows the robot to pre-train on large-scale general data, master basic behavioral patterns, and reduce initial errors caused by differences in its own dynamic characteristics. Reinforcement learning uses a reward function to optimize the robot's trajectory in real time during actual execution, dynamically adjusting subsequent behaviors based on the error of each action. In addition, online trajectory optimization and error compensation mechanisms can also monitor and correct trajectory deviations in real time. For example, the feedback control algorithm adjusts the robot's movement speed and direction based on the error, effectively controlling error accumulation and ensuring that the robot's actual trajectory remains close to the expected trajectory during long-term continuous motion, thereby improving the stability and success rate of task execution.

[0060] 3. Advantages in addressing the problem of insufficient model generalization: Selecting an appropriate model architecture can effectively process time series data and learn behavioral characteristics; transfer learning allows the model to first learn common knowledge on general data, giving it a certain level of basic adaptability; fine-tuning with specific task data allows the model to adapt to specific scenarios; evaluation and optimization methods such as cross-validation prevent model overfitting, enabling the model to extract universal behavioral patterns and characteristics from training data. When the robot faces new environments, new task requirements, or interference factors, it can quickly adjust and generate appropriate trajectories based on the learned knowledge and patterns, significantly enhancing the model's application capabilities in different scenarios.

[0061] 4. Advantages in addressing data scarcity and labeling difficulties: Semi-automatic labeling tools are combined with manual verification, using algorithms to improve efficiency through preliminary labeling, and professional verification to ensure accuracy. This significantly reduces labeling time and costs while ensuring labeling quality in complex tasks such as heart surgery robot data labeling. These measures enable more efficient use of limited high-quality data, reduce data acquisition and processing costs, and enable the training of high-performance models even in data scarcity, ensuring the quality of continuous robot trajectory generation. DETAILED DESCRIPTION

[0062] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below.

[0063] The present invention provides a method for generating a continuous trajectory of a robot based on behavior cloning, which includes the following specific steps:

[0064] S1, data collection and preprocessing: First, use multiple sensors to collect data, then clean and reduce noise, and then optimize the data annotation;

[0065] The specific steps of S1 are as follows:

[0066] S11, Multi-sensor Fusion Acquisition: Data collection is performed using multiple sensors, including cameras, lidar, and inertial measurement units, to obtain more comprehensive and accurate human demonstration behavior data and reduce missing and incomplete data.

[0067] S12, data cleaning and noise reduction: The collected data is processed using filtering algorithms to remove noise interference. For data points with outliers, statistical methods are used to identify and correct them. At the same time, missing data is supplemented through data interpolation methods to ensure data integrity and accuracy. The filtering algorithms include Kalman filtering and Gaussian filtering;

[0068] S13, Data Annotation Optimization: Use semi-automatic annotation tools combined with manual verification to annotate data. First, use algorithms to perform preliminary annotations on the data, and then have personnel check and correct them to improve annotation efficiency and accuracy.

[0069] S2, model construction and training: First, select the corresponding model architecture based on the task information, then train multiple models with different structures or parameter initializations, and then use a training strategy that combines transfer learning and reinforcement learning to train the model;

[0070] The specific steps of S2 are as follows:

[0071] S21, Model Architecture Selection: Based on the complexity of the task and the characteristics of the data, select the corresponding deep learning model architecture, which includes recurrent neural networks, variant long short-term memory networks, gated recurrent units, convolutional neural networks, and combined models of recurrent neural networks;

[0072] S22, model integration: First, train multiple models with different structures or parameter initializations, and then fuse the model outputs based on the integration strategy;

[0073] Setting up model integration has the following effects:

[0074] 1. Reduce the uncertainty and error of a single model. Different models may perform well in different scenarios or data. Integration can combine their advantages to improve the accuracy and stability of trajectory generation.

[0075] 2. When a model deviates under specific circumstances, the outputs of other models can be used to supplement and correct it, reducing trajectory generation errors caused by the failure of a single model and improving the robot's ability to perform tasks in complex environments.

[0076] S23, Improved Training Strategy: First, pre-train on a large-scale general dataset to learn basic behavioral patterns and feature representations. Then, fine-tune the model based on task-specific data to improve its adaptability to specific tasks. At the same time, a reinforcement learning mechanism is introduced to guide the robot to continuously optimize its trajectory during actual execution through a reward function, thereby reducing the accumulation of imitation errors.

[0077] The specific steps of S23 are as follows:

[0078] S231, Transfer Learning:

[0079] S2311, select a general dataset: First, select a large-scale general dataset that is relevant to the robot's target task, and then check the dataset's completeness, annotation accuracy, and data diversity;

[0080] S2312, Pre-training Model: First, set training parameters based on the fused model. Then, during the training process, use a cross-validation method to divide the common dataset into a training set and a validation set. Regularly evaluate model performance to prevent overfitting. The training parameters include learning rate, batch size, and number of iterations. Then, input the common dataset into the model for training, so that the model learns the common behavior patterns and feature representations in the data.

[0081] S2313, Model Fine-tuning: First, divide the data collected for the target task into a training set and a validation set. Then, based on the pre-trained model, freeze the network layers and keep their parameters unchanged. Then, based on the task data, unfreeze the network layers related to the target task to retrain the parameters and adapt the model to the needs of the specific task. Then, reduce the learning rate and the amplitude of parameter updates during the fine-tuning process to avoid overfitting to the specific task data. Increase the number of iterations to allow the model time to learn the characteristics of the specific task.

[0082] S232, Reinforcement Learning:

[0083] S2321, define state, action, and reward function: First, determine the state representation of the robot during the task, including the robot's own state information and environmental information. Then, define the set of actions that the robot can perform and discretize each action so that the model can process and output it. Then, design a reasonable reward function to guide the robot to learn the optimal behavior. The reward function is designed based on the task objectives.

[0084] S2322, select a reinforcement learning algorithm: select a corresponding reinforcement learning algorithm based on the complexity of the task and the characteristics of the data, the reinforcement learning algorithm includes a deep Q network, a policy gradient algorithm, and a deep deterministic policy gradient algorithm;

[0085] S2323, training process: First, initialize the policy network for generating actions and the value network for evaluating state values. The policy network receives the state as input and outputs the probability distribution of actions or specific action values. The value network receives the state as input and outputs the value estimate of the state. Then, let the robot perform actions in the environment and select actions based on the current state. Observe the new state and reward obtained after performing the action. Then, store the state, action, reward, and new state information in the experience replay buffer. At this time, data is randomly sampled from the experience replay buffer periodically to update the parameters of the policy network and value network. Based on this, through continuous interaction and learning with the environment, the robot gradually optimizes its own action strategy and generates a better trajectory.

[0086] S233, Combining Transfer Learning with Reinforcement Learning:

[0087] S2331, Strategy Fusion: Using the pre-trained model obtained through transfer learning as the initial policy network for reinforcement learning, and leveraging the general knowledge and task-specific knowledge learned from the pre-trained model to provide an initial strategy for reinforcement learning;

[0088] S2332, Joint Training: During reinforcement learning training, newly acquired experience data is regularly integrated with task-specific data to fine-tune the model. Simultaneously, based on the training results of reinforcement learning, the parameters of the transfer learning model are dynamically adjusted to achieve collaborative optimization of the two.

[0089] S24, Model Evaluation and Optimization: Establish reasonable model evaluation indicators to regularly evaluate the model during training, and adjust model parameters and training strategies based on the evaluation results;

[0090] S3, trajectory generation and optimization: First, a trajectory is generated based on the preprocessed data and the constructed model. Then, a smoothing algorithm is used to process the original trajectory generated by the model. After that, an optimization algorithm is used to adjust the trajectory online. At the same time, the deviation between the actual motion trajectory of the robot and the expected trajectory is analyzed.

[0091] The specific steps of S3 are as follows:

[0092] S31, model-based trajectory generation: pre-processed data is fed into a trained model, which then outputs a continuous trajectory for the robot. When generating the trajectory, the robot's dynamic constraints and environmental information are considered to ensure that the generated trajectory is feasible in practice.

[0093] S32, trajectory smoothing: using a smoothing algorithm to process the original trajectory generated by the model;

[0094] The following effects can be achieved by setting trajectory smoothing:

[0095] 1. Improve the smoothness of robot movement, reduce mechanical wear and energy consumption caused by uneven trajectory, and extend the service life of the robot;

[0096] 2. Reduce the impact and vibration during robot movement, improving the safety and reliability of robot operation. This is especially true in tasks requiring high motion accuracy, such as precision assembly and surgical operations. Smooth trajectories can ensure high-quality completion of tasks.

[0097] 3. By adding the above steps to the detailed solution and further optimizing the data, model, and trajectory levels, the performance of the robot continuous trajectory generation method based on behavioral cloning can be comprehensively improved, enabling the robot to perform even better in practical applications.

[0098] S33, online trajectory optimization: During the robot's trajectory execution, environmental information and robot status data are collected in real time, and the trajectory is adjusted online using an optimization algorithm;

[0099] The specific steps of S33 are as follows:

[0100] S331, Environment and Task Analysis: First, build an environment model based on the data, then evaluate the feasibility of the current trajectory based on the robot's current task goals and its own state;

[0101] The specific steps of S331 are as follows:

[0102] S3311, Environmental Modeling: Based on preprocessed data, the robot's local environment model is constructed or updated in real time. For dynamic environments, the location and state changes of obstacles in the map are updated promptly.

[0103] S3312, Mission Objective Evaluation: Based on the robot's current mission objective and its own state, the feasibility of the current trajectory is evaluated. If the current trajectory is found to be infeasible or there is room for optimization, trajectory replanning is triggered.

[0104] S332, trajectory replanning algorithm selection and execution: First, select the corresponding trajectory replanning algorithm based on the environment complexity, task requirements and robot performance, and then generate a new trajectory based on the environment model and the selected algorithm;

[0105] The specific steps of S332 are as follows:

[0106] S3321, Algorithm Selection: Select a corresponding trajectory replanning algorithm based on the environment complexity, task requirements, and robot performance. The trajectory replanning algorithm includes a fast search random tree algorithm and its variants, a graph search-based algorithm, and a trajectory planning algorithm based on deep reinforcement learning.

[0107] S3322, perform replanning: use the current robot position as the starting point of the new trajectory and the target position as the end point, combine the updated environment model, and use the selected algorithm to generate a new trajectory;

[0108] S333, trajectory execution and control: First, the replanned trajectory is sent to the robot's motion control system, and then the feedback control algorithm is used to adjust the robot's motion parameters in real time;

[0109] The specific steps of S333 are as follows:

[0110] S3331, trajectory distribution: Convert the replanned trajectory into a command format acceptable to the robot controller and distribute it to the robot's motion control system;

[0111] S3332, Real-time Control: As the robot executes a new trajectory, sensors monitor the robot's actual motion in real time and compare it with the desired trajectory. A feedback control algorithm is then used to adjust the robot's motion parameters in real time based on the deviation, ensuring the robot accurately tracks the new trajectory.

[0112] S34, Error Compensation and Correction: Establish an error compensation mechanism to analyze the deviation between the robot's actual motion trajectory and the expected trajectory, adjust the robot's control parameters in real time, and reduce error accumulation;

[0113] S4, Experiment and Verification: First, simulate and test the generated robot continuous trajectory, and then verify the generated robot continuous trajectory in actual scenarios;

[0114] The specific steps of S4 are as follows:

[0115] S41, Simulation Experiment: Build a model similar to the actual scenario in a simulation environment, test the generated robot continuous trajectory, and evaluate the performance and generalization ability of the model by changing the environmental parameters and task requirements;

[0116] S42, Actual Scenario Experiment: Conduct robot trajectory generation experiments in actual scenarios to further verify the effectiveness of the solution. During the experiment, record the robot's operating data and task completion status, compare and analyze them with the simulation experiment results, summarize the experience and lessons, and optimize and improve the solution.

[0117] Although the present invention has been described above with reference to embodiments, various modifications may be made thereto and equivalent components may be substituted without departing from the scope of the present invention. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of such combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A robot continuous trajectory generation method based on behavior cloning, characterized in that: The specific steps are as follows: S1, data collection and preprocessing: First, use multiple sensors to collect data, then clean and reduce noise, and then optimize the data annotation; S2, model construction and training: First, select the corresponding model architecture based on the task information, then train multiple models with different structures or parameter initializations, and then use a training strategy that combines transfer learning and reinforcement learning to train the model; S3, trajectory generation and optimization: First, a trajectory is generated based on the preprocessed data and the constructed model. Then, a smoothing algorithm is used to process the original trajectory generated by the model. After that, an optimization algorithm is used to adjust the trajectory online. At the same time, the deviation between the actual motion trajectory of the robot and the expected trajectory is analyzed. S4, Experiment and Verification: First, simulate and test the generated robot continuous trajectory, and then verify the generated robot continuous trajectory in actual scenarios; The specific steps of S2 are as follows: S21, Model Architecture Selection: Select the corresponding deep learning model architecture based on the complexity of the task and the characteristics of the data; S22, model integration: First, train multiple models with different structures or parameter initializations, and then fuse the model outputs based on the integration strategy; S23, improved training strategy: first pre-train on a large-scale general dataset to learn basic behavioral patterns and feature representations, then fine-tune the model based on the data of specific tasks to improve the model's adaptability to specific tasks. At the same time, introduce a reinforcement learning mechanism to guide the robot to continuously optimize its trajectory during actual execution through the reward function, thereby reducing the accumulation of imitation errors.

2. The method for generating continuous trajectories of a robot based on behavior cloning according to claim 1, characterized in that: The specific steps of S1 are as follows: S11, multi-sensor fusion acquisition: using multiple sensors to collect data, so as to obtain more comprehensive and accurate human demonstration behavior data through multi-sensor fusion, and reduce data missing and incomplete situations. The multiple sensors include cameras, lidar, and inertial measurement units; S12, data cleaning and noise reduction: The collected data is processed using filtering algorithms to remove noise interference. For data points with outliers, statistical methods are used to identify and correct them. At the same time, missing data is supplemented through data interpolation methods to ensure data integrity and accuracy. The filtering algorithms include Kalman filtering and Gaussian filtering; S13, data labeling optimization: Use semi-automatic labeling tools combined with manual verification to label data. First, use the algorithm to perform preliminary labeling on the data, and then have personnel check and correct it to improve labeling efficiency and accuracy.

3. The method for generating continuous trajectories of a robot based on behavior cloning according to claim 1, characterized in that: The deep learning model architecture includes a combination of recurrent neural network, variant long short-term memory network, gated recurrent unit, convolutional neural network, and recurrent neural network; S24, Model Evaluation and Optimization: Establish reasonable model evaluation indicators to regularly evaluate the model during the training process, and adjust model parameters and training strategies based on the evaluation results.

4. The method for generating continuous trajectories of a robot based on behavior cloning according to claim 3, characterized in that: The specific steps of S23 are as follows: S231, transfer learning: S2311, select a general dataset: First, select a large-scale general dataset that is relevant to the robot's target task, and then check the dataset's completeness, annotation accuracy, and data diversity; S2312, Pre-training Model: First, set training parameters based on the fused model. Then, during the training process, use a cross-validation method to divide the common dataset into a training set and a validation set. Regularly evaluate model performance to prevent overfitting. The training parameters include learning rate, batch size, and number of iterations. Then, input the common dataset into the model for training, so that the model learns the common behavior patterns and feature representations in the data. S2313, Model Fine-tuning: First, divide the data collected for the target task into a training set and a validation set. Then, based on the pre-trained model, freeze the network layers and keep their parameters unchanged. Then, based on the task data, unfreeze the network layers related to the target task to retrain the parameters and adapt the model to the needs of the specific task. Then, reduce the learning rate and the amplitude of parameter updates during the fine-tuning process to avoid overfitting to the specific task data. Increase the number of iterations to allow the model time to learn the characteristics of the specific task. S232, Reinforcement Learning: S2321, define state, action, and reward function: First, determine the state representation of the robot during the task, including the robot's own state information and environmental information. Then, define the set of actions that the robot can perform and discretize each action so that the model can process and output it. Then, design a reasonable reward function to guide the robot to learn the optimal behavior. The reward function is designed based on the task objectives. S2322, select a reinforcement learning algorithm: select a corresponding reinforcement learning algorithm based on the complexity of the task and the characteristics of the data, the reinforcement learning algorithm includes a deep Q network, a policy gradient algorithm, and a deep deterministic policy gradient algorithm; S2323, training process: First, initialize the policy network for generating actions and the value network for evaluating state values. The policy network receives the state as input and outputs the probability distribution of actions or specific action values. The value network receives the state as input and outputs the value estimate of the state. Then, let the robot perform actions in the environment and select actions based on the current state. Observe the new state and reward obtained after performing the action. Then, store the state, action, reward, and new state information in the experience replay buffer. At this time, data is randomly sampled from the experience replay buffer periodically to update the parameters of the policy network and value network. Based on this, through continuous interaction and learning with the environment, the robot gradually optimizes its own action strategy and generates a better trajectory. S233, Combining Transfer Learning with Reinforcement Learning: S2331, Strategy Fusion: Using the pre-trained model obtained through transfer learning as the initial policy network for reinforcement learning, and leveraging the general knowledge and task-specific knowledge learned from the pre-trained model to provide an initial strategy for reinforcement learning; S2332, Joint Training: During the reinforcement learning training process, newly acquired experience data is regularly integrated with specific task data to re-fine-tune the model. At the same time, based on the training effect of reinforcement learning, the parameters of the transfer learning model are dynamically adjusted to achieve coordinated optimization of the two.

5. The method for generating continuous trajectories of a robot based on behavior cloning according to claim 1, characterized in that: The specific steps of S3 are as follows: S31, model-based trajectory generation: inputting preprocessed data into a trained model, and the model outputs a continuous trajectory of the robot. When generating the trajectory, the robot's dynamic constraints and environmental information are considered to ensure that the generated trajectory is feasible in practice; S32, trajectory smoothing: using a smoothing algorithm to process the original trajectory generated by the model; S33, online trajectory optimization: During the robot's trajectory execution, environmental information and robot status data are collected in real time, and the trajectory is adjusted online using an optimization algorithm; S34, Error compensation and correction: Establish an error compensation mechanism to analyze the deviation between the robot's actual motion trajectory and the expected trajectory, adjust the robot's control parameters in real time, and reduce error accumulation.

6. The method for generating continuous trajectories of a robot based on behavior cloning according to claim 5, characterized in that: The specific steps of S33 are as follows: S331, environment and task analysis: first build an environment model based on the data, then evaluate the feasibility of the current trajectory based on the robot's current task goal and its own state; S332, trajectory replanning algorithm selection and execution: First, select the corresponding trajectory replanning algorithm based on the environment complexity, task requirements and robot performance, and then generate a new trajectory based on the environment model and the selected algorithm; S333, trajectory execution and control: First, the re-planned trajectory is sent to the robot's motion control system, and then the feedback control algorithm is used to adjust the robot's motion parameters in real time.

7. The method for generating continuous trajectories of a robot based on behavior cloning according to claim 6, characterized in that: The specific steps of S331 are as follows: S3311, environment modeling: based on the preprocessed data, build or update the robot's local environment model in real time. For dynamic environments, update the position and state changes of obstacles in the map in a timely manner; S3312, Mission Objective Evaluation: Based on the robot's current mission objective and its own state, the feasibility of the current trajectory is evaluated. If the current trajectory is found to be infeasible or there is room for optimization, trajectory replanning is triggered.

8. The method for generating continuous trajectories of a robot based on behavior cloning according to claim 6, characterized in that: The specific steps of S332 are as follows: S3321, algorithm selection: selecting a corresponding trajectory replanning algorithm based on the complexity of the environment, task requirements and robot performance, wherein the trajectory replanning algorithm includes a fast search random tree algorithm and its variants, an algorithm based on graph search, and a trajectory planning algorithm based on deep reinforcement learning; S3322, perform replanning: use the current robot position as the starting point of the new trajectory and the target position as the end point, combine the updated environment model, and use the selected algorithm to generate a new trajectory.

9. The method for generating continuous trajectory of a robot based on behavior cloning according to claim 6, characterized in that: The specific steps of S333 are as follows: S3331, trajectory distribution: converting the re-planned trajectory into an instruction format that can be received by the robot controller, and distributing it to the robot's motion control system; S3332, Real-time Control: As the robot executes a new trajectory, sensors monitor the robot's actual motion in real time and compare it with the desired trajectory. A feedback control algorithm is then used to adjust the robot's motion parameters in real time based on the deviation, ensuring the robot can accurately track the new trajectory.

10. The method for generating continuous trajectory of a robot based on behavior cloning according to claim 1, characterized in that: The specific steps of S4 are as follows: S41, simulation experiment: building a model similar to the actual scene in a simulation environment, testing the generated robot continuous trajectory, and evaluating the performance and generalization ability of the model by changing the environmental parameters and task requirements; S42, Actual Scenario Experiment: Conduct robot trajectory generation experiments in actual scenarios to further verify the effectiveness of the solution. During the experiment, record the robot's operating data and task completion status, compare and analyze them with the simulation experiment results, summarize the experience and lessons, and optimize and improve the solution.