Method, device, equipment, medium and product for optimizing motion trajectory of robot

Abstract

The application provides a method and device for optimizing the motion trajectory of a robot, equipment, a medium and a product, and relates to the technical field of artificial intelligence. The application realizes fine evaluation of the quality of the motion trajectory, and accurate positioning of the abnormal range and abnormal motion state. The target function is used to locally optimize the abnormal motion state of the abnormal range, ensuring that the optimized motion trajectory meets the standard intention of expert behavior and guarantees the task target, while improving the fluency and environmental safety of the motion trajectory. The application realizes a closed-loop optimization mechanism from quality quantization to abnormal positioning to smooth repair of the motion trajectory, which can effectively reduce the abnormal trajectory segments in the existing generated motion trajectory of the robot. The final output of the optimized motion trajectory conforms to the high-quality data required by the robot imitation learning training field, providing reliable technical support for breaking through the bottleneck of insufficient training data in the imitation learning field.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium and product for optimizing the motion trajectory of a robot. Background Technology

[0002] With the rapid development of robotics and artificial intelligence, imitation learning (IL) and reinforcement learning (RL) have become mainstream technological paths for enabling robots to master complex skills. These methods endow robots with the ability to perform diverse tasks by learning from demonstration data or through trial and error in interaction with the environment. However, the training effect of both imitation learning and reinforcement learning is highly dependent on a large amount of high-quality demonstration or experience data. Traditionally, collecting demonstration data by remotely operating robots by human experts is not only costly and time-consuming, but also difficult to scale, failing to meet the massive data requirements of modern robot large-scale model training. Therefore, the contemporary robotics field needs to efficiently and cost-effectively synthesize high-quality robot demonstration trajectory data (robot motion trajectories).

[0003] Existing methods for generating high-quality robot motion trajectories include data-driven approaches using generative models, particularly by utilizing advanced deep learning networks such as diffusion models to directly learn the distribution patterns and decision-making logic contained in massive amounts of expert demonstration data. This allows for the generation of entirely new motion trajectories that resemble expert behavior, with the aim of expanding and generalizing expert data.

[0004] However, the robot trajectories generated by data-driven methods in generative models often contain many abnormal trajectory segments, which directly limits the performance ceiling of subsequent robot learning strategies. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for optimizing the motion trajectory of a robot, in order to solve the problem that there are many abnormal trajectory segments in the robot motion trajectory generated in the prior art, and to reduce the abnormal trajectory segments in the robot motion trajectory.

[0006] In a first aspect, this application provides a method for optimizing the motion trajectory of a robot, comprising: Input the robot's motion trajectory into the optimization model, and obtain the optimized motion trajectory output by the optimization model; The optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module. The behavior intent inference module extracts features from each action state of the motion trajectory and calculates the reward value for each action state based on the extracted feature vectors. The abnormal behavior detection module identifies the abnormal range based on the reward value and feature vectors. The abnormal behavior repair module corrects the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory. The objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

[0007] In one embodiment, at least one correction term includes a smoothness penalty term, a task objective preservation term, a boundary connection term, and a collision penalty term; the smoothness penalty term ensures the optimized motion trajectory is smooth; the task objective preservation term ensures the optimized motion trajectory focuses on the task objective; the boundary connection term ensures a smooth transition between the corrected state and its actions within the abnormal range and the uncorrected state and its actions; the collision penalty term reduces the collision risk in the optimized motion trajectory; the objective function is determined based on the following: Based on the corrected weighting coefficients, the smoothness penalty term, the task objective preservation term, the boundary connection term, and the collision penalty term are weighted and summed to obtain the weighted correction term. The objective function is determined based on the weighted adjustment term and the target cumulative reward value.

[0008] In one embodiment, the feature vector is extracted based on the following method: Feature extraction is performed on the action state to obtain the target Euclidean distance, the robot's average joint angle change rate, the robot's end effector speed smoothness, the task completion degree of the action state, the task deviation degree of the action state, and the number of collisions; the target Euclidean distance is the Euclidean distance between the robot's end effector and the task target; The feature vector is obtained by concatenating the target Euclidean distance, average joint angle change rate, velocity smoothness, task completion degree, task deviation degree, and number of collisions.

[0009] In one embodiment, the exception range is determined based on the following: When the reward value of an action state is lower than the instantaneous reward threshold, the action state is identified as an abnormal action state. When the cumulative reward value of at least one consecutive action state is lower than the cumulative reward threshold, at least one consecutive action state is identified as an abnormal segment. When the rate of change of the reward value of an action state exceeds the range of the reward threshold, the action state is identified as an abnormal action state; or, when the rate of change of at least one reward value of at least one consecutive action state exceeds the range of the reward threshold, at least one consecutive action state is identified as an abnormal segment. If a collision event is detected in an action state based on the feature vector of the action state, the action state will be identified as an abnormal action state. Determine the scope of the anomaly based on the abnormal action state and / or abnormal fragment.

[0010] In one embodiment, the behavioral intent inference module is trained based on the following method: The robot's expert demonstration trajectory was used as the first sample motion trajectory; The sample task and reward function are input into the policy network to obtain the second sample motion trajectory output by the policy network. The policy network is used to generate at least one trajectory data based on the sample task and calculate the cumulative trajectory reward value of each trajectory data according to the reward function. The trajectory data with the largest cumulative trajectory reward value is taken as the second sample motion trajectory. Based on the first and second sample motion trajectories, the reward function of the reward function network is optimized and trained to obtain the optimized reward function. During the optimization training process, the reward function network generates the first sample cumulative reward value of the first sample motion trajectory and the second sample cumulative reward value of the second sample motion trajectory. Based on the comparison result of the first sample cumulative reward value and the second sample cumulative reward value, the first sample motion trajectory is identified as a standard behavior trajectory, and the second sample motion trajectory is identified as a non-standard behavior trajectory. Based on the optimized reward function, an optimized reward function network is obtained to obtain the behavioral intent inference module.

[0011] In one embodiment, the reward function network is further used to generate a first feature expectation of the first sample motion trajectory and a second feature expectation of the second sample motion trajectory. After obtaining the optimized reward function, the network further includes: Input the sample task and the optimized reward function into the policy network to obtain the optimized second sample motion trajectory output by the policy network; Based on the first sample motion trajectory and the optimized second sample motion trajectory, the optimized reward function is further optimized and trained. The policy network and reward function are trained alternately and iteratively until the difference between the expectation of the first feature and the expectation of the second feature is less than a set difference. Then the training ends and the final optimized reward function is obtained. Based on the final optimized reward function, obtain the final optimized reward function network to obtain the behavioral intent inference module.

[0012] Secondly, this application also provides a device for optimizing the motion trajectory of a robot, comprising: The trajectory optimization module is used to input the robot's motion trajectory into the optimization model and obtain the optimized motion trajectory output by the optimization model. The optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module. The behavior intent inference module extracts features from each action state of the motion trajectory and calculates the reward value for each action state based on the extracted feature vectors. The abnormal behavior detection module identifies the abnormal range based on the reward value and feature vectors. The abnormal behavior repair module corrects the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory. The objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

[0013] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for optimizing the motion trajectory of any of the robots described above.

[0014] Fourthly, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for optimizing the motion trajectory of any of the robots described above.

[0015] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements a method for optimizing the motion trajectory of any of the robots described above.

[0016] This application provides a method, apparatus, device, medium, and product for optimizing robot motion trajectories. By calculating the reward value for each action state, it achieves a refined evaluation of motion trajectory quality. Precise localization of abnormal ranges and abnormal action states is achieved through reward values ​​and feature vectors. Local directional optimization of abnormal action states within abnormal ranges is performed using an objective function, ensuring that the optimized motion trajectory conforms to the standard intent of expert behavior, guarantees task objectives, and improves the smoothness of the motion trajectory and environmental safety. This application achieves a closed-loop optimization mechanism for motion trajectory from quality quantification to anomaly localization to smooth repair through deep collaboration between the behavior intent inference module, abnormal behavior detection module, and abnormal behavior repair module. This effectively reduces abnormal trajectory segments in existing generated robot motion trajectories. The optimized motion trajectory output by this application finally meets the high-quality data requirements of robot imitation learning training grounds, providing reliable technical support for overcoming the bottleneck of scarce training data in the field of imitation learning. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the method for optimizing the motion trajectory of a robot provided in this application.

[0019] Figure 2 This is the second flowchart illustrating the method for optimizing the motion trajectory of the robot provided in this application.

[0020] Figure 3 This is a schematic diagram of the structure of the robot motion trajectory optimization device provided in this application.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Existing automatically generated robot motion trajectories often suffer from the following defects: redundant movements, motion stuttering, inefficient paths, suboptimal operations, or potential collision risks. The core shortcoming of robot motion trajectory generation lies in its inability to accurately identify the root causes of abnormal behavior and its lack of quantitative understanding of the behavioral intentions of human experts. This results in generated motion trajectories that often exhibit unnatural and non-fluid mechanical characteristics, making it difficult to match the operational logic of real-world scenarios.

[0024] First, existing mechanisms for evaluating the quality of robot motion trajectories are rather crude. Their reward functions or evaluation criteria are typically strongly tied to specific task objectives (such as successful grasping or reaching the target point), making them function-oriented evaluations. This approach can only determine whether a motion trajectory "meets functional standards," but lacks effective quantitative evaluation of key physical characteristics such as the smoothness, efficiency, and safety of the trajectory itself. This results in the system's inability to distinguish the highest-quality trajectory from multiple trajectories that can complete the task. Second, the generation of existing robot motion trajectories generally lacks the ability to finely detect and optimize local anomalies. Whether based on global policy planning or end-to-end generation models, existing methods struggle to effectively identify and locate local defects (such as sudden velocity changes, abnormal joint angle fluctuations, or unnecessary jitter) when local problems occur in the generated motion trajectory sequence. This leads to motion trajectory correction often requiring replanning of the entire sequence or relying on manual intervention, resulting in low optimization efficiency.

[0025] This application will use the optimization of robot motion trajectory as an example for illustration. For ease of understanding, the relevant terms and concepts of the optimization model that may be involved in this application embodiment will be introduced below.

[0026] A reward function is a numerical signal returned by the environment after a model performs an action in a specific state, used to quantify the "goodness" or "badness" of that action. The reward function defines the model's learning objective. It can transform a complex or even fuzzy task into a clear, optimizable mathematical problem. Furthermore, during model training, the reward signal is the direct basis for driving model policy updates.

[0027] The following is combined with Figures 1 to 4 This application describes a method, apparatus, and electronic device for optimizing the motion trajectory of a robot.

[0028] Figure 1 This is one of the flowcharts illustrating the method for optimizing the robot's motion trajectory provided in this application, such as... Figure 1 As shown, the method for optimizing the robot's motion trajectory includes step S100.

[0029] S100: Input the robot's motion trajectory into the optimization model and obtain the optimized motion trajectory output by the optimization model.

[0030] The optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module. The behavior intent inference module extracts features from each action state of the motion trajectory and calculates the reward value for each action state based on the extracted feature vectors. The abnormal behavior detection module identifies the abnormal range based on the reward value and feature vectors. The abnormal behavior repair module corrects the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory. The objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

[0031] The robot's motion trajectory includes the demonstration trajectory or synthesized demonstration trajectory during the training process. The motion trajectory comprises a state sequence and an action sequence. A state sequence consists of multiple consecutive states; for example, a state sequence of... ,in, For the first There are several states. An action sequence consists of multiple consecutive actions; for example, an action sequence is... ,in, For the first There are several actions. There is a one-to-one correspondence between states and actions. A motion trajectory consists of multiple consecutive action states; each action state includes a state and a corresponding action. For example, the motion trajectory is... ,in, For the first Each action state. Furthermore, the motion trajectory also includes labels for the target task, its subtasks, and key node markers for the target task or its subtasks during execution.

[0032] The optimized model includes a behavioral intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module.

[0033] like Figure 2 As shown in the embodiments of this application, the method for optimizing the motion trajectory of a robot includes an offline training stage and an online optimization stage. This application solves the core problems of existing technologies, such as reliance on manual binary evaluation (success / failure), difficulty in identifying abnormal trajectories, and unnatural trajectories after repair. Its core process is divided into an offline training stage and an online optimization stage.

[0034] The offline training phase includes steps 1.1, 1.2, and 1.3.

[0035] Step 1.1: Collect high-quality expert demonstration trajectories that can be directly used for imitation learning training through teleoperation control, motion control algorithms and other technical means, record the robot's state parameters, action commands and task-related data at each time step, and obtain the first sample motion trajectory.

[0036] Step 1.2: Based on the motion trajectory of the first sample, train the reward function network to obtain the behavior intention inference module, and learn the final optimized reward function that can quantify the quality of the motion trajectory.

[0037] Step 1.3: Calculate and determine the detection rules and parameter thresholds for the abnormal behavior detection module based on the statistical characteristics of the expert demonstration trajectory, providing a core evaluation model and judgment basis for the subsequent online optimization process. Based on the statistical characteristics of the standard first sample motion trajectory, calculate and determine the detection rules and parameter thresholds for abnormal action states and abnormal ranges, thus obtaining the abnormal behavior detection module.

[0038] Furthermore, the offline training phase also includes constructing an abnormal behavior repair module. An objective function is constructed to obtain the abnormal behavior repair module. Based on the trained behavioral intent inference module, abnormal behavior detection module, and abnormal behavior repair module, an optimized model is obtained.

[0039] Optionally, a behavioral intent inference module is trained. Initial detection rules and parameter thresholds for abnormal action states and abnormal ranges are determined to obtain an initial abnormal behavior detection module. An initial objective function is constructed to obtain an initial abnormal behavior repair module. Based on the trained behavioral intent inference module, initial abnormal behavior detection module, and initial abnormal behavior repair module, an initial preset model is obtained. The motion trajectories of samples and their optimized labels are obtained to construct labeled training samples. Based on the labeled training samples, the initial preset model is trained to obtain an optimized model.

[0040] The online optimization phase includes steps 2.1, 2.2, and 2.3.

[0041] Step 2.1: When the robot's motion trajectory is input into the optimization model, the behavior intent inference module of the optimization model calculates the reward value of each action state of the motion trajectory, realizing a refined quantitative evaluation of trajectory quality. The behavior intent inference module can accurately calculate the reward value of each action state of the motion trajectory. The reward value can be used to evaluate the standardization (or professionalism) of each action state of the motion trajectory. The behavior intent inference module extracts features from each action state of the motion trajectory to obtain various feature vectors. The behavior intent inference module inputs the feature vectors into the final optimized reward function to obtain the reward value of each action state.

[0042] Step 2.2: The abnormal behavior detection module calls the detection rules and parameter thresholds for abnormal action states and abnormal ranges configured in the offline stage, and marks the abnormal action states, abnormal ranges, and specific abnormal types in the motion trajectory based on the reward value analysis results. The abnormal behavior detection module accurately detects action states in the motion trajectory (e.g., the synthetic demonstration trajectory) that deviate from the intended behavior, contain redundant or suboptimal actions, have uneven motion, or have potential collisions. It provides objective quality assessment results for each action state or step segment, accurately locates specific time steps, actions, or motion trajectory segments that do not meet the "ideal" behavior standard, obtains the abnormal range, and passes the abnormal range to the subsequent abnormal behavior repair module, providing clear guidance for targeted optimization. Optionally, the abnormal behavior detection module has the ability to automatically identify and locate various abnormal action states and abnormal ranges in the synthetic demonstration trajectory, and can achieve anomaly labeling accurate to the time step.

[0043] Step 2.3: The abnormal behavior repair module performs targeted optimization processing on the marked abnormal range, generating a corrected trajectory that meets smoothness requirements and conforms to expert behavioral intent, ultimately outputting the optimized motion trajectory. The abnormal behavior repair module performs precise local correction on the abnormal action states within the identified abnormal range. By improving the behavioral quality of abnormal action states within the abnormal range, the module maximizes its alignment with the expert-level behavioral intent (or standard behavioral intent) defined by the reward function, thereby improving the smoothness, efficiency, and safety of the motion trajectory. The abnormal behavior repair module optimizes the abnormal action states within the abnormal range, with the optimization objective being to minimize the value of the objective function. Based on the optimization results of the abnormal behavior repair module, the optimized motion trajectory is obtained.

[0044] The behavioral intent inference module, abnormal behavior detection module, and abnormal behavior repair module of the optimization model in this application achieve closed-loop optimization of motion trajectory quality through close collaboration. The behavioral intent inference module provides objective quantitative standards for anomaly detection, the abnormal behavior detection module accurately indicates the optimization scope and direction for the abnormal behavior repair module, and the abnormal behavior repair module achieves precise improvement of trajectory quality based on the output results of the former two.

[0045] The robot motion trajectory optimization method provided in this application achieves refined quality evaluation of the motion trajectory by calculating the reward value of each action state. It accurately locates the abnormal range and abnormal action states through reward values ​​and feature vectors. By using an objective function to locally optimize the abnormal action states within the abnormal range, it ensures that the optimized motion trajectory conforms to the standard intent of expert behavior, guarantees the task objective, and improves the smoothness of the motion trajectory and environmental safety. This application achieves a closed-loop optimization mechanism for motion trajectory from quality quantification to anomaly localization to smooth repair through deep collaboration between the behavior intent inference module, the abnormal behavior detection module, and the abnormal behavior repair module. This effectively reduces abnormal trajectory segments in existing generated robot motion trajectories. The optimized motion trajectory output by this application finally meets the high-quality data requirements of robot imitation learning training grounds, providing reliable technical support for overcoming the bottleneck of scarce training data in the field of imitation learning.

[0046] The need for high-quality imitation learning data is crucial for robot training, as this data directly impacts the robot's performance and effectiveness in tasks such as complex operations, motion planning simulation, and policy model training. However, in practical applications, due to limitations in motion trajectory generation technology, a lack of expert demonstration data, and interference from complex scenes, existing generated motion trajectories often suffer from problems such as motion redundancy, motion stuttering, inefficient paths, and potential collisions. These issues severely restrict the efficiency and quality of robot imitation learning, resulting in trained robot policies that fail to meet the needs of real-world scenarios.

[0047] The robot motion trajectory optimization method provided in this application achieves end-to-end optimization of motion trajectory quality through a two-stage architecture of offline training and online optimization. The offline training stage relies on high-quality expert demonstration trajectories to complete the reward function training for the behavioral intent inference module and the configuration of detection rules and parameter thresholds for the abnormal behavior detection module, laying a quantitative evaluation foundation for subsequent optimization. The online optimization stage calculates the reward values ​​of each action state by the behavioral intent inference module, achieving refined quantification of motion trajectory quality. Utilizing the multi-dimensional detection rules and parameter thresholds of the abnormal behavior detection module, the scope and type of anomalies are accurately analyzed and located. Finally, the abnormal behavior repair module performs localized optimization of abnormal action states within the abnormal range based on the objective function, ensuring that the corrected motion trajectory conforms to expert intent, guarantees task objectives, and improves motion smoothness and environmental safety. The optimization model's behavioral intent inference module, abnormal behavior detection module, and abnormal behavior repair module work together in a deep collaboration to form a closed-loop optimization mechanism from quality quantification to anomaly localization to smooth repair. This effectively solves the quality problems of motion redundancy, motion stuttering, and path inefficiency in existing generated motion trajectories. Finally, it outputs high-quality synthetic data that meets the needs of robot imitation learning training grounds, providing reliable technical support for breaking through the bottleneck of training data scarcity in the field of imitation learning.

[0048] Based on the above embodiments, the feature vector is extracted in the following manner: Feature extraction is performed on the action state to obtain the target Euclidean distance, the robot's average joint angle change rate, the robot's end effector speed smoothness, the task completion degree of the action state, the task deviation degree of the action state, and the number of collisions; the target Euclidean distance is the Euclidean distance between the robot's end effector and the task target; The feature vector is obtained by concatenating the target Euclidean distance, average joint angle change rate, velocity smoothness, task completion degree, task deviation degree, and number of collisions.

[0049] The behavior intent inference module extracts features from each action state of the motion trajectory, generating feature vectors for each action state. Action states include state, action (action command), and task objective (or sub-task objective). State includes the robot's end effector pose, motion velocity, joint angles, joint angular velocities, gripper opening / closing state, and the pose information of objects in the environment. Action commands encompass robot joint drive commands, end effector motion control commands, etc. The task objective (or sub-task objective) includes sub-task objective classification labels and key node markers during the execution of the sub-task objective.

[0050] The position of the robot's end effector is obtained based on the action state. The Euclidean distance between the robot's end effector position and the target position is calculated to obtain the target Euclidean distance. The formula for calculating the target Euclidean distance is as follows.

[0051] ; in, The target Euclidean distance, This refers to the position of the robot's end effector. The location of the mission objective.

[0052] Based on the motion state, obtain the rate of change of each joint angle of the robot. The average rate of change of the robot's joint angle is obtained by averaging all the rates of change. The formula for calculating the average rate of change of joint angle is as follows.

[0053] ; in, For the robot's first Rate of change of joint angle For the robot's first Each joint From the perspective of time, For the robot's first Each joint From the perspective of time, The sampling interval between two adjacent action states of the robot. This represents the total number of joints in the robot. This represents the average rate of change of joint angle.

[0054] Calculate the velocity smoothness of the robot's end effector. Based on the velocity difference between two consecutive frames, calculate the instantaneous acceleration of the robot in each motion state. Then, using the acceleration variance within a sliding window, calculate the velocity smoothness of the robot's end effector. The velocity smoothness of the robot's end effector is calculated based on the instantaneous acceleration of the robot's current time step, the instantaneous acceleration of the previous multiple time steps (e.g., the instantaneous acceleration of the previous four time steps), and the average acceleration within the sliding window. The formula for calculating velocity smoothness is as follows.

[0055] ; in, For the speed smoothness of the robot's end effector, Let be the instantaneous acceleration of the robot at its current time step. , , and These are the instantaneous accelerations for the first four time steps. This represents the average acceleration within the sliding window.

[0056] The task completion rate of an action state measures the progress of the current task. The closer the task completion rate is to 1, the closer the task objective (or sub-task objective) is to completion. The task completion rate is calculated based on the distance from the end effector corresponding to the action state to the target object (e.g., the task objective is to pick up the target object), and the distance from the end effector to the target object at the start of the task objective. The formula for calculating the task completion rate is as follows.

[0057] ; in, The task completion rate in the action state. The distance from the end effector to the target object during the action state. The distance from the end effector to the target object at the start of the task objective.

[0058] Calculate the task deviation from the motion state. Extract the Euler angles (roll, pitch, and yaw) of the robot's end effector from the motion state. Obtain the Euler angles (target posture) when the robot completes the task objective. Completing the task objective includes the robot's end effector picking up the target object. Calculate the absolute value of the deviation based on the Euler angles of the end effector and the Euler angles when the robot completes the task objective, obtaining the task deviation. The task deviation measures the degree of alignment between the robot's posture and the target posture when the robot completes the task objective. Euler angles include yaw, pitch, and roll. The formula for calculating the task deviation is as follows.

[0059] ; in, For task deviation, The yaw angle of the robot's end effector, extracted from the action state. The pitch angle of the robot's end effector, extracted from the motion state. The roll angle of the robot's end effector, extracted from the action state. The yaw angle represents the robot's target orientation. The pitch angle represents the robot's target orientation. The roll angle is the target orientation of the robot.

[0060] Calculate the number of collisions for each action state. Based on the action state, extract the contact force of the robot's end effector at each moment. When the contact force exceeds a contact force threshold, increment the collision count for that action state by 1. The collision count is used to measure the safety of the action state. The formula for calculating the collision count is as follows.

[0061] ; in, For the robot's end effector in The number of collisions at any given moment For the robot's end effector in The number of collisions at any given moment The contact force of the robot's end effector. This is the contact force threshold.

[0062] The target Euclidean distance, average joint angle change rate, velocity smoothness, task completion rate, task deviation rate, and number of collisions are concatenated to obtain the feature vector. The formula for calculating the feature vector is as follows.

[0063] ; in, Action state eigenvectors, For state, For action, The target Euclidean distance, This represents the average rate of change of joint angle. For speed smoothness, For task completion rate, For task deviation, This represents the number of collisions.

[0064] Task completion rate includes the degree of completion of the task objective of the motion trajectory, or the degree of completion of sub-tasks within the task objective. Task deviation rate includes the degree of deviation of the robot from the task objective, or the degree of deviation of the robot from the sub-tasks within the task objective.

[0065] This application determines feature vectors based on target Euclidean distance, average joint angle change rate, velocity smoothness, task completion degree, task deviation degree, and number of collisions. This not only extracts task completion information when the robot executes the task objective, but also extracts fine-grained quality information such as the efficiency, smoothness, and safety of the robot's action state during task execution. Subsequently, reward values ​​are calculated based on feature vectors, achieving precise quantification of quality differences in motion trajectories in terms of physical characteristics, task fit, and environmental interaction safety.

[0066] This application achieves a refined and quantitative evaluation of motion trajectory quality, overcoming the limitations of existing technologies in identifying defects in motion trajectories. Existing technologies use binary labels or single task objectives to construct reward functions, only able to determine whether the motion trajectory "completed the task," unable to distinguish fine-grained quality differences such as "efficiency, smoothness, and safety." This application, by extracting multi-dimensional kinematic features and constructing feature vectors, can accurately quantify the quality differences of motion trajectories in terms of physical characteristics, task fit, and environmental interaction safety.

[0067] Based on the above embodiments, the behavior intent inference module is trained in the following manner: The robot's expert demonstration trajectory was used as the first sample motion trajectory; The sample task and reward function are input into the policy network to obtain the second sample motion trajectory output by the policy network. The policy network is used to generate at least one trajectory data based on the sample task and calculate the cumulative trajectory reward value of each trajectory data according to the reward function. The trajectory data with the largest cumulative trajectory reward value is taken as the second sample motion trajectory. Based on the first and second sample motion trajectories, the reward function of the reward function network is optimized and trained to obtain the optimized reward function. During the optimization training process, the reward function network generates the first sample cumulative reward value of the first sample motion trajectory and the second sample cumulative reward value of the second sample motion trajectory. Based on the comparison result of the first sample cumulative reward value and the second sample cumulative reward value, the first sample motion trajectory is identified as a standard behavior trajectory, and the second sample motion trajectory is identified as a non-standard behavior trajectory. Based on the optimized reward function, an optimized reward function network is obtained to obtain the behavioral intent inference module.

[0068] Constructing high-quality expert demonstration trajectories for the robot. High-quality expert demonstration trajectories for the robot are acquired using techniques such as teleoperation control and motion control algorithms. The robot's sample state, sample actions, and sample task labeling data are recorded at each time step to obtain the first sample motion trajectory. The first sample motion trajectory includes at least one first sample motion trajectory. The first sample motion trajectory consists of multiple consecutive sample action states, where each sample action state includes a sample state and a corresponding sample action. For example, the first sample motion trajectory is... ,in, For the first Individual sample action state, For the first Each sample state For the first Each sample action. Furthermore, the first sample motion trajectory also includes the sample task, the labeling of the sample subtasks of the sample task, and the marking of key sample nodes during the execution of the sample task or sample subtask.

[0069] The expert demonstration trajectories need to undergo rigorous screening to ensure high quality, ensuring that the first sample motion trajectory has efficient, smooth and collision-free characteristics, so as to provide a reliable sample basis for the subsequent training of the reward function network.

[0070] Policy Network The parameterization is performed using a deep neural network structure. The policy network acts as the generator, implemented using a multilayer perceptron. The policy network represents the robot's current behavioral policy, i.e., given the state. Output the action to be performed at the time. Its core optimization objective is to maximize the cumulative reward value obtained from the current reward function. The policy network receives the current state. , to obtain specific actions The learnable parameters of the policy network, such as weights and biases, are determined by... It indicates that dynamic adjustments are made through training. To optimize the behavioral decision-making capabilities of the policy network.

[0071] Reward function network The parameterization is performed as a deep neural network structure. The reward function network acts as a discriminator, implemented using a multilayer perceptron. The reward function network uses the sample motion trajectory... As input, the output is a scalar representing the quality of the behavior at the current moment, namely the sample reward value. The reward function network includes a reward function. The reward function extracts features from each sample's action state in the sample motion trajectory, obtaining a feature vector for each sample. This feature vector is obtained by concatenating the sample's target Euclidean distance, average joint angle change rate, velocity smoothness, task completion rate, task deviation rate, and collision count for each sample's action state. The reward function then operates on these feature vectors to obtain the reward value for each sample's action state. The learnable parameters of the reward function network, such as weights and biases, are determined by... This indicates that through continuous iterative optimization during training, the reward function is made capable of accurately distinguishing different quality behaviors.

[0072] The training process of the behavioral intent inference module includes the optimization of the policy network and the updating of the reward function.

[0073] First, the reward function network... parameters and policy networks parameters Perform initialization settings.

[0074] The optimization of the policy network includes the following steps: Input the sample task and the initially set reward function into the policy network. Train the policy network under the guidance of the reward function. The policy network learns to optimize action execution to maximize the cumulative reward value obtained from the current reward function. Specifically, the sample task is used to perform large-scale interaction with the robot and the scene in a simulation environment, collecting at least one trajectory data generated during the execution of the sample task to construct a trajectory dataset. Using a standard reinforcement learning algorithm (e.g., PPO), the policy network is trained based on the trajectory dataset and the reward signal (e.g., cumulative trajectory reward value) generated under the current reward function. parameters Iterative optimization is performed. The core objective of this stage is to improve the adaptability of the policy network in making decisions based on the current reward function. The policy network calculates the cumulative reward value of each trajectory data according to the reward function, and selects the trajectory data with the largest cumulative reward value as the second sample motion trajectory. .

[0075] The update of the reward function includes the following steps. Based on the first sample motion trajectory With the second sample trajectory For reward function parameters The reward function is updated to more accurately distinguish between "expert-level high-quality behavior" (first sample trajectory) and "strategy-generated suboptimal behavior" (second sample trajectory), ultimately outputting an optimized reward function that can reliably quantify behavior quality. The first and second sample trajectories are input into the reward function network. The reward function network generates a first-sample cumulative reward value for the first sample trajectory and a second-sample cumulative reward value for the second sample trajectory. The reward function network compares the first-sample cumulative reward value and the second-sample cumulative reward value. If the first-sample cumulative reward value is greater than the second-sample cumulative reward value, the first sample trajectory is identified as a standard behavior trajectory, and the second sample trajectory is identified as a non-standard behavior trajectory.

[0076] During the update process of the reward function, the reward function parameters The system will achieve a two-way optimization goal through updates: on the one hand, it will maximize the first sample cumulative reward value of the expert trajectory (the first sample motion trajectory), and on the other hand, it will relatively reduce the second sample cumulative reward value of the second sample motion trajectory generated by the policy network, thereby strengthening the distinction between high-quality behavior and suboptimal behavior.

[0077] Optionally, during the update of the reward function, a first feature expectation is generated based on the average of all sample feature vectors of the first sample's motion trajectory. A second feature expectation is generated based on the average of all sample feature vectors of the second sample's motion trajectory. The difference between the first and second feature expectations is used to verify the reasonableness of the first and second sample cumulative reward values. The calculation formulas for the first and second feature expectations are as follows.

[0078] ; in, The expectation of the first feature, For the second feature expectation, This represents the number of sample motion trajectory segments in the first sample's motion trajectory. This represents the number of sample motion trajectory segments in the second sample motion trajectory. These are the sample motion trajectory segments belonging to the first sample motion trajectory. This refers to a segment of the motion trajectory of a sample that belongs to the second sample's motion trajectory. This is the feature vector of the sample's action state in the sample's motion trajectory segment. The time step for each sample motion trajectory segment.

[0079] Optionally, compare the expected value of the first feature and the expected value of the second feature, and calculate the difference in expected value. Based on the difference in expected value, verify the reasonableness of the cumulative reward value of the first sample and the cumulative reward value of the second sample. For example, when the difference in expected value is large, the difference in the cumulative reward value of the first sample and the cumulative reward value of the second sample should also be large. When the difference in expected value is small, the difference in the cumulative reward value of the first sample and the cumulative reward value of the second sample should also be small.

[0080] Based on the optimized reward function, an optimized reward function network is obtained to obtain the behavioral intent inference module.

[0081] This application generates a second sample motion trajectory by training a policy network, providing a training sample basis for the subsequent training of the reward function network. Based on the first and second sample motion trajectories, this application trains the reward function network, enabling it to recognize both standard and non-standard behavioral trajectories, thus optimizing the reward function. This application combines maximum entropy inverse reinforcement learning to train a continuous-value reward function, which can accurately quantify the quality differences in motion trajectories in terms of physical characteristics, task fit, and environmental interaction safety.

[0082] Based on the above embodiments, the reward function network is further used to generate a first feature expectation of the first sample motion trajectory and a second feature expectation of the second sample motion trajectory. After obtaining the optimized reward function, the following steps are also included: Input the sample task and the optimized reward function into the policy network to obtain the optimized second sample motion trajectory output by the policy network; Based on the first sample motion trajectory and the optimized second sample motion trajectory, the optimized reward function is further optimized and trained. The policy network and reward function are trained alternately and iteratively until the difference between the expectation of the first feature and the expectation of the second feature is less than a set difference. Then the training ends and the final optimized reward function is obtained. Based on the final optimized reward function, obtain the final optimized reward function network to obtain the behavioral intent inference module.

[0083] The training process of the behavioral intent inference module involves alternating iterations of policy network optimization and reward function updates.

[0084] After obtaining the optimized reward function, the sample task and the optimized reward function are input into the policy network to obtain the optimized second sample motion trajectory output by the policy network, so as to update the policy network again.

[0085] Based on the first sample motion trajectory and the optimized second sample motion trajectory, the reward function network (or reward function) is retrained and optimized.

[0086] The policy network and reward function are iteratively optimized and trained alternately according to the above steps until the difference between the expected value of the first feature and the expected value of the second feature is less than the set difference. Then the iterative optimization and training ends and the final optimized reward function is obtained.

[0087] Based on the final optimized reward function, obtain the final optimized reward function network to obtain the behavioral intent inference module.

[0088] After each reward function update, the optimized reward function further enhances its ability to identify optimal behavior (standard behavior trajectory) and its penalty mechanism for suboptimal behavior (non-standard behavior trajectory). After each update of the policy network, the policy network can generate a second sample motion trajectory that is closer to the first sample motion trajectory based on the optimized reward function and the sample task.

[0089] As the difference between the expected first feature of the first sample motion trajectory and the expected second feature of the second sample motion trajectory becomes smaller, the reward function becomes more and more accurate in identifying the first sample motion trajectory as a standard behavior trajectory and the second sample motion trajectory as a non-standard behavior trajectory.

[0090] This application achieves progressive enhancements in the reward function's ability to identify standard behaviors (optimal behaviors) and its penalty mechanism for suboptimal behaviors (non-standard behaviors) through iterative optimization of the policy network and reward function. Ultimately, a highly reliable reward function is obtained. The optimized reward function can accurately quantify and evaluate the quality of any robot's motion trajectory segments, providing a core decision-making basis for the subsequent correction and optimization of synthesized demonstration trajectories.

[0091] The final optimized reward function outputs a reward value for the current action state upon receiving the robot's current state and current action. This optimized reward function accurately quantifies the "ideality" of the action state, assigning high reward values ​​to efficient, smooth, collision-free actions that align with the core intent of the task, while giving low reward values ​​or even penalized scores to suboptimal actions, redundant behaviors, or flawed operations.

[0092] Based on the above embodiments, the anomaly range is determined in the following manner: When the reward value of an action state is lower than the instantaneous reward threshold, the action state is identified as an abnormal action state. When the cumulative reward value of at least one consecutive action state is lower than the cumulative reward threshold, at least one consecutive action state is identified as an abnormal segment. When the rate of change of the reward value of an action state exceeds the range of the reward threshold, the action state is identified as an abnormal action state; or, when the rate of change of at least one reward value of at least one consecutive action state exceeds the range of the reward threshold, at least one consecutive action state is identified as an abnormal segment. If a collision event is detected in an action state based on the feature vector of the action state, the action state will be identified as an abnormal action state. Determine the scope of the anomaly based on the abnormal action state and / or abnormal fragment.

[0093] In the online optimization phase of the motion trajectory, the robot's motion trajectory is input into the optimization model. The behavior intent inference module of the optimization model extracts features from each action state of the motion trajectory, obtaining feature vectors for each action state. The behavior intent inference module then inputs the feature vectors of each action state into the final optimized reward function, obtaining the reward value for each action state output by the final optimized reward function.

[0094] The abnormal behavior detection module calls upon the detection rules and parameter thresholds for abnormal action states and abnormal ranges configured in the offline phase, analyzes the reward value of each action state, and identifies abnormal action states and abnormal ranges. An abnormal range includes one abnormal action state, or two or more consecutive abnormal action states.

[0095] The detection rules for abnormal action states and abnormal ranges include four major analysis dimensions: instantaneous reward analysis rules, cumulative reward analysis rules, reward trend analysis rules, and collision detection rules. Parameter thresholds include instantaneous reward threshold, cumulative reward threshold, and the range of reward threshold changes.

[0096] Based on instantaneous reward analysis rules, abnormal motion states are identified. For each motion state in the motion trajectory... (Alternatively, time step), set the instantaneous reward threshold. If the reward value of an action state is lower than the instantaneous reward threshold, the action state is identified as an abnormal action state. Instantaneous reward analysis rules can detect prolonged inefficient behavior, unnecessary detours, and action states (or the execution process of subtasks of the task objective) with low overall quality.

[0097] Based on the cumulative reward analysis rules, abnormal segments are identified. The cumulative reward value is calculated for each segment of the motion trajectory. The motion trajectory segment includes at least one continuous motion state. A cumulative reward threshold is set. If the cumulative reward value of a motion trajectory segment is lower than the cumulative reward threshold, the motion trajectory segment is identified as an anomalous segment. The cumulative reward analysis rule can detect long-term inefficient behavior, unnecessary detours, and motion trajectory segments with low overall quality.

[0098] Abnormal motion states or abnormal segments are identified based on reward trend analysis rules. The reward value curve of the motion trajectory is smoothed using the moving average method, and abnormal motion states or abnormal segments are identified by analyzing the change trend of reward value over time. When the rate of change of reward value of an motion state exceeds the reward threshold change range, the motion state is identified as an abnormal motion state. Alternatively, when the rate of change of reward value of each consecutive motion state exceeds the reward threshold change range, the consecutive motion states are identified as abnormal segments. The rate of change of reward value exceeding the reward threshold change range includes the following situations: (1) A sharp drop in reward value in a short period of time may indicate a serious error (such as a collision). (2) A long-term low level of reward value indicates that the robot is trapped in a suboptimal loop or executes an inefficient path. (3) Abnormal large fluctuations in reward value may reflect the robot's motion instability or decision ambiguity.

[0099] Abnormal action states are identified based on collision detection rules. The number of collisions for each action state is extracted from its feature vector to identify the collision events. Any collision event is marked as a serious anomaly. If a collision event exists in an action state, it is identified as an abnormal action state.

[0100] Based on the abnormal action states and / or abnormal segments identified above, the scope of the abnormality is determined.

[0101] Furthermore, the abnormal behavior detection module also outputs the anomaly type and its score for each anomaly range. Anomaly types include low reward, collisions, and non-smooth motion. The anomaly score provides a quantitative score for each detected anomaly range, representing the severity of the anomaly. The anomaly range includes the motion state at the start and end time steps of the motion trajectory that require correction.

[0102] This application uses multi-dimensional detection rules, such as instantaneous reward analysis rules, cumulative reward analysis rules, reward trend analysis rules, and collision detection rules, as well as parameter thresholds to identify abnormal ranges. It performs detection based on reward values ​​that can reflect the quality and intent of behavior, enabling the abnormal behavior detection module to not only identify physically erroneous behaviors, but also to accurately identify and quantify those suboptimal behaviors that are physically without obvious errors but are inefficient (such as redundant actions, detours, etc.).

[0103] Traditional motion trajectory generation methods struggle to effectively detect and accurately identify unnatural movements, redundant actions, inefficient paths, potential collision risks, or abnormal / suboptimal behaviors that deviate from the intended behavior due to defects in the source demonstration quality. These defects directly limit the effectiveness of motion trajectories in robot imitation learning. The abnormal behavior detection module in this application, through quantitative analysis and a multi-dimensional detection mechanism, provides key technical support for overcoming this limitation.

[0104] This application achieves precise localization and type identification of anomalies, solving the problem of insufficient anomaly detection capabilities in existing technologies. Current motion trajectory generation lacks a dedicated anomaly detection module. This application designs a multi-dimensional anomaly range detection mechanism, which, through the coordinated operation of four dimensions—instantaneous reward analysis rules, cumulative reward analysis rules, reward trend analysis rules, and collision detection rules—can accurately mark abnormal action states, abnormal segments, and specific types, achieving full-type anomaly localization from physical errors to inefficient behaviors, providing a clear basis for subsequent targeted repair.

[0105] Based on the above embodiments, at least one correction term includes a smoothness penalty term, a task target preservation term, a boundary connection term, and a collision penalty term; the smoothness penalty term is used to ensure that the optimized motion trajectory is smooth; the task target preservation term is used to ensure that the optimized motion trajectory focuses on the task target; the boundary connection term is used to achieve a smooth transition between the corrected state and its actions within the abnormal range and the uncorrected state and its actions; the collision penalty term is used to reduce the collision risk in the optimized motion trajectory; the objective function is determined based on the following method: Based on the corrected weighting coefficients, the smoothness penalty term, the task objective preservation term, the boundary connection term, and the collision penalty term are weighted and summed to obtain the weighted correction term. The objective function is determined based on the weighted adjustment term and the target cumulative reward value.

[0106] Once in the online optimization phase, when a new motion trajectory is input, the behavioral intent inference module calculates the reward value for each action state of the trajectory, achieving a refined and quantitative evaluation of the trajectory's quality. The abnormal behavior detection module calls upon the detection rules and parameter thresholds configured in the offline phase, combines the reward values ​​to analyze and mark abnormal action states, abnormal segments, and specific abnormal types within the trajectory. The abnormal behavior repair module performs targeted optimization processing on the marked abnormal range, generating a corrected trajectory that meets smoothness requirements and conforms to expert behavioral intent, ultimately outputting the optimized motion trajectory.

[0107] The abnormal behavior repair module receives the abnormal range output by the abnormal behavior detection module, uses the reward value learned by the behavior intent inference module as the core optimization target, and performs automatic correction on the marked abnormal range, ultimately generating a smooth and efficient optimized motion trajectory for the robot. The abnormal behavior repair module performs precise local correction on the instantaneous abnormal points (abnormal action states) and suboptimal trajectory segments (abnormal fragments) identified by the abnormal behavior detection module. The specific implementation steps are as follows.

[0108] Identify the specific time range (including start time) in the motion trajectory that needs correction based on the anomaly range. and end time This includes the specific time frame and the corresponding continuous abnormal action states within that time frame. For example, the continuous abnormal action states within the abnormal range are... .in, The first in the abnormal range Abnormal action states at time steps This represents the initial abnormal action state within the abnormal range. This represents the end of an abnormal action state within the abnormal range. Each abnormal action state... It includes information such as the robot's end effector's 3D position, 3D orientation, and gripper status. The abnormal behavior repair module corrects abnormal motion states within the abnormal range according to the objective function, resulting in an optimized motion trajectory.

[0109] The objective function is determined as follows: Based on adjusted weight coefficients, a weighted sum is performed on the smoothness penalty term, the task objective preservation term, the boundary connectivity term, and the collision penalty term to obtain a weighted adjusted term. The objective function is then determined based on the weighted adjusted term and the cumulative target reward value. The adjusted weight coefficients include those for the smoothness penalty term, the task objective preservation term, the boundary connectivity term, and the collision penalty term. The formula for calculating the objective function is as follows.

[0110] ; in, This represents the start time of the abnormal range. This represents the end time of the abnormal range. The composite cost value is within the abnormal range. Accumulated reward value for the target of abnormal action states (including continuous and non-continuous abnormal action states) within the abnormal range. Abnormal action state The corresponding state, Abnormal action state The corresponding action, The corrected weighting coefficient for the smoothness penalty term. The corrected weighting coefficients for the boundary connectivity terms. The corrected weighting coefficient for the collision penalty term. Adjusted weighting coefficients for items that maintain the task objective. As a smoothness penalty term, For boundary connection terms, For collision penalties, Maintain the item for the task objective.

[0111] This represents the reward maximization term, which accumulates reward values ​​by maximizing the learned target, driving the movement trajectory to correct towards the ideal behavior.

[0112] The smoothness penalty term ensures the smoothness of the end effector's motion process by penalizing and correcting the velocity, acceleration, and sum of squares of acceleration of the motion trajectory in the operating space.

[0113] The mission objective preservation term penalizes the operational space deviation between the key motion states (e.g., grab points, placement points, etc.) of the corrected motion trajectory and the keyframes of the original motion trajectory, ensuring that the core mission objective is not affected by the correction.

[0114] Boundary connectivity terms penalize and correct the abnormal range of the motion trajectory by adjusting the initial abnormal action state (starting point). And end abnormal action state (end point) The pose difference between the connection point and the original motion trajectory in the operating space enables a smooth transition between the corrected abnormal range and the uncorrected area.

[0115] The collision penalty is applied based on the depth of penetration between the robot and the environment, effectively avoiding the risk of collision.

[0116] Calculate the composite cost value of the abnormal range based on the objective function. Correct the abnormal action states within the abnormal range based on the composite cost value to obtain the corrected abnormal range. Calculate the composite cost value of the corrected abnormal range based on the objective function. Iteratively correct the abnormal action states within the abnormal range until the composite cost value of the abnormal range reaches its minimum or tends to stabilize, then stop correcting the abnormal range. Obtain the optimized motion trajectory based on all corrected abnormal ranges.

[0117] Furthermore, during the operation space optimization process, a hard constraint must be imposed to ensure the physical feasibility of the motion trajectory: the inverse kinematics constraint requires that for each end effector's action state in the decision variables, there must exist a corresponding robot joint pose that can achieve that pose. This nonlinear constraint guarantees the reachability of the trajectory from a mechanical feasibility perspective. The optimal end effector pose sequence is solved through iterative search, minimizing the objective function while satisfying all constraints. After optimization, the optimized motion trajectory is output and converted into a joint pose sequence in joint space by the inverse kinematics solver. Additionally, at the connection points... and Interpolation smoothing technology is used to ensure that the final output joint trajectory remains continuous and smooth.

[0118] This application constructs an objective function for abnormal ranges, which includes a cumulative reward value, a smoothness penalty term, a task objective preservation term, a boundary connectivity term, and a collision penalty term. While retaining the original data of non-abnormal ranges, it only corrects abnormal ranges, which can meet the efficiency requirements of rapid optimization of large-scale training data.

[0119] This application achieves localized targeted repair of motion trajectories, improving optimization efficiency and trajectory naturalness. Existing motion trajectory adjustments rely on global replanning or manual correction, resulting in low optimization efficiency. This application proposes a localized targeted repair strategy, targeting the abnormal range marked by anomaly detection, using an objective function as the optimization objective, and only performing corrections on abnormal segments (i.e., abnormal trajectory fragments). This application avoids ineffective adjustments to non-abnormal regions, improving the quality of the abnormal range while ensuring the continuity and smoothness of the corrected trajectory fragments with the original motion trajectory through boundary connectivity terms and smoothness penalty terms. The repair efficiency is significantly higher than global replanning, and the optimized motion trajectory better reflects the natural characteristics of expert behavioral intentions.

[0120] The optimized model in this application, through its efficient optimized architecture, is adapted to the needs of large-scale imitation learning data generation. Existing methods for generating and optimizing motion trajectories are coupled, making it difficult to meet the efficiency requirements of large-scale data generation. This application employs a two-stage architecture of offline training and online optimization. Complex reward function training, detection rules, and parameter threshold configuration are handled in the offline training stage, while the online optimization stage performs only lightweight quantization evaluation and local optimization. This balances evaluation accuracy and data generation efficiency, enabling efficient output of large-scale, high-quality training data, thus meeting the batch data requirements of robot imitation learning training environments.

[0121] In fields such as robot R&D, intelligent manufacturing, and service robots, high-quality imitation learning data is the core foundation for driving robot strategy optimization and technology implementation. Currently, the industry generally faces the problems of high cost of collecting expert demonstration data and low quality of synthesized data, resulting in low robot training efficiency and insufficient strategy generalization ability. This application, through a full-process motion trajectory optimization system, can transform low-quality motion trajectories into high-quality training data that meets the standards of "efficiency, fluency, and safety," significantly reducing the reliance on human expert demonstrations and providing a stable supply of high-quality data for robot training grounds and intelligent robot R&D companies, thus solving the core bottleneck of training data scarcity in the robot training process.

[0122] The robot motion trajectory optimization method provided in this application has strong versatility and adaptability, and can be widely applied to various robot task scenarios. The robot simulation training platform, as the core data optimization module of the simulation platform, provides standardized, high-quality training data for robot training in a virtual environment, reducing the trial-and-error costs of physical robots. In the field of industrial robots, for tasks such as robotic arm grasping and assembly, it optimizes the smoothness and efficiency of synthetic trajectories, improving the operational accuracy and work efficiency of industrial robots.

[0123] This application significantly improves the data quality of motion trajectories through technological innovations such as refined quantitative evaluation and precise anomaly repair, thereby enhancing the efficiency and performance of robot strategy learning. The motion trajectory optimization method of this application reduces investment in expensive expert demonstration equipment and manual teaching, lowering the cost of generating high-quality motion data. Simultaneously, robot strategies supported by high-quality data possess stronger environmental adaptability and task reliability, helping companies launch more competitive intelligent robot products and accelerating the commercialization of the technology. Furthermore, the offline training and online optimization architecture of this application supports large-scale batch data processing, meeting the efficient data requirements of industrial-grade mass production scenarios and further enhancing the value of industrial applications.

[0124] The following describes the robot motion trajectory optimization device provided in this application. The robot motion trajectory optimization device described below and the robot motion trajectory optimization method described above can be referred to in correspondence.

[0125] like Figure 3 As shown, a device for optimizing the motion trajectory of a robot includes: The trajectory optimization module 301 is used to input the robot's motion trajectory into the optimization model and obtain the optimized motion trajectory output by the optimization model. The optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module. The behavior intent inference module extracts features from each action state of the motion trajectory and calculates the reward value for each action state based on the extracted feature vectors. The abnormal behavior detection module identifies the abnormal range based on the reward value and feature vectors. The abnormal behavior repair module corrects the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory. The objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

[0126] The robot motion trajectory optimization device provided in this application achieves refined quality evaluation of the motion trajectory by calculating the reward value of each action state. It accurately locates the abnormal range and abnormal action states through reward values ​​and feature vectors. By using an objective function to locally optimize the abnormal action states within the abnormal range, it ensures that the optimized motion trajectory conforms to the standard intent of expert behavior, guarantees the task objective, and improves the smoothness of the motion trajectory and environmental safety. This application achieves a closed-loop optimization mechanism for motion trajectory from quality quantification to abnormal location and smooth repair through deep collaboration between the behavior intent inference module, abnormal behavior detection module, and abnormal behavior repair module. This effectively reduces abnormal trajectory segments in existing generated robot motion trajectories. The optimized motion trajectory output by this application meets the high-quality data requirements of robot imitation learning training grounds, providing reliable technical support for overcoming the bottleneck of scarce training data in the field of imitation learning.

[0127] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0128] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for optimizing the robot's motion trajectory. This method includes: inputting the robot's motion trajectory into an optimization model and obtaining the optimized motion trajectory output by the optimization model; wherein the optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module; the behavior intent inference module is used to extract features from each action state of the motion trajectory and calculate the reward value of each action state based on the extracted feature vectors; the abnormal behavior detection module is used to identify the abnormal range based on the reward value and feature vectors; the abnormal behavior repair module is used to correct the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory; the objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

[0129] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robot motion trajectory optimization method provided by the above methods. The method includes: inputting the robot's motion trajectory into an optimization model and obtaining the optimized motion trajectory output by the optimization model; wherein, the optimization model includes a behavior intention inference module, an abnormal behavior detection module, and an abnormal behavior repair module; the behavior intention inference module is used to extract features from each action state of the motion trajectory and calculate the reward value of each action state based on the extracted feature vectors; the abnormal behavior detection module is used to identify the abnormal range based on the reward value and feature vectors; the abnormal behavior repair module is used to correct the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory; the objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

[0131] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for optimizing the motion trajectory of a robot provided by the methods described above. This method includes: inputting the robot's motion trajectory into an optimization model and obtaining the optimized motion trajectory output by the optimization model; wherein the optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module; the behavior intent inference module is used to extract features from each action state of the motion trajectory and calculate the reward value for each action state based on the extracted feature vectors; the abnormal behavior detection module is used to identify an abnormal range based on the reward value and feature vectors; the abnormal behavior repair module is used to correct the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory; the objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for optimizing the motion trajectory of a robot, characterized in that, include: Input the robot's motion trajectory into the optimization model, and obtain the optimized motion trajectory output by the optimization model; The optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module. The behavior intent inference module extracts features from each action state of the motion trajectory and calculates the reward value for each action state based on the extracted feature vectors. The abnormal behavior detection module identifies an abnormal range based on the reward value and the feature vectors. The abnormal behavior repair module corrects the abnormal action states within the abnormal range based on an objective function to obtain the optimized motion trajectory. The objective function is determined based on the target cumulative reward value of the abnormal action states within the abnormal range and at least one correction term.

2. The method for optimizing the motion trajectory of a robot according to claim 1, characterized in that, At least one of the correction terms includes a smoothness penalty term, a task target preservation term, a boundary connection term, and a collision penalty term; the smoothness penalty term ensures the optimized motion trajectory is smooth; the task target preservation term ensures the optimized motion trajectory focuses on the task target; the boundary connection term enables a smooth transition between the corrected state and its actions within the abnormal range and the uncorrected state and its actions; the collision penalty term reduces the collision risk in the optimized motion trajectory; the objective function is determined based on the following method: Based on the corrected weighting coefficients, the smoothness penalty term, the task objective preservation term, the boundary connection term, and the collision penalty term are weighted and summed to obtain the weighted correction term. The objective function is determined based on the weighted correction term and the target cumulative reward value.

3. The method for optimizing the motion trajectory of a robot according to claim 1, characterized in that, The feature vectors were extracted in the following manner: Feature extraction is performed on the action state to obtain the target Euclidean distance, the robot's average joint angle change rate, the robot's end effector speed smoothness, the task completion degree of the action state, the task deviation degree of the action state, and the number of collisions; the target Euclidean distance is the Euclidean distance between the robot's end effector and the task target; The feature vector is obtained by concatenating the target Euclidean distance, the average joint angle change rate, the velocity smoothness, the task completion degree, the task deviation degree, and the number of collisions.

4. The method for optimizing the motion trajectory of a robot according to claim 1, characterized in that, The range of anomalies was determined based on the following method: When the reward value of the action state is lower than the instantaneous reward threshold, the action state is identified as an abnormal action state. When the cumulative reward value of at least one consecutive action state is lower than the cumulative reward threshold, the at least one consecutive action state is identified as an abnormal segment. When the rate of change of the reward value of an action state exceeds the range of the reward threshold, the action state is identified as the abnormal action state. Alternatively, when the rate of change of at least one reward value of the at least one consecutive action state exceeds the range of the reward threshold change, the at least one consecutive action state is identified as the abnormal segment. If a collision event is identified in the action state based on the feature vector of the action state, the action state is identified as the abnormal action state. The abnormal range is determined based on the abnormal action state and / or the abnormal segment.

5. The method for optimizing the motion trajectory of a robot according to claim 1, characterized in that, The behavioral intent inference module is trained based on the following method: The expert demonstration trajectory of the robot is used as the first sample motion trajectory; The sample task and reward function are input into the policy network to obtain the second sample motion trajectory output by the policy network; the policy network is used to generate at least one trajectory data based on the sample task, and calculate the cumulative trajectory reward value of each trajectory data according to the reward function, and take the trajectory data with the largest cumulative trajectory reward value as the second sample motion trajectory; Based on the first sample motion trajectory and the second sample motion trajectory, the reward function of the reward function network is optimized and trained to obtain the optimized reward function; during the optimization training process, the reward function network generates a first sample cumulative reward value for the first sample motion trajectory and a second sample cumulative reward value for the second sample motion trajectory; based on the comparison result of the first sample cumulative reward value and the second sample cumulative reward value, the first sample motion trajectory is identified as a standard behavior trajectory, and the second sample motion trajectory is identified as a non-standard behavior trajectory. Based on the optimized reward function, an optimized reward function network is obtained to obtain the behavioral intent inference module.

6. The method for optimizing the motion trajectory of a robot according to claim 5, characterized in that, The reward function network is further used to generate a first feature expectation of the first sample motion trajectory and a second feature expectation of the second sample motion trajectory. After obtaining the optimized reward function, the network further includes: The sample task and the optimized reward function are input into the policy network to obtain the optimized second sample motion trajectory output by the policy network. Based on the first sample motion trajectory and the optimized second sample motion trajectory, the optimized reward function is further optimized and trained. The policy network and the reward function are subjected to alternating iterative optimization training until the difference between the first feature expectation and the second feature expectation is less than a set difference. Then the alternating iterative optimization training ends and the final optimized reward function is obtained. Based on the final optimized reward function, obtain the final optimized reward function network to obtain the behavior intention inference module.

7. A device for optimizing the motion trajectory of a robot, characterized in that, include: The trajectory optimization module is used to input the robot's motion trajectory into the optimization model and obtain the optimized motion trajectory output by the optimization model. The optimization model includes a behavior intent inference module, an abnormal behavior detection module, and an abnormal behavior repair module. The behavior intent inference module is used to extract features from each action state of the motion trajectory and calculate the reward value of each action state based on the extracted feature vectors. The abnormal behavior detection module is used to identify the abnormal range based on the reward value and feature vectors. The abnormal behavior repair module is used to correct the abnormal action states within the abnormal range based on the objective function to obtain the optimized motion trajectory. The objective function is determined based on the target cumulative reward value of the abnormal action state within the abnormal range and at least one correction term.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method for optimizing the motion trajectory of the robot as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for optimizing the motion trajectory of the robot as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for optimizing the motion trajectory of the robot as described in any one of claims 1 to 6.

Images