DDPG multi-target trajectory planning method for robot fusion error feedforward

Abstract

The application discloses a kind of DDPG multi-objective trajectory planning methods for robot fusion error feedforward, specific steps are as follows: first, the kinematics of robot and comprehensive error mapping model are established, then neural network is used to predict end trajectory error, and according to this, fusion error feedforward is constructed DDPG trajectory planning agent, while designing multidimensional composite reward function, to guide the optimization strategy of agent, realize multi-objective balance.The method effectively suppresses system disturbance through feedforward compensation mechanism, significantly improves trajectory tracking accuracy and dynamic response speed, enhances the stability and adaptability of robot in unstructured scene, provides a new technical path for high-precision robot control, and has important application value.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of intelligent motion control, intelligent manufacturing and reinforcement learning for industrial robots, and specifically relates to a DDPG multi-objective trajectory planning method for robot fusion error feedforward. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing technologies, industrial robots are increasingly being used in high-precision operations such as processing and assembly. Robot trajectory planning, as a core element determining the quality and efficiency of operations, has always been an important research direction in the field of robotics.

[0003] Existing robot trajectory planning methods can be mainly divided into the following three categories:

[0004] I. Traditional Interpolation Planning Methods. Traditional trajectory planning methods are typically based on the robot's ideal kinematic model, generating motion trajectories in joint space or Cartesian space through polynomial interpolation, spline curves, or numerical optimization. While these methods are computationally efficient and simple to implement, their trajectory planning relies entirely on the ideal kinematic model, failing to consider the impact of multi-source coupled errors such as geometric parameter errors, joint transmission errors, flexible deformation, and thermal deformation on the end-effector pose accuracy. This results in significant end-effector tracking deviations during actual execution, making it difficult to meet the requirements of high-precision operations.

[0005] II. Trajectory Planning Methods Based on Deep Reinforcement Learning. In recent years, Deep Reinforcement Learning (DRL) methods have been gradually introduced into the field of robot trajectory planning. Among them, the Deep Deterministic Policy Gradient (DDPG) algorithm has shown good applicability in robot motion control due to its ability to handle continuous motion spaces. However, existing DDPG-based trajectory planning methods typically construct the state space only using ideal state variables such as theoretical joint angles and target deviations, without considering the end-effector pose shift caused by multi-source errors under actual working conditions. This leads to the planned strategy deviating from the expected effect during actual execution, resulting in decreased tracking accuracy.

[0006] III. Independent Error Compensation Methods Based on Neural Networks. Some studies employ methods such as BP neural networks to model and compensate for robot end-effector positioning errors. They establish a mapping relationship between joint states and end-effector errors through offline training, and then perform post-correction on the theoretical command pose. While these methods can reduce static errors to some extent, error prediction and trajectory planning are disconnected, compensation information is not integrated into the planning and decision-making process, and the compensation effect is greatly affected by changes in operating conditions, making real-time optimization in dynamic scenarios difficult.

[0007] All three of the above schemes separate error prediction from trajectory decision-making, making it difficult to simultaneously balance multiple objectives such as accuracy, timeliness, and energy consumption, and resulting in poor generalization ability under complex operations. Summary of the Invention

[0008] This invention aims to construct a BP error feedforward fusion DDPG collaborative planning framework, predict the end-effector pose error in advance and embed it into the reinforcement learning state space, design multi-dimensional composite rewards, achieve multi-objective balanced optimization, significantly reduce the residual error of robot dynamic tracking, and improve motion stability and operation accuracy under unstructured chemical conditions.

[0009] To achieve the objectives of this invention, the technical solution adopted is as follows:

[0010] A multi-objective trajectory planning method for robot fusion error feedforward (DDPG) includes the following steps:

[0011] S1. Establish the kinematic model and comprehensive error mapping model of the robot; the kinematic model is established based on the improved DH parameter method, including the forward kinematic model and the velocity kinematic model; the comprehensive error mapping model is a unified mapping relationship between multiple source errors and end-effector pose errors established based on geometric parameter errors, joint transmission errors, flexible deformation errors, thermal deformation errors and control and measurement errors.

[0012] S2. Construct a BP neural network error prediction model, taking the robot's joint motion state and working condition characteristics as input, and outputting the end-effector position error and posture error; construct a sample set and train it to obtain the predicted value of the end-effector pose error;

[0013] S3. Construct a DDPG trajectory planning agent based on DDPG, and introduce the predicted end pose error value obtained in step S2 into the state space to perform trajectory planning.

[0014] S4. Design a multi-dimensional composite reward function to guide the DDPG trajectory planning agent to achieve multi-objective balance in trajectory planning;

[0015] S5. Continuously train the DDPG trajectory planning agent in a simulation environment that integrates the BP neural network error prediction model, output continuous joint actions, and generate an optimized trajectory that meets the requirements of multiple objectives.

[0016] Furthermore, in step S1, the kinematic model is established based on the improved DH parameter method, as follows:

[0017] Based on the improved DH parameter definition, from the coordinate system To coordinate system The homogeneous transformation is decomposed into a revolving shape. Rotation along the axis Axis translation, rotation Axis rotation and along The four basic transformations of axis translation are:

[0018]

[0019] Its matrix form is:

[0020]

[0021] in, Indicates the first The coordinate system of the first link relative to the first link coordinate system Pose transformation matrix of each link coordinate system Indicates the length of the link. Indicates the link twist angle. Indicates joint offset. This indicates that the joint angle is zero-position offset;

[0022] The forward kinematics model is the homogeneous transformation matrix of the end coordinate system relative to the base coordinate system, as follows:

[0023]

[0024] The velocity-kinematic model is obtained by differentiating the forward kinematic equations:

[0025]

[0026] in, Let be the velocity vector of the end effector in Cartesian space. For joint velocity vectors, Let Jacobian matrix be used for the robot.

[0027] Furthermore, the construction of the comprehensive error mapping model in step S1 is as follows:

[0028] ;

[0029] in, For the end-effector pose error, For geometric parameter errors, This is due to joint transmission error. For flexible deformation error, For thermally induced deformation error, For measurement error;

[0030] This can be further expressed as a linear mapping of the generalized error input vector:

[0031]

[0032] in, This is the comprehensive error mapping matrix. This is the generalized error input vector.

[0033] Furthermore, the input for constructing the BP neural network error prediction model in step S2 is:

[0034] ;

[0035] in, For joint variable vectors, and These are joint velocity and joint acceleration, used to characterize the robot's motion state; This refers to the end load or equivalent processing force. Characteristics of joint or ambient temperature;

[0036] The output is the predicted end-effector pose error:

[0037] ;

[0038] in,

[0039] Therefore, a data-driven error prediction model is established:

[0040] ;

[0041] in, This represents the nonlinear mapping function established by the BP neural network. and These are the network weights and bias parameters, respectively.

[0042] Further, the sample set in step S2 is represented as follows:

[0043] ;

[0044] in, For the sample size, For the system input quantity corresponding to the k-th sample, For the first The actual terminal error corresponding to each sample;

[0045] ;

[0046] In the formula, The actual end-effector pose obtained by measurement, The theoretical end pose is calculated from the kinematic model.

[0047] Furthermore, the error feedforward reconstruction state in the state space during step S3 is represented as follows:

[0048] ;

[0049] in, The current joint angle, The current joint velocity, For the target joint angle, To account for the terminal residual bias after considering the prediction error of the BP neural network, This is the current predicted value of the terminal error.

[0050] Furthermore, the multidimensional composite reward function in step S4 is expressed as follows:

[0051] ;

[0052] in, As a reward for end-point tracking accuracy, Rewards for time efficiency As an energy consumption reward, As a reward for trajectory smoothness, To restrain punishment, Rewards for reaching the target; , , , , , These are the weighting coefficients for each item.

[0053] Furthermore, the end-point tracking accuracy bonus It is expressed as follows:

[0054] ;

[0055] in,

[0056] ;

[0057] In the formula, For the target theoretical end pose, This is the current theoretical end pose. This is the current predicted value of the terminal error;

[0058] The time efficiency reward It is expressed as follows:

[0059] ;

[0060] in, To achieve the target's reward weight, This is the penalty coefficient for single-step time.

[0061] Energy consumption reward It is expressed as follows:

[0062] ;

[0063] in, For the current continuous action;

[0064] The trajectory smoothness reward It is expressed as follows:

[0065] ;

[0066] in, This refers to the continuous action from the previous moment;

[0067] The constraint penalty It is expressed as follows:

[0068] ;

[0069] in, , and These represent the allowable ranges for joint position, velocity, and acceleration, respectively. , , and This corresponds to the penalty coefficient;

[0070] The target reached reward It is expressed as follows:

[0071] ;

[0072] in, Rewards for reaching the goal The target error threshold.

[0073] Furthermore, in step S5, the DDPG trajectory planning agent adopts an Actor-Critic dual-network structure to reconstruct the state with error feedforward. As input to the Actor network, in continuous action As output, the Critic network is used to evaluate state-action pairs. The value of Q is determined by the fact that its input consists of a state vector and an action vector, and its output is a scalar Q-value.

[0074] ;

[0075] Furthermore, the DDPG trajectory planning agent uses soft updates to update the Actor network parameters and Critic network parameters:

[0076] ;

[0077] ;

[0078] In the formula, This is the soft update coefficient. , These are the parameters for the Actor network and the Critic network, respectively. , These are the target Actor network parameters and the target Critic network parameters, respectively.

[0079] Furthermore, in step S5, the DDPG trajectory planning agent is also trained by incorporating BP neural network error feedforward, specifically as follows:

[0080] a. First, initialize the Actor network, Critic network, target network, and experience replay pool, and load the already trained BP neural network error prediction model;

[0081] b. Randomly generate or set the starting point and target point according to the task requirements to construct the initial state;

[0082] c. The Actor network outputs continuous actions based on the current state and adds exploration noise to enhance the policy search capability. After the environment receives the actions, it updates the joint states and calculates the theoretical end pose. At the same time, it calls the BP network to predict the current end error and then reconstructs the state at the next moment.

[0083] d. Calculate the current reward based on the multidimensional composite reward function, store the interaction samples in the experience replay pool, and then randomly sample a small batch of data from it to update the parameters of the Critic network and Actor network, while performing a soft update on the target network.

[0084] e. Determine if the termination condition is met:

[0085] If the condition is not met, proceed to the next round of interaction;

[0086] If the conditions are met, the trained DDPG agent is output.

[0087] The technical solution of the present invention has the following beneficial technical effects:

[0088] 1. This invention integrates BP network prediction error feedforward into DDPG planning to construct an integrated collaborative optimization framework; compared with traditional planning and ordinary DDPG, the average tracking error at the end point is reduced by 62.23%, the position error compensation rate reaches 57.42%, and the attitude compensation rate reaches 55.77%;

[0089] 2. The multi-dimensional composite reward designed in this invention achieves a balance between multiple objectives such as timeliness, energy consumption, smoothness, and accuracy, significantly reduces the overall trajectory cost, and is more adaptable to variable load and multi-error coupled working conditions, effectively improving the stability of high-precision operation of six-axis robots. Attached Figure Description

[0090] Figure 1 This is a diagram illustrating the method steps of the present invention;

[0091] Figure 2 This is a schematic diagram of the structure of a six-degree-of-freedom serial industrial robot in an embodiment of the present invention;

[0092] Figure 3 This is a flowchart of the BP neural network training process in an embodiment of the present invention;

[0093] Figure 4 This is a flowchart of the DDPG agent training process with fused error feedforward in an embodiment of the present invention;

[0094] Figure 5 This is a simulation platform architecture diagram constructed using a six-degree-of-freedom serial industrial robot as an example in this invention.

[0095] Figure 6 This is a graph of the BP neural network loss function in an embodiment of the present invention;

[0096] Figure 7 This is a scatter plot of the BP neural network error prediction results in an embodiment of the present invention.

[0097] Figure 8 This is a comparison diagram of robot position error before and after error compensation in an embodiment of the present invention;

[0098] Figure 9 This is a comparison diagram of robot posture error before and after error compensation in an embodiment of the present invention;

[0099] Figure 10 This is a schematic diagram comparing the key points of each method in the embodiments of the present invention;

[0100] Figure 11 This is a comparison chart of multi-objective comprehensive cost indicators in embodiments of the present invention. Detailed Implementation

[0101] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0102] Example 1:

[0103] This embodiment uses a typical six-degree-of-freedom serial industrial robot as an example. Its structure consists of a base, connecting rods, rotary joints, and an end effector. Figure 2 As shown.

[0104] This embodiment provides a DDPG multi-objective trajectory planning method for robot fusion error feedforward, such as... Figure 1 As shown, the specific steps include the following:

[0105] Step 1: Establish the robot's kinematics and integrated error mapping model. First, establish the robot's base coordinate system, joint coordinate system, and tool coordinate system, and derive the forward and inverse kinematics models based on the DH parameter method.

[0106] This embodiment uses the improved DH parameter method to establish the kinematic model. This method sets up a local coordinate system between adjacent links, transforming the spatial pose relationship into a standard rotation and translation transformation. The parameter meanings are clearer, and it is also convenient to introduce error terms later.

[0107] For the Establish a link coordinate system for each joint. The pose relationship between adjacent coordinate systems is described by four DH parameters: link length, link twist angle, joint offset, and joint angle. This embodiment focuses on a six-DOF (degrees of freedom) rotary joint robot, where each joint variable can be represented as:

[0108] ;

[0109] in, For the first The joint angle. After considering the zero-position offset, the first joint angle... The actual joint angle of a joint can be written as:

[0110] ;

[0111] In the formula, The joint angles given to the robot control system, For the first Initial angle offset of each joint.

[0112] Based on the improved DH parameter definition, from the coordinate system To coordinate system The homogeneous transformation can be decomposed into a cyclic transformation. Rotation along the axis Axis translation, rotation Axis rotation and along The four basic transformations of axis translation are:

[0113] ;

[0114] Its matrix form is:

[0115] ;

[0116] In the formula, Indicates the first The coordinate system of the first link relative to the first link coordinate system The pose transformation matrix of each link coordinate system. This matrix contains both attitude rotation information and position translation information, and can be used to describe the spatial motion relationship between adjacent joints of the robot.

[0117] The improved DH parameters of the six-DOF industrial robot are shown in Table 1.

[0118] Table 1. Improved DH Parameter Table for Six-DOF Industrial Robots

[0119]

[0120] in, , , Determined by the robot's body structure, These are the input variables in the motion control process. During actual modeling, the parameters in the table can be assigned values ​​based on the structural dimensions of the specific robot model.

[0121] Forward kinematics of a robot is used to solve for the pose of the end effector relative to the base coordinate system using joint variables. Based on the coordinate transformation relationship of the cascaded mechanism, the homogeneous transformation matrix of the end effector coordinate system {6} relative to the base coordinate system {0} is:

[0122] ;

[0123] Expanding it, it can be represented as:

[0124] ;

[0125] In the formula, Let be the attitude matrix of the end effector relative to the base coordinate system. This is the position vector of the end effector. (The matrix contains...) , , These represent the directional components of the three coordinate axes of the terminal coordinate system in the base coordinate system.

[0126] Therefore, the robot's end-effector pose is a nonlinear function of the joint angles and structural parameters, which can be expressed as:

[0127] ;

[0128] in, Represents the end-effector pose vector. , , These are attitude angle parameters.

[0129] Trajectory planning also requires describing the mapping between end-effector velocity and joint velocity. Differentiating the forward kinematic equations yields the velocity-kinematic model:

[0130] ;

[0131] In the formula, Let be the velocity vector of the end effector in Cartesian space. For joint velocity vectors, Let be the Jacobian matrix of the robot. For a six-DOF tandem robot, the Jacobian matrix can be expressed as:

[0132] ;

[0133] in, The Jacobian matrix represents the linear velocity. This represents the Jacobian matrix of angular velocity. The Jacobian matrix describes the mapping from joint velocity to end-effector velocity and can be used to analyze singular configurations, velocity constraints, error amplification, and trajectory smoothness.

[0134] Step 2: Taking into account factors such as geometric parameters, joint transmission, flexible deformation, and thermal deformation, establish a comprehensive error mapping model for the robot end effector to quantify the impact of each error source on the end effector pose accuracy.

[0135] The end-effector pose error of industrial robots is caused by a combination of geometric parameter deviations, joint transmission errors, structural flexibility deformation, thermal deformation, and control and measurement errors. The open-chain structure of a six-DOF serial robot allows deviations from the preceding stage to be transmitted along the kinematic chain and amplified at the end, resulting in pose errors that are nonlinear, coupled, and dependent on operating conditions.

[0136] Geometric parameter errors are a significant source of pose errors in industrial robot end effectors, primarily caused by manufacturing and assembly errors, joint axis misalignment, and structural wear after long-term operation. Under the improved DH modeling framework, robot geometric parameters include link lengths. Linkage torsion angle Joint offset and joint angle zero offset There is usually a small deviation between the actual robot parameters and the theoretical parameters, which can be expressed as:

[0137]

[0138] In the formula, , , and These represent the link length error, link torsion angle error, joint offset error, and joint angle error, respectively. Since the links of a serial robot are connected sequentially, any deviation in the parameters of any coordinate transformation matrix will change the position and orientation of the subsequent coordinate system, ultimately causing the end effector's pose to deviate from the theoretical value.

[0139] Let the robot's geometric parameter error vector be:

[0140] ;

[0141] The end-effector pose deviation caused by geometric parameter errors can be approximated as:

[0142] ;

[0143] In the formula, This is the geometric error sensitivity matrix, used to describe the degree of influence of deviations in various geometric parameters on the end-effector pose error.

[0144] Joint transmission errors mainly originate from factors such as reducer backlash, gear meshing errors, encoder resolution limitations, servo system tracking errors, and friction dead zones. In actual operation, the joint angle given by the robot control system... The actual output angle of the joint is not entirely consistent with the actual output angle of the joint. The actual output of each joint can be expressed as:

[0145] ;

[0146] in, This represents the joint transmission error. For a six-DOF robot, the joint error vector is:

[0147] ;

[0148] The end-effector pose error caused by joint transmission error can be represented by the Jacobian matrix as follows:

[0149] ;

[0150] In the formula, Let be the Jacobian matrix of the robot in the current configuration. As shown in Equation (2-15), the end-effector error corresponding to the same joint error is different in different configurations. When the robot approaches a singular or extended configuration, the condition number of the Jacobian matrix increases, and small joint errors are significantly amplified, thereby reducing the end-effector positioning accuracy.

[0151] For serial robots, joint flexibility is one of the main sources of flexible deformation error. Let the external load on the robot's end effector be... The joint torque can then be expressed by the Jacobian matrix transpose relation as follows:

[0152] ;

[0153] If the joint stiffness matrix is:

[0154] ;

[0155] The elastic deformation of the joint caused by external load can be expressed as:

[0156] ;

[0157] Further, the end-effector pose error caused by joint flexibility can be obtained:

[0158] .

[0159] Thermal deformation can cause changes in link length, joint offset, and zero-position angle drift. Its effects are related to structural parameters, running time, motion frequency, load, and ambient temperature, and need to be taken into account in high-precision continuous operation.

[0160] Control and measurement errors also affect the accuracy of the end effector. Control errors include trajectory interpolation errors, servo lag, and tracking deviations caused by velocity and acceleration constraints; measurement errors mainly come from sensor resolution, calibration equipment accuracy, sampling noise, and coordinate system registration errors. If the modeling samples are noisy, the training quality of the error model will also decrease.

[0161] Let the desired joint trajectory be The actual joint trajectory is Then the control tracking error is:

[0162] ;

[0163] The end-effector pose error caused by control tracking error can be expressed as:

[0164] ;

[0165] Therefore, during the error modeling and sample acquisition stages, it is necessary to pay attention to the accuracy of the measurement equipment, sampling noise, and coordinate system registration error simultaneously to avoid noise directly entering the training labels.

[0166] After the above error sources are superimposed, the end effector error of the industrial robot can be expressed as:

[0167] ;

[0168] In the formula, For geometric parameter errors, This is due to joint transmission error. For flexible deformation error, For thermally induced deformation error, To control tracking error, This refers to measurement errors. Because various errors are coupled together, and some errors exhibit nonlinearity, time-varying characteristics, and uncertainty, high-precision compensation is difficult to achieve solely through analytical models.

[0169] As the foregoing analysis shows, the end-effector pose error is influenced by the current configuration, structural parameters, external loads, operating status, and environmental conditions. To describe the propagation law of multi-source errors into Cartesian space and to provide input features and sample labels for the subsequent BP neural network, a unified comprehensive error mapping model needs to be constructed.

[0170] The integrated error mapping model is based on forward kinematics and incorporates joint variable errors, structural parameter errors, and working condition disturbances such as load, temperature, and speed into the end-effector pose error analysis.

[0171] When the robot system has multiple sources of error, the actual end-effector pose can be expressed as:

[0172] ;

[0173] in, For joint variable vectors, It is a parameter vector composed of DH parameters, zero-position offset, and structural dimensions. Indicates joint variable error. Indicates structural parameter error. This represents operational state variables such as load, temperature, and speed. From this, the end-effector pose error can be obtained:

[0174] ;

[0175] Applying a first-order approximation to the vicinity of the theoretical pose yields:

[0176] ;

[0177] In the formula, Represents the joint variable error mapping matrix. Represents the structural parameter error mapping matrix. This represents the matrix showing the influence of the running state variables on the terminal error. It can be written in matrix form as follows:

[0178] ;

[0179] in, It can be represented by the robot Jacobian matrix, which mainly reflects the influence of joint angle error and control tracking error on the end-effector pose. This is the parameter error sensitivity matrix, reflecting the influence of DH parameter deviation, zero-position offset, and structural dimension deviation on the end pose; This is the state error mapping matrix, used to describe the influence of non-geometric factors such as load, temperature, and speed on the end-point error.

[0180] Under the improved DH modeling method, the homogeneous transformation matrix between the coordinate systems of adjacent links of the robot is derived from... , , and It is determined by four parameters. Let the first one be... The parameter error vector of each link is:

[0181] ;

[0182] All geometric error parameters of a six-degree-of-freedom robot can be combined as follows:

[0183] ;

[0184] The end-effector pose error caused by geometric parameter errors can be expressed as:

[0185] ;

[0186] In the formula, The geometric error sensitivity matrix can be obtained by taking the partial derivatives of the end-effector pose function with respect to each geometric parameter.

[0187] .

[0188] Non-geometric errors include joint transmission errors, flexible deformation errors, thermally induced deformation errors, and control tracking errors. Compared to geometric errors, non-geometric errors are more susceptible to operating conditions and exhibit time-varying and uncertainties. Both joint transmission errors and control tracking errors can be equivalently represented as joint variable deviations. Let the equivalent joint error be:

[0189] ;

[0190] In the formula, Mechanical transmission errors caused by factors such as reducer backlash, encoder resolution, and transmission lag. This refers to the control error caused by servo control lag and trajectory tracking deviation. The corresponding end-effector error is:

[0191] ;

[0192] Secondly, the flexible deformation under external loads can be described using a joint stiffness model. Based on static equilibrium, the joint torque is... If the joint stiffness matrix is The end-effector error caused by flexible deformation is:

[0193] ;

[0194] In the formula, This represents the end-effector load vector. For machining robots, It can be composed of cutting force, contact force, or load gravity, and its magnitude and direction vary with machining posture, feed rate, and material removal state. Therefore, flexibility error is not only related to external load, but also closely related to robot configuration.

[0195] Thermally induced deformation can be equivalently represented as the drift of structural parameters due to temperature changes. Let the parameter deviation caused by temperature changes be... Then the thermal error can be expressed as:

[0196] ;

[0197] in, This represents the thermal error sensitivity matrix. When thermal deformation is primarily manifested as changes in connecting rod length and joint offset... It can be regarded as a submatrix of the geometric error sensitivity matrix in the direction of thermal parameters.

[0198] Combining the mapping relationship between geometric and non-geometric errors, the overall error of the robot's end effector can be expressed as:

[0199] ;

[0200] Substituting the errors of each item, we get:

[0201] ;

[0202] In the formula, This is to measure random error terms such as noise and coordinate system registration error. For easier unified modeling later, various error variables can be combined into a generalized error input vector:

[0203] ;

[0204] Accordingly, the comprehensive error mapping matrix can be written as:

[0205] ;

[0206] The comprehensive error mapping model can then be simplified to:

[0207] ;

[0208] The end effector's overall error is determined by the robot's configuration, operating state, and multiple error variables, revealing the influence paths of different error sources. However, in real-world systems, errors such as joint clearance, friction dead zones, local thermal drift, and dynamic flexibility changes are difficult to measure directly, and these error sources are coupled with each other, limiting the accuracy of compensation based solely on analytical models.

[0209] Based on the comprehensive error mapping model, this embodiment introduces a BP neural network, using joint state and operating condition characteristics as input, to learn the nonlinear mapping relationship of end-effector error:

[0210] ;

[0211] In the formula, and These are joint velocity and joint acceleration, used to characterize the robot's motion state; This refers to the end load or equivalent processing force. This refers to joint or ambient temperature characteristics. When all operating conditions cannot be directly measured under experimental conditions, observable variables such as joint angle, theoretical end-effector pose, load mass, and running time can be selected as network inputs based on actual acquisition conditions.

[0212] The output of the BP neural network is the predicted end-effector pose error:

[0213] ;

[0214] This allows us to establish a data-driven error prediction model:

[0215] ;

[0216] In the formula, This represents the nonlinear mapping function established by the BP neural network. and These are the network weights and bias parameters, respectively.

[0217] Training samples can be constructed by collecting the theoretical and actual poses of the robot under different joint configurations and working conditions:

[0218] ;

[0219] in, For the sample size, For the first The actual end-effector error corresponds to each sample. The actual end-effector error can be obtained by subtracting the measured pose from the theoretical pose:

[0220] ;

[0221] In the formula, The actual end-effector pose obtained by measurement, The theoretical end pose is calculated from the kinematic model.

[0222] Step 3: Based on the characteristics of the robot's end effector error coupling and working condition correlation, a BP neural network error prediction model is designed. The joint motion state and working condition characteristics are used as inputs to predict the end effector position and attitude errors, and the prediction results are used for feedforward compensation.

[0223] Backpropagation (BP) neural networks are multi-layer feedforward networks trained based on error backpropagation. They have the ability to approximate nonlinear functions and are suitable for predicting robot end-effector errors.

[0224] A typical backpropagation (BP) neural network consists of an input layer, hidden layers, and an output layer. The input layer receives state variables such as joint angles, velocity, acceleration, external loads, and temperature. The hidden layers perform nonlinear feature transformations, and the output layer provides predictions for end-effector position and attitude errors.

[0225] In this embodiment, the network inputs state features such as joint angles, velocity, acceleration, external load, and temperature, and the output is the predicted value of the end-effector position and attitude error. The goal of training the BP network is to adjust the weights and biases so that the prediction error gradually approaches the measured error label.

[0226] The forward propagation process of a BP neural network refers to the process by which the input sample is passed layer by layer through weighted summation, bias stacking, and activation function transformation to obtain the final output. Let the number of neurons in the hidden layer be... Then the net input of the hidden layer is:

[0227] ;

[0228] The hidden layer output is:

[0229] ;

[0230] In the formula, This refers to the activation function of the hidden layer. The role of the activation function is to introduce nonlinear mapping capabilities, enabling the network to fit complex nonlinear relationships. Commonly used activation functions include the Sigmoid function, Tanh function, and ReLU function.

[0231] The relationship between input variables and output error in robot error prediction is typically non-linear. A backpropagation (BP) network utilizes non-linear activation functions in the hidden layers to extract error mapping features. The net input to the output layer is:

[0232] ;

[0233] If the error prediction task is a continuous value regression problem, the output layer typically uses a linear activation function, and the network prediction output is:

[0234] ;

[0235] Therefore, the input-output mapping relationship established by a single hidden layer BP neural network can be written as:

[0236] .

[0237] The training objective of a BP neural network is to make the predicted output... As close as possible to the actual output .

[0238] The actual output of the robot's end-effector error prediction task is the difference between the measured pose and the theoretical pose, i.e., the error label. Let the training set be:

[0239] ;

[0240] in, For the sample size, For the first One input sample, This represents the corresponding true error vector. The commonly used mean squared error loss function can be expressed as:

[0241] ;

[0242] In the formula, For the network to the first The predicted output for each sample.

[0243] The backpropagation algorithm calculates the gradient of the loss function with respect to the weights and biases of each layer using chain rule differentiation, and updates the network parameters along the negative gradient direction. The weight matrix... For example, its update formula is:

[0244] ;

[0245] The formula for updating the bias vector is:

[0246] ;

[0247] In the formula, For learning rate, This represents the network layer number. The learning rate determines the parameter update step size. An excessively large learning rate can cause network training oscillations or even divergence; an excessively small learning rate results in slow convergence and increased training time. Therefore, in actual training, the learning rate needs to be set appropriately based on the error convergence curve and validation set performance. If the output layer uses a linear activation function and mean squared error loss, its error signal can be expressed as:

[0248] ;

[0249] The hidden layer error signal can be obtained by backpropagation of the output layer error signal:

[0250] ;

[0251] In the formula, This indicates element-wise multiplication. This is the derivative of the activation function.

[0252] Error prediction at the end effector of industrial robots is essentially a nonlinear regression problem with multiple inputs and outputs. Factors such as joint angles, velocity, acceleration, external loads, and temperature collectively alter the error distribution, and these factors are not simply linearly superimposed. Backpropagation (BP) neural networks can learn the implicit relationships between inputs and outputs through sample training, making them suitable for describing error mapping under complex working conditions. It is important to note that the network's prediction performance depends not only on the model structure but also on the appropriateness of the sample coverage and input features.

[0253] Input feature design needs to balance error representation capability and practical data acquisition. Based on a comprehensive error mapping model, this embodiment selects joint configuration, motion state, and operating condition variables as the main features.

[0254] ;

[0255] In the formula, Joint angle vector, For joint velocity vectors, The joint acceleration vector, This refers to the end-effector load or equivalent processing force. These are temperature-related characteristics.

[0256] Joint angles determine the robot's spatial configuration and are fundamental variables affecting geometric error propagation and Jacobian matrix changes. Under different configurations, the same error source will produce different end-effector errors, thus requiring joint angles to be used as basic input features for error prediction models.

[0257] Joint velocity and acceleration reflect the dynamic state of a robot during motion. When performing high-speed or frequently start-stop movements, servo hysteresis, inertial loads, and structural flexibility become more pronounced, and the end effector error also changes accordingly. Furthermore, introducing velocity and acceleration features helps the network learn the relationship between dynamic errors and motion states.

[0258] For tasks involving contact forces or machining forces, the elastic deformation of joints and links caused by end-effector loads is a significant source of flexibility errors. If the platform can acquire force sensor data or estimate machining loads, the external load vector can be used as an input feature.

[0259] Considering the model's expressive power and data acquisition conditions, this embodiment designs the BP neural network input vector as follows:

[0260] ;

[0261] In the formula, The input vector is a feature vector representing the operating conditions, which can be composed of variables such as end-effector load, temperature, runtime, task type, or theoretical end-effector pose. If only static error prediction is performed, the input vector can be simplified to:

[0262] ;

[0263] In the formula, This is the theoretical end-effector pose calculated from the forward kinematics model. This input format can both preserve joint space configuration information and introduce Cartesian space pose features, making it suitable for static error prediction under workspace sampling conditions.

[0264] The output of the BP neural network must be consistent with the end-effector error compensation target. This embodiment aims to predict the deviation of the robot's end effector from its theoretical pose; therefore, the network output is set as the end-effector pose error vector.

[0265] ;

[0266] In the formula, , , For end position error, , , This represents the end-effector attitude error. For some tasks that only focus on end-effector positioning accuracy, the output can be simplified to a three-dimensional position error:

[0267] ;

[0268] During the sample collection process, let the first... The theoretical end pose of each sampling point is: The actual measured pose is Then its error label can be obtained by subtracting the two:

[0269] ;

[0270] Among them, theoretical pose The actual pose is calculated from the robot's forward kinematics model. This can be obtained using a laser tracker, vision measurement system, or other high-precision measurement equipment. Input-output sample pairs can then be constructed, when the number of samples is... At that time, the training sample set can be represented as:

[0271] ;

[0272] Through the above mapping design, the BP neural network learns the nonlinear function between the robot's state features and the end-effector error:

[0273] ;

[0274] In the formula, This represents the network prediction error. The prediction result can be used for end-effector pose compensation, and can also be used as part of the state space for subsequent DDPG trajectory planning, enabling the planning agent to have error awareness.

[0275] Network topology significantly affects the performance of BP neural networks. To address the complex nonlinear mapping between continuous input and output variables in robot error prediction, this embodiment employs a multi-layer feedforward BP neural network consisting of an input layer, hidden layers, and an output layer as the prediction model.

[0276] The number of neurons in the input layer is determined by the dimension of the input features. The dimension of the input layer is:

[0277] ;

[0278] Hidden layers are used to extract nonlinear relationships from input features. The number of layers and neurons needs to be determined comprehensively based on the sample size, error complexity, and training effect. Too few neurons lead to insufficient expressive power and underfitting; too many neurons increase the number of parameters and overfitting. This embodiment uses an empirical formula to determine the initial range of their number:

[0279] ;

[0280] In the formula, This represents the number of neurons in the hidden layer. This is an empirical adjustment constant, typically ranging from 1 to 10. In actual training, multiple sets of comparative experiments can be conducted around this range to determine the final network size based on the validation set error and convergence stability.

[0281] ;

[0282] In the formula, and These are the weight matrices from the input layer to the hidden layer and from the hidden layer to the output layer, respectively. and These are the bias vectors for the hidden layer and the output layer, respectively. is the activation function for the hidden layer.

[0283] Since the normalized error data is often distributed within a finite interval, and the Tanh function has a zero-center property, which is beneficial to improving training stability, this embodiment preferentially selects the Tanh function as the hidden layer activation function:

[0284] .

[0285] Because the dimensions of features such as joint angle, end position, load, and temperature are different, the input and output data need to be normalized before training to reduce the impact of scale differences on convergence speed and prediction accuracy.

[0286] This embodiment uses the minimum-max normalization method to map the sample data to... or Interval. For any input feature Its normalized expression is:

[0287] ;

[0288] In the formula, and The first The minimum and maximum values ​​of each input feature in the training set.

[0289] The network output also needs to be normalized. Let the first... The output error components are Then its normalization result is:

[0290] ;

[0291] After training, the network outputs normalized predicted values, which need to be denormalized to recover the actual physical quantities. The denormalization formula is:

[0292] ;

[0293] By normalizing and denormalizing, the impact of different units on model training can be reduced, thereby improving the network's convergence speed and prediction stability.

[0294] The end-effector error predicted by a BP neural network can be used for robot pose compensation. Let the theoretical end-effector pose of the robot be... The network prediction error is The compensated target pose can then be expressed as:

[0295] ;

[0296] In the formula, The desired pose after compensation. The meaning of this compensation concept is: when the network predicts that the robot will produce a deviation in a certain state... In this case, the command pose is reversed in advance to make the final pose after actual execution as close as possible to the desired pose.

[0297] In subsequent DDPG multi-objective trajectory planning, the error prediction results can not only be used for offline compensation, but also as part of the agent's state space.

[0298] Robot end-effector error is related to factors such as joint configuration, workspace position, movement speed, load, and ambient temperature. To ensure that the training samples closely approximate the actual error distribution, the sampling range should cover commonly used workspaces while retaining data under different joint combinations and typical operating conditions. The sample set can be constructed following the process of "joint space sampling - forward kinematics calculation - end-effector pose measurement - error label generation." Sampling should avoid mechanical limits, singular configurations, and areas where collisions may occur; otherwise, while increasing the amount of training data, it may reduce the model's effectiveness within the normal operating range.

[0299] ;

[0300] The joint space of a six-degree-of-freedom robot can then be represented as:

[0301] .

[0302] Actual sampling should avoid mechanical limits, obvious singular configurations, and collision-risk configurations. This embodiment uses stratified random sampling: first, the motion range of each joint is divided, and then sampling points are randomly generated within the range to improve workspace coverage.

[0303] Let the first The joint angle vectors of the sampling points are:

[0304] .

[0305] Based on the established forward kinematics model, its theoretical end pose can be calculated:

[0306] ;

[0307] The robot is then controlled to move to this joint configuration, and the actual end-effector pose is obtained through external high-precision measuring equipment.

[0308] ;

[0309] By subtracting the theoretical pose from the actual pose, we can obtain the first... Terminal error labels for each sample:

[0310] ;

[0311] In dynamic error prediction scenarios, variables such as joint velocity, joint acceleration, running time, load mass, or equivalent external load should also be collected simultaneously to form more complete input features. The input vector of each sample can be represented as:

[0312] ;

[0313] In the formula, For the first The operating condition feature vectors corresponding to each sampling point can be used to construct a complete sample set:

[0314] ;

[0315] In the formula, This represents the total number of samples. This sample set contains both robot state features and error labels generated from measured data, and can be used for supervised training of BP neural networks.

[0316] Raw data acquisition is often affected by measurement noise, random vibrations, communication delays, and outliers. If used directly for network training, it may lead to unstable model convergence or decreased generalization ability. Therefore, data preprocessing is required before training. The preprocessing of raw data acquisition should first include outlier removal, repeated measurement averaging, and normalization, and then divide the data into training, validation, and test sets.

[0317] The normalization parameters are obtained only from the training set statistics to avoid leaking test information.

[0318] ;

[0319] In the formula, The number of times the same sampling point is measured repeatedly.

[0320] Secondly, the input and output data need to be normalized. This embodiment uses the min-max normalization method to map the data to... Interval:

[0321] ;

[0322] in, and These are the minimum and maximum values ​​of the corresponding variables in the training set, respectively.

[0323] After preprocessing, the sample set is divided into training, validation, and test sets:

[0324] ;

[0325] in, Used for network parameter learning. Used to adjust network structure and training hyperparameters. Used to ultimately evaluate the model's predictive ability on unknown samples. Typically, it can be done according to... or Divide it according to the proportion.

[0326] The training objective is to minimize the deviation between the predicted error and the true error label. Let the network be trained on the nth... The output of each sample is The actual error is The loss function then takes the form of mean squared error:

[0327] ;

[0328] In the formula, This represents the number of training samples. During training, the gradients of the loss function with respect to the network weights and biases are calculated, and the parameters are updated using gradient descent or its improved algorithms.

[0329] ;

[0330] In the formula, Represents the set of network parameters. For learning rate, This represents the number of iterations.

[0331] The basic process of training a BP neural network is as follows: Figure 3 As shown.

[0332] The model training parameters can be set according to the sample size and experimental conditions. Table 2 shows the design basis for the BP neural network training parameters.

[0333] Table 2 BP Neural Network Training Parameter Settings

[0334]

[0335] To evaluate the performance of the BP neural network error prediction model, it is necessary to set indicators based on factors such as error magnitude, fit, and compensation effect. This embodiment uses root mean square error, mean absolute error, and maximum absolute error as the main evaluation indicators.

[0336] The root mean square error can be expressed as:

[0337] ;

[0338] The mean absolute error can be expressed as:

[0339] ;

[0340] The maximum absolute error can be expressed as:

[0341] ;

[0342] RMSE reflects the overall prediction error and is more sensitive to large errors; MAE represents the average prediction bias; MAXE is used to evaluate stability under extreme samples.

[0343] The compensation effect is evaluated by comparing the end-effector pose error norm before and after compensation.

[0344] ;

[0345] The residual error after compensation is:

[0346] ;

[0347] The error compensation rate can then be defined as:

[0348] ;

[0349] In the formula, To compensate for the previous average error, This represents the average residual error after compensation. A higher compensation rate indicates a more significant effect of the BP neural network in predicting and compensating for end-effector pose errors.

[0350] Step 4: Building upon Step 3, the BP prediction error is introduced into the DDPG deep reinforcement learning trajectory planning process to reconstruct the agent's state space. Based on the characteristics of the trajectory planning problem, a reinforcement learning MDP model is established, and the state space, action space, state transition, and termination conditions are designed to achieve collaborative optimization of "prediction-planning".

[0351] Traditional offline trajectory planning struggles to adapt to multi-source errors and dynamic disturbances. This embodiment constructs a continuous control agent based on DDPG and introduces BP prediction errors into the state space to achieve collaborative optimization of "prediction-planning".

[0352] The goal of trajectory planning is to generate continuous joint trajectories from the initial state to the target state, while satisfying kinematic constraints, joint limitations, and task requirements.

[0353] ;

[0354] The trajectory should satisfy the following boundary conditions:

[0355] ;

[0356] Meanwhile, to ensure the safety and stability of the robot's movement, the position, velocity, and acceleration of each joint must meet the following constraints:

[0357] ;

[0358] ;

[0359] ;

[0360] In the formula, and The first The upper and lower limits of the position of each joint. and For the upper and lower limits of speed, and These are the upper and lower limits of acceleration.

[0361] For high-precision tasks, trajectory planning must not only ensure the endpoint reaches the target point, but also consider metrics such as running time, energy consumption, trajectory smoothness, and endpoint tracking accuracy. The comprehensive optimization objective can be expressed as:

[0362] ;

[0363] In the formula, Indicates runtime metrics. Indicates energy consumption index, Indicators representing trajectory smoothness This indicates the end-point tracking accuracy index. , , and The weights are the coefficients for each objective. Since there may be conflicts between different objectives, such as shortening the running time potentially leading to increased energy consumption and decreased trajectory smoothness, this problem is essentially a multi-objective constrained optimization problem.

[0364] Reinforcement learning learns optimal policies through interaction between an agent and its environment. In robot trajectory planning, the agent selects actions based on its current state, and the environment updates its state accordingly and returns a reward signal. This process can be modeled as a Markov decision process, whose quintuple is defined as:

[0365] ;

[0366] In the formula, For state space, For the action space, Let be the state transition probability. For the reward function, is the discount factor. For deterministic robot simulation environments, given the current state and actions, the next state can usually be determined by a kinematic or dynamic model; therefore, the state transition can be expressed as:

[0367] ;

[0368] in, for Current state The goal of reinforcement learning is to find the policy function, which outputs actions to the agent. This maximizes the agent's cumulative discounted reward throughout the entire task:

[0369] ;

[0370] in, For the future Step reward. The optimal strategy can be expressed as:

[0371] .

[0372] In this embodiment, the state space describes the robot's motion state and task progress, the action space represents the trajectory control variables of the agent, and the reward function is used to evaluate the planning quality. The agent learns to select actions in different states through continuous interaction with the environment to generate trajectories that meet multiple objectives.

[0373] The state space should simultaneously describe the robot's motion state, remaining target distance, and error prediction information. The basic state includes the current joint angle, joint velocity, target joint angle, current end-effector pose, and target end-effector pose.

[0374] ;

[0375] In the formula, The current joint angle, The current joint velocity, For the target joint angle, This is the current theoretical end pose. This represents the theoretical end-effector pose. This state can simultaneously describe the joint space error and the remaining distance in Cartesian space.

[0376] The base state only reflects the theoretical motion and does not consider actual end-point errors. If planning is based solely on the theoretical state, the generated trajectory is prone to end-point deviation under the influence of multiple sources of error. To enhance the strategy's adaptability to errors, this embodiment introduces the prediction error of the BP neural network into the state space.

[0377] Based on the established error prediction model, the predicted terminal error value under the current state is:

[0378] ;

[0379] In the formula, The input features for the BP neural network can consist of the current joint angle, joint velocity, joint acceleration, and operating condition variables. The reconstructed state after considering error feedforward is defined as follows:

[0380] ;

[0381] If the compensated terminal residual error is also considered, an error-perceived terminal deviation term can be further constructed:

[0382] ;

[0383] In the formula, This represents the actual pose estimation of the end effector after considering prediction errors. The state vector can then be written as:

[0384] ;

[0385] Through the aforementioned state reconstruction, the agent can not only perceive the current joint state and target distance, but also obtain the system error information predicted by the BP model, thereby considering the impact of end-effector deviation on trajectory execution in advance when selecting actions.

[0386] The action space determines the trajectory control method of the agent. Continuous trajectory planning requires the action variables to be continuous to avoid the accuracy loss caused by discrete actions. The DDPG algorithm uses a deterministic strategy to output continuous actions, making it suitable for this type of problem. Actions can be defined as joint velocity increments, acceleration commands, or position increments. Considering trajectory smoothness and control feasibility, this embodiment designs the action as joint velocity increments:

[0387] ;

[0388] In a given sampling period Under these conditions, joint velocity and joint position can be updated as follows:

[0389] ;

[0390] ;

[0391] To ensure that the actions meet the robot's motion constraints, the action output needs to be limited:

[0392] ;

[0393] Additionally, after the state update, it is necessary to check whether the joint position, velocity, and acceleration meet the constraints. If the action causes the robot to go out of bounds or enter an infeasible area, a penalty should be imposed in the reward function, or the current round should be terminated directly.

[0394] In environmental interaction, the agent outputs actions based on the current state; the environment updates joint velocity and joint position accordingly, calculates the theoretical end pose and BP prediction error at the next moment, and generates a new reconstructed state.

[0395] Given the current state and actions First, the joint velocity and joint angle are updated; then, the theoretical end pose at the next moment is calculated using the forward kinematics model.

[0396] ;

[0397] The joint state and operating condition characteristics at the next time step are then input into the BP neural network to obtain the error prediction value for the next time step:

[0398] ;

[0399] Finally, the state for the next time step is generated according to the state reconstruction formula:

[0400] ;

[0401] in,

[0402] ;

[0403] To ensure clear training task boundaries, it is necessary to set round termination conditions. The current trajectory planning round can be terminated when any of the following conditions are met:

[0404] 1. The robot's end effector reaches the target area, meaning the end effector error is less than a given threshold.

[0405] 2. The joint position, velocity, or acceleration exceeds the constraint range.

[0406] 3. The robot enters a strange configuration, a collision zone, or an inaccessible area.

[0407] 4. The number of moves in the current round exceeds the maximum allowed number of moves.

[0408] The condition for the terminal to reach the target area can be expressed as:

[0409] ;

[0410] In the formula, The threshold for reaching the target is set. If this condition is met, it means that the end-effector pose estimation, after considering prediction errors, is close to the target pose, and the trajectory planning task can be considered complete.

[0411] Step 5: Building upon Step 4, design a composite reward function that incorporates runtime, energy consumption, trajectory smoothness, and end-point tracking accuracy. This enables the planning strategy to have error awareness, allowing the agent to balance accuracy, efficiency, and safety when planning the trajectory. The reward function after incorporating error feedforward needs to be consistent with the trajectory planning task, reflecting runtime efficiency, energy consumption, and smoothness, while also responding to residual deviations at the end point.

[0412] In multi-objective trajectory planning for industrial robots, a single reward term is insufficient to evaluate trajectory quality. Focusing solely on rapid arrival can lead to sudden speed changes and high energy consumption; conversely, limiting energy consumption alone may result in an overly conservative trajectory. Therefore, this embodiment incorporates operational efficiency, energy consumption, smoothness, end-point tracking accuracy, and safety constraints into the reward function.

[0413] The reward function after fusion error feedforward needs to be consistent with the trajectory planning task, reflecting operational efficiency, energy consumption, and smoothness, while also responding to residual bias at the end. Since DDPG relies on continuous interactive learning, the reward term should be kept as continuously learnable as possible to avoid training oscillations caused by overly abrupt penalty settings. For inactive actions such as joint out-of-bounds movements, collisions, and singular configurations, explicit penalties should be set to encourage the agent to actively avoid invalid regions during the search process.

[0414] ;

[0415] In the formula, As a reward for end-point tracking accuracy, Rewards for time efficiency As an energy consumption reward, As a reward for trajectory smoothness, To constrain penalties, Rewards for reaching the target; , , , , and These are the corresponding weighting coefficients. By adjusting the weights, a balance can be achieved between different performance objectives.

[0416] End-point tracking accuracy is one of the direct optimization objectives of the trajectory planning method in this embodiment. The residual end-point deviation after considering the prediction error of the BP neural network can be expressed as:

[0417] ;

[0418] In the formula, For the target theoretical end pose, This is the current theoretical end pose. This represents the terminal error predicted by the BP neural network. Compared to simply using the theoretical terminal distance, It can reflect the degree of approximation of the actual pose after taking into account system errors, and is more suitable as an accuracy evaluation index for fusion error feedforward planning.

[0419] To encourage the agent to reduce terminal residual error, the accuracy reward is designed to be in the form of negative distance:

[0420] .

[0421] Operational efficiency is a commonly used metric in trajectory planning. Shortening execution time can improve the cycle time, but excessive pursuit of speed will increase energy consumption, impact, and tracking errors. Therefore, time efficiency rewards need to be designed in conjunction with energy consumption and smoothness constraints.

[0422] Taking into account both the time cost and the proximity to the goal, the time efficiency reward can be designed as follows:

[0423] ;

[0424] In the formula, To achieve the target's reward weight, This is the penalty coefficient for single-step time.

[0425] The energy consumption of a robot during operation is closely related to joint torque, joint speed, and motion time.

[0426] In actual simulations or experiments, if a complete dynamic model cannot be established temporarily, joint velocity or motion amplitude can be used as an approximate indicator of energy consumption. Since a larger joint motion amplitude and more drastic velocity changes generally mean higher energy consumption, a reward based on motion amplitude can be constructed.

[0427] .

[0428] Trajectory smoothness directly affects robot vibration, shock, and execution stability. Abrupt changes in velocity, acceleration, or jerk can reduce tracking accuracy and exacerbate flexible deformation and end-effector vibration; therefore, smoothness constraints need to be introduced into the reward function.

[0429] If motion is defined as an increment of joint velocity, then motion changes can also reflect trajectory smoothness. In this case, a smoothness reward can be constructed using the difference between adjacent motions:

[0430] ;

[0431] This reward can suppress drastic changes in the agent's control actions, making the trajectory more continuous and smooth. In high-precision machining or assembly tasks, appropriately increasing the weight of the smoothness reward can reduce end-effector vibration and tracking errors.

[0432] In addition to time, energy consumption, smoothness, and accuracy metrics, trajectory planning must also meet motion safety constraints. Penalties should be imposed if a movement causes joints to cross boundaries, exceed speed limits, exceed acceleration limits, or enter a collision zone.

[0433] ;

[0434] In the formula, , and These represent the allowable ranges for joint position, velocity, and acceleration, respectively. , , and This corresponds to the penalty coefficient. If a serious constraint violation occurs, the current round can be terminated simultaneously to prevent the agent from continuing to explore infeasible trajectories.

[0435] When the robot's end effector reaches the target area, a significant positive reward should be given to reinforce the behavior of successfully completing the task. The reward for reaching the target can be defined as:

[0436] ;

[0437] In the formula, Rewards for reaching the goal This represents the target error threshold. This reward, combined with the time penalty, motivates the agent to reach the target as quickly as possible while maintaining accuracy.

[0438] After merging the above reward items, this embodiment constructs a multidimensional composite reward function for fusion error feedforward:

[0439] ;

[0440] In the formula, each weight coefficient is used to adjust the importance of different optimization objectives during the training process. If the task requires high accuracy at the end point, the weight coefficient can be increased. If more emphasis is placed on operational efficiency, it can be improved. If the task requires high motion stability, then it can be improved. In actual training, the reward weights should be adjusted based on the convergence curve, trajectory quality, and task requirements.

[0441] The normalized composite reward is represented as:

[0442] ;

[0443] In the formula, For the number of reward items, For the normalized first Individual reward items, These correspond to the weights. By combining normalization and weighting, training stability can be improved and the multi-objective optimization effect can be enhanced.

[0444] Step Six: Based on Step Five, construct and train the DDPG trajectory planning agent, which mainly includes: Actor-Critic network structure design, experience playback and exploration strategies, construction of a training environment with fused error feedforward, training stability and parameter settings, etc.

[0445] DDPG is a policy-dependent reinforcement learning algorithm for continuous action spaces. Its core consists of an Actor network, a Critic network, a target network, and an experience replay pool. The Actor network outputs a deterministic action based on the current state, and the Critic network evaluates the value of state-action pairs. Let the Actor network be:

[0446] ;

[0447] In the formula, This is the current state. To output the action, These are the parameters of the Actor network. The Critic network is used to estimate the action value function.

[0448] ;

[0449] In the formula, These are the parameters of the Critic network. The goal of DDPG is to learn the optimal deterministic policy. This allows the agent to obtain the maximum cumulative reward during trajectory planning:

[0450] .

[0451] The Critic network is trained by minimizing the temporal difference error. This is applied to transfer samples obtained from the empirical replay pool. The target Q value is:

[0452] ;

[0453] In the formula, For the target Actor network, For the target Critic network, and These are the corresponding target network parameters. Let be the discount factor. The loss function of the Critic network is:

[0454] .

[0455] The Actor network is updated using the gradient of a deterministic policy, which is approximately:

[0456] .

[0457] Through the above update process, the Actor network gradually learns action output strategies that can obtain higher long-term rewards, while the Critic network continuously corrects its estimation of the value of the current strategy.

[0458] The DDPG trajectory planning agent constructed in this embodiment adopts an Actor-Critic dual-network structure. The input to the Actor network is the error feedforward reconstruction state defined in Section 4.1. The output is a continuous action. If the motion is defined as the velocity increment of six joints, then the Actor network output dimension is 6.

[0459] .

[0460] Actor networks can employ a multi-layer fully connected structure, with the number of nodes in the input and output layers corresponding to the state and action dimensions, respectively. Hidden layers use ReLU or Tanh activation functions to extract features, while the output layer uses the Tanh activation function and undergoes action-scale mapping to constrain the output to an acceptable range.

[0461] ;

[0462] In the formula, This represents the hidden layer mapping of the Actor network. This is the upper limit of the motion amplitude. This indicates element-wise multiplication.

[0463] Critic networks are used to evaluate state-action pairs. Its value lies in its input, which consists of a state vector and an action vector, and its output is a scalar Q-value.

[0464] .

[0465] The Critic network first extracts state features, then concatenates them with action vectors and inputs them into subsequent fully connected layers to evaluate action value. This structure can simultaneously consider the current state, prediction error, and the impact of actions on long-term rewards. To improve training stability, DDPG constructs target networks with identical structures for both the Actor and Critic networks, and their parameters are soft-updated.

[0466] ;

[0467] ;

[0468] In the formula, This is the soft update coefficient, which is usually taken as a small value. Soft updates can avoid drastic changes in the target Q value and improve the stability of the training process.

[0469] In reinforcement learning training, continuously sampled state transition samples are highly correlated. If used directly for training, they can easily lead to unstable convergence. DDPG employs an empirical replay mechanism, storing interactive samples in a replay pool and randomly sampling small batches of data to update the network.

[0470] Because DDPG's Actor network outputs deterministic actions, it is prone to early entrapment in local optima without an exploration mechanism. To enhance its exploration capabilities, noise is superimposed on the Actor output during training.

[0471] ;

[0472] In the formula, To explore noise.

[0473] The DDPG trajectory planning agent needs to be trained in an interactive environment. This environment consists of a robot kinematics model, joint constraints, a backpropagation (BP) error prediction model, a reward function, and a state transition module. During each interaction, the agent outputs joint velocity increments or equivalent continuous control actions based on the current state. The environment updates the joint states accordingly and calculates the new theoretical end-effector pose, prediction error, reward value, and termination flag. During environment initialization, the starting joint angles and target pose can be set according to task requirements or randomly generated within a given range to improve the policy's adaptability to different task conditions.

[0474] During environment initialization, the starting joint angle is set randomly or according to task requirements. and target joint angle And calculate the target theoretical end pose:

[0475] ;

[0476] In each interaction step, the environment first updates the joint velocity and joint position based on the agent's output action:

[0477] ;

[0478] ;

[0479] The theoretical end-effector pose is then calculated using the forward kinematics model:

[0480]

[0481] Then, the trained BP neural network is invoked to predict the terminal error in the current state:

[0482] ;

[0483] Construct error-perceived residual bias based on prediction error:

[0484] ;

[0485] Finally, the next state is generated. The reward value is calculated based on the composite reward function. If the robot reaches the target area, violates constraints, or exceeds the maximum number of steps, the environment returns a termination flag.

[0486] By incorporating the BP error prediction model into the planning environment and participating in DDPG training, the agent continuously receives error feedforward information during training, rather than performing isolated compensation after trajectory generation.

[0487] The training process for the DDPG agent with fused error feedforward is as follows.

[0488] like Figure 4 As shown, this process describes the training procedure for DDPG trajectory planning using a backpropagation (BP) neural network with error feedforward. First, the Actor network, Critic network, target network, and experience replay pool are initialized, and a pre-trained BP neural network error prediction model is loaded. Then, starting and target points are randomly generated or set according to task requirements to construct the initial state. The Actor network outputs continuous actions based on the current state, and exploration noise is added to enhance the policy search capability. Upon receiving the actions, the environment updates the joint states and calculates the theoretical end-effector pose, while simultaneously calling the BP network to predict the current end-effector error, thus reconstructing the state for the next time step. Based on this, the system calculates the current reward according to the composite reward function and stores the interaction samples in the experience replay pool. Subsequently, small batches of data are randomly sampled from this pool to update the parameters of the Critic and Actor networks, while the target network undergoes a soft update. Finally, the system determines whether the termination condition is met: if not, it continues to the next round of interaction; if met, it outputs the trained DDPG agent. This process introduces error prediction results feedforward into the reinforcement learning training process, enabling the trajectory planning strategy to perceive end-effector errors in advance during the training phase, thereby improving the adaptability of the planning results to actual execution deviations. DDPG training is affected by hyperparameters such as learning rate, discount factor, soft update coefficient, replay pool capacity, batch size, and exploration noise. Improper parameter settings may lead to oscillations, slow convergence, or getting stuck in local optima, and need to be adjusted in conjunction with the trajectory planning task.

[0489] Table 3 shows the design format of DDPG training parameter settings.

[0490] Table 3 DDPG Agent Training Parameter Settings

[0491]

[0492] Step Seven: Finally, taking a six-DOF serial industrial robot as the object, a simulation platform is constructed around "kinematic modeling—error prediction—trajectory planning—performance evaluation". The platform includes a kinematics module, a multi-source error simulation module, a BP neural network error prediction module, a DDPG trajectory planning module, and a result evaluation module.

[0493] This embodiment uses a six-degree-of-freedom serial industrial robot as an example, and constructs a simulation platform around "kinematic modeling—error prediction—trajectory planning—performance evaluation". The platform includes a kinematics module, a multi-source error simulation module, a BP neural network error prediction module, a DDPG trajectory planning module, and a result evaluation module. Its overall architecture is as follows: Figure 5 As shown.

[0494] The simulation platform consists of modules for kinematics calculation, multi-source error simulation, backpropagation (BP) error prediction, direct directional projection (DDPG) trajectory planning, and result evaluation. The kinematics module calculates the theoretical pose of the end effector; the multi-source error simulation module simulates geometric errors, transmission errors, flexible deformation, and random noise; the BP module predicts the end effector error based on state characteristics; the DDPG module, after incorporating the predicted error, makes continuous motion decisions in the state space; and the evaluation module compares indicators such as compensation effect, convergence, trajectory smoothness, and energy consumption. The platform data stream starts from the robot's state input, passes through forward kinematics to obtain the theoretical end effector pose, then superimposes the error model to obtain the actual deviation, and returns the prediction result to the trajectory planning module.

[0495] The core data flow of the simulation platform can be described as follows: First, the robot state is input, and then the theoretical end-effector pose is calculated using a forward kinematics model. The terminal error is then generated by the error simulation module or obtained from experimental measurements. To form the actual end pose Next, the backpropagation neural network is trained using the sample set to output the prediction error. Finally, the prediction error is used as the state feedforward input to the DDPG agent for trajectory planning and action decision-making.

[0496] The simulation environment uses an improved DH kinematic model and simulates multi-source errors through geometric parameter perturbation, joint offset, flexible approximation and random noise to generate training and validation samples.

[0497] ;

[0498] The theoretical pose of the end effector can then be represented by the forward kinematics model as follows:

[0499] ;

[0500] In the formula, It includes end-effector position and orientation information. To simulate the multi-source error effects encountered by the robot during actual operation, the actual end-effector pose can be represented as:

[0501] ;

[0502] in, This is the comprehensive end-effector error. Based on the established comprehensive error mapping model, the comprehensive error can be composed of geometric error, joint transmission error, flexible deformation error, thermally induced deformation error, and random noise, among other components.

[0503] .

[0504] During the simulation phase, error terms can be set in combination: geometric error is represented by small perturbations of the DH parameter, transmission error is simulated by joint angle offset or random perturbation, flexibility error is approximated by end load and configuration, and measurement noise uses a small-amplitude random variable with zero mean to construct training samples with nonlinear and coupling characteristics.

[0505] Let the first The theoretical pose corresponding to each sample is: The actual pose obtained through simulation or measurement is The error label is:

[0506] .

[0507] In this embodiment, the prediction error of the BP neural network is used as the feedforward state of the DDPG agent, enabling the planning process to detect possible end-point deviations under the current configuration in advance.

[0508] The collaborative operation consists of two phases: offline training and online planning. In the offline phase, the BP error prediction model is trained and embedded into the DDPG environment. In the online phase, the agent outputs continuous actions based on the state of the fused prediction error, and the planning quality is evaluated through a composite reward.

[0509] ;

[0510] In the formula, The input vector consists of joint angles, joint velocities, joint accelerations, and operational characteristics. After the BP model is trained, it is embedded in the DDPG trajectory planning environment to generate error feedforward states.

[0511] During the online planning phase, at each decision step, the agent invokes the BP network based on the current robot state to predict the terminal error and constructs a reconstructed state:

[0512] ;

[0513] Subsequently, DDPG's Actor network outputs continuous actions based on the reconstruction state:

[0514] ;

[0515] The environment updates the robot's joint states based on the actions and evaluates the quality of the current actions using a composite reward function.

[0516] ;

[0517] in, The normalized sub-reward term includes metrics such as terminal accuracy, time efficiency, energy consumption, smoothness, and constraint penalties. Through the aforementioned closed-loop process, the DDPG agent can learn during training how to utilize error prediction information to improve trajectory planning quality.

[0518] The collaborative operation process can be summarized in the following steps:

[0519] 1. Establish the robot's kinematic model and set the error simulation parameters.

[0520] 2. Collect or generate joint state samples and calculate the error between theoretical pose and actual pose.

[0521] 3. Construct training, validation, and test sets to train the BP neural network error prediction model.

[0522] 4. Use the BP network output as the error feedforward feature in the DDPG state space.

[0523] 5. Training a DDPG trajectory planning agent based on a composite reward function.

[0524] 6. Compare the error compensation effects and trajectory planning performance of different methods.

[0525] To evaluate the performance of the "prediction-planning" collaborative method, this embodiment sets evaluation indicators from aspects such as error prediction accuracy, positioning compensation effect, and trajectory planning quality.

[0526] Error prediction accuracy was evaluated using RMSE, MAE, and MAXE.

[0527] The positioning compensation effect can be evaluated by the change in average error before and after compensation and the error compensation rate:

[0528] ;

[0529] In the formula, To compensate for the previous average error, This is the average residual error after compensation.

[0530] Trajectory planning quality can be evaluated from aspects such as running time, energy consumption, trajectory smoothness, and end-point tracking error. Trajectory smoothness can be represented by the Jerk index.

[0531] ;

[0532] The end-point tracking error can be expressed as:

[0533] ;

[0534] In the formula, For reference end pose, This represents the actual end-effector pose after accounting for errors.

[0535] Furthermore, the algorithm's training performance can also be evaluated through the cumulative reward curve, the number of convergence rounds, and the task success rate. The task success rate is defined as:

[0536] ;

[0537] In the formula, The number of tests to successfully reach the target area. This represents the total number of tests.

[0538] The above indicators correspond to error prediction capability, compensation effect, planning quality, and reinforcement learning training process, respectively.

[0539] Example 2:

[0540] This embodiment verifies the error compensation performance of a BP neural network. The end-effector error of industrial robots is characterized by multi-source coupling and uneven spatial distribution, making it difficult to interpret model performance based solely on training set errors. This embodiment divides the dataset into training, validation, and test sets, evaluates the predictive ability of the BP neural network on unseen samples, and analyzes it from dimensions such as compensation rate, residual error distribution, and spatial generalization ability.

[0541] Joint state samples are generated or collected within the robot's main workspace, and error labels are formed by the difference between the theoretical pose and the actual pose. The input features and output labels are normalized and then divided into training, validation, and test sets.

[0542] ;

[0543] in, This represents the total number of samples. The error label is obtained by subtracting the theoretical end-effector pose from the actual end-effector pose.

[0544] ;

[0545] In the formula, The end-effector pose is obtained from actual measurement or simulation. The theoretical end pose is calculated using the forward kinematics model.

[0546] To improve training stability, both input features and output labels are normalized. This embodiment uses the min-max normalization method to map the data to... Interval:

[0547] ;

[0548] During model training, the changes in loss on both the training and validation sets should be observed simultaneously. Figure 6 It can be seen that the training set loss and validation set loss of the BP neural network decrease continuously with the increase of the number of training rounds, and tend to stabilize in the later stage. The loss decreases rapidly in the early stage of training, indicating that the network can quickly learn the nonlinear mapping relationship between joint state, working condition characteristics and end error; the gap between the two curves is small in the later stage of training, and the validation set loss does not show a significant rebound, indicating that the model has not experienced significant overfitting and has good generalization ability.

[0549] After the BP neural network is trained, the test set samples are input into the network to obtain the terminal error prediction value:

[0550] ;

[0551] Prediction error Error with respect to the actual value By comparison, the model prediction residuals can be obtained:

[0552] ;

[0553] To quantitatively evaluate prediction accuracy, this embodiment uses RMSE, MAE, and MAXE as evaluation metrics. If all three metrics are low, and the prediction error curve can follow the actual error trend well, it indicates that the BP neural network can effectively fit the error distribution of the robot's end effector.

[0554] Table 4 presents the statistical data on error prediction accuracy.

[0555] Table 4. Statistics on the Prediction Accuracy of BP Neural Network

[0556]

[0557] As shown in Table 4, the BP neural network exhibits good accuracy in position error prediction. The RMSEs in the X, Y, and Z directions are 0.4417 mm, 0.4054 mm, and 0.3804 mm, respectively, and the MAEs are 0.3427 mm, 0.3300 mm, and 0.3037 mm, respectively, with the Z direction showing the smallest prediction error. The overall position error has an RMSE of 0.7101 mm, an MAE of 0.6553 mm, and a maximum error of 1.4051 mm. All indicators remain within a relatively low three-dimensional spatial geometric scale, and the maximum error does not show abnormal divergence, indicating that the model can stably predict the end-effector position deviation.

[0558] In attitude error prediction, the RMSE values ​​for the Rx, Ry, and Rz directions were 0.0618deg, 0.0694deg, and 0.0573deg, respectively, which are generally low. The RMSE for the overall attitude error was 0.1091deg, the MAE was 0.0991deg, and the maximum error was 0.3169deg. Figure 7 As can be seen, the predicted values ​​of each error component are distributed around the ideal regression line without significant nonlinear drift, indicating a strong consistency between the prediction results and the actual errors. Therefore, this model not only generalizes strongly nonlinear mapping relationships well, but also demonstrates that the BP network can effectively fit the multidimensional error characteristics of the robot's end effector, providing reliable data support for subsequent robot accuracy compensation.

[0559] Error compensation is achieved by inversely superimposing the backpropagation (BP) prediction error onto the theoretical target pose. Experiments compare the average error, maximum error, and error distribution before and after compensation, and calculate the compensation rate to evaluate the degree of improvement in positioning accuracy.

[0560] ;

[0561] This compensation method is equivalent to performing a reverse correction on the theoretical pose before the robot executes, so that the actual end-effector pose after execution is as close as possible to the target pose. Let the first... The error for each sample is:

[0562] ;

[0563] The residual error after compensation is:

[0564] ;

[0565] The average error before compensation and the average residual error after compensation are respectively:

[0566] ;

[0567] ;

[0568] Error compensation rate is defined as:

[0569] .

[0570] Depend on Figure 8 and Figure 9 It can be seen that after using a BP neural network for feedforward compensation of prediction errors, both the robot's end-effector position and posture errors are significantly reduced. Before compensation, the average position error was 1.5390 mm, and after compensation, the average residual error was 0.6553 mm, with an error compensation rate of 57.42%. Before compensation, the average posture error was 0.2241 degrees, and after compensation, the average residual error was 0.0991 degrees, with an error compensation rate of 55.77%. This indicates that the BP neural network can effectively learn the nonlinear mapping relationship between the robot's end-effector error and joint states and working conditions, and that feedforward compensation effectively weakens the impact of multi-source errors on the end-effector pose accuracy. The error distribution after compensation is generally closer to the small error range, with reduced dispersion, indicating that this method not only reduces the average error but also improves the stability of the error distribution.

[0571] Example 3:

[0572] This embodiment evaluates the comprehensive performance of DDPG multi-objective planning. To assess the continuous trajectory planning capability of the DDPG agent, this embodiment sets up a point-to-point task from the initial pose to the target pose, comparing the results under the same initial point, target point, joint constraints, maximum number of steps, and error model. In each planning task, the agent provides joint velocity increments or equivalent continuous control actions based on the current state. The environment then updates the joint states and calculates the theoretical end-effector pose, BP prediction error, and composite reward value. To ensure comparability of the comparison results, all three methods use the same initial point, target point, joint constraints, maximum number of steps, and simulation error model. Figure 10 As shown, this embodiment sets up three sets of comparison methods. The first set is the traditional trajectory planning method, which uses conventional interpolation trajectory or single-objective optimization method as the benchmark scheme. The second set is the ordinary DDPG method, which does not include the BP error prediction result in the state space, and only plans based on the theoretical joint state and target deviation. The third set is the BP-DDPG collaborative planning method, which adds the BP network prediction error to the DDPG state space and uses a multi-dimensional composite reward function training strategy.

[0573] The quality of trajectory planning is evaluated from aspects such as running time, approximate energy consumption indicators, trajectory smoothness, and end-point tracking error.

[0574] For a discrete trajectory, the running time can be expressed as:

[0575] ;

[0576] In the formula, The number of steps to execute for the trajectory. The sampling period is [value]. The length of the joint spatial trajectory can be expressed as:

[0577] ;

[0578] If we temporarily disregard the complete dynamic model, we can use the amplitude of motion or the sum of squares of joint velocities as an approximate indicator of energy consumption:

[0579] ;

[0580] Trajectory smoothness can be evaluated using the Jerk index:

[0581] ;

[0582] End-effector tracking error is used to reflect the deviation between the actual end-effector pose and the reference pose after taking into account the influence of errors.

[0583] ;

[0584] In the formula, For reference end pose, This is the actual end-effector pose estimation after taking errors into account.

[0585] Table 5 Comparison of Trajectory Quality for Different Planning Methods

[0586]

[0587] According to Table 5, the three trajectory planning methods showed significant differences in various metrics. Traditional trajectory planning took 85.1777 ms, with a trajectory length of 0.6889 m, an approximate energy consumption index of 7.3933, and an end-tracking error of 1.8446 mm. Ordinary DDPG planning had the shortest planning time at only 31.2910 ms and a trajectory length of 0.6257 m, but its approximate energy consumption index increased to 44.2529, and its end-tracking error was 1.8585 mm. BP-DDPG collaborative planning took 127.1912 ms, with a trajectory length of 1.2774 m and an approximate energy consumption index of 64.6317, but its end-tracking error decreased to 0.7020 mm. This indicates that after fusing error feedforward, the algorithm sacrificed computational efficiency and energy consumption, but significantly improved end-tracking accuracy.

[0588] To verify the role of BP prediction error in DDPG planning, this embodiment compares ordinary DDPG with BP-DDPG. Ordinary DDPG makes decisions based solely on theoretical states, while BP-DDPG incorporates prediction errors and residual biases into the state and reward, thus proactively reducing the impact of multi-source errors on end-tracking accuracy.

[0589] In conventional DDPG methods, the agent can only determine whether the end effector is close to the target based on a theoretical model, making it difficult to perceive the actual end effector offset caused by geometric errors, flexible deformation, and transmission errors. The target distance can typically be expressed as:

[0590] ;

[0591] The BP-DDPG method uses the residual bias after considering the prediction error:

[0592] ;

[0593] The effect of error feedforward state reconstruction can be analyzed by comparing the residual errors before and after compensation. Let the average tracking error at the end of a standard DDPG be... The average tracking error at the end of the BP-DDPG is Then the error improvement rate can be expressed as:

[0594] .

[0595] Table 6 Comparison of Error Feedforward State Reconstruction Effects

[0596]

[0597] Table 6 shows that error feedforward state reconstruction significantly improves trajectory accuracy. The average tracking error of ordinary DDPG is 1.8585 mm, and the maximum tracking error is 2.2696 mm; the average tracking error of BP-DDPG collaborative planning is reduced to 0.7020 mm, and the maximum tracking error is reduced to 1.1945 mm. Compared with ordinary DDPG, the average tracking error is reduced by approximately 62.23%, and the maximum tracking error is reduced by approximately 47.37%. This indicates that BP prediction error can not only be used for offline compensation, but also as feedforward information in the DDPG state space, enabling the agent to perceive end-point deviations in advance and make proactive adjustments during the planning process.

[0598] There are often conflicts between the time, energy consumption, smoothness, and accuracy objectives in robot trajectory planning. To verify the effect of the composite reward function in this embodiment, it is necessary to analyze the performance of the BP-DDPG method on multiple objectives.

[0599] This embodiment uses a normalized comprehensive score to evaluate the multi-objective performance of different methods. Let a certain method be... The normalization results for each indicator are The overall performance index can then be expressed as:

[0600] ;

[0601] Overall performance indicators The smaller the value, the better the overall performance of the planning method.

[0602] like Figure 11 As shown, from the perspective of multi-objective comprehensive cost indicators, the comprehensive costs of traditional trajectory planning, ordinary DDPG, and BP-DDPG collaborative planning are 85.6733, 97.7609, and 45.1824, respectively. BP-DDPG collaborative planning has the lowest comprehensive cost, approximately 53.78% lower than ordinary DDPG and approximately 47.26% lower than traditional trajectory planning. Therefore, although BP-DDPG is not optimal in terms of planning time, trajectory length, and energy consumption approximation indicators, it has significant advantages in end-point tracking accuracy and comprehensive evaluation, indicating that the "error prediction-trajectory planning" collaborative mechanism can effectively improve the trajectory execution quality under the influence of multi-source errors.

[0603] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A multi-objective trajectory planning method for robot fusion error feedforward based on DDPG, characterized in that, Includes the following steps: S1. Establish the kinematic model and comprehensive error mapping model of the robot; the kinematic model is established based on the improved DH parameter method, including the forward kinematic model and the velocity kinematic model; the comprehensive error mapping model is a unified mapping relationship between multiple source errors and end-effector pose errors established based on geometric parameter errors, joint transmission errors, flexible deformation errors, thermal deformation errors and control and measurement errors. S2. Construct a BP neural network error prediction model, taking the robot's joint motion state and working condition characteristics as input, and outputting the end-effector position error and posture error; construct a sample set and train it to obtain the predicted value of the end-effector pose error; S3. Construct a DDPG trajectory planning agent based on DDPG, and introduce the predicted end pose error value obtained in step S2 into the state space to perform trajectory planning. S4. Design a multi-dimensional composite reward function to guide the DDPG trajectory planning agent to achieve multi-objective balance in trajectory planning; S5. Continuously train the DDPG trajectory planning agent in a simulation environment that integrates the BP neural network error prediction model, output continuous joint actions, and generate an optimized trajectory that meets the requirements of multiple objectives.

2. The DDPG multi-objective trajectory planning method for robot fusion error feedforward as described in claim 1, characterized in that, In step S1, the kinematic model is established based on the improved DH parameter method, as follows: Based on the improved DH parameter definition, from the coordinate system To coordinate system The homogeneous transformation is decomposed into a revolving shape. Rotation along the axis Axis translation, rotation Axis rotation and along The four basic transformations of axis translation are: ； Its matrix form is: ； in, Indicates the first The coordinate system of the first link relative to the first link coordinate system Pose transformation matrix of each link coordinate system Indicates the length of the link. Indicates the link twist angle. Indicates joint offset. This indicates that the joint angle is zero-position offset; The forward kinematics model is the homogeneous transformation matrix of the end coordinate system relative to the base coordinate system, as follows: ； The velocity-kinematic model is obtained by differentiating the forward kinematic equations: ； in, Let be the velocity vector of the end effector in Cartesian space. For joint velocity vectors, Let Jacobian matrix be used for the robot.

3. The DDPG multi-objective trajectory planning method for robot fusion error feedforward as described in claim 2, characterized in that, The construction of the comprehensive error mapping model in step S1 is as follows: ； in, For the end-effector integrated pose error, For geometric parameter errors, This is due to joint transmission error. For flexible deformation error, For thermally induced deformation error, For measurement error; This can be further expressed as a linear mapping of the generalized error input vector: ； in, This is the comprehensive error mapping matrix. This is the generalized error input vector.

4. The DDPG multi-objective trajectory planning method for robot fusion error feedforward as described in claim 3, characterized in that, The input for constructing the BP neural network error prediction model in step S2 is: ； in, For joint variable vectors, and These are joint velocity and joint acceleration, used to characterize the robot's motion state; This refers to the end load or equivalent processing force. Characteristics of joint or ambient temperature; The output is the predicted end-effector pose error value: ； in, Therefore, a data-driven error prediction model is established: ； in, This represents the nonlinear mapping function established by the BP neural network. and These are the network weights and bias parameters, respectively.

5. The DDPG multi-objective trajectory planning method for robot fusion error feedforward as described in claim 4, characterized in that, The sample set in step S2 is represented as follows: ； in, For the sample size, For the system input quantity corresponding to the k-th sample, For the first The actual terminal error corresponding to each sample; ； In the formula, The actual end-effector pose obtained by measurement, The theoretical end pose is calculated from the kinematic model.

6. The DDPG multi-objective trajectory planning method for robot fusion error feedforward according to claim 5, characterized in that, The error feedforward reconstruction state in the state space during step S3 is represented as follows: ； in, The current joint angle, The current joint velocity, For the target joint angle, To account for the terminal residual bias after considering the prediction error of the BP neural network, This is the current predicted value of the terminal error.

7. The DDPG multi-objective trajectory planning method for robot fusion error feedforward as described in claim 6, characterized in that, The multidimensional composite reward function in step S4 is expressed as follows: ； in, As a reward for end-point tracking accuracy, Rewards for time efficiency As an energy consumption reward, As a reward for trajectory smoothness, To restrain punishment, Rewards for reaching the target; , , , , , These are the weighting coefficients for each item.

8. The DDPG multi-objective trajectory planning method for robot fusion error feedforward as described in claim 7, characterized in that, The end-tracking accuracy bonus It is expressed as follows: ； in, ； In the formula, For the target theoretical end pose, This is the current theoretical end pose. This is the current predicted value of the terminal error; The time efficiency reward It is expressed as follows: ； in, To achieve the target's reward weight, This is the penalty coefficient for single-step time. Energy consumption reward It is expressed as follows: ； in, For the current continuous action; The trajectory smoothness reward It is expressed as follows: ； in, This refers to the continuous action from the previous moment; The constraint penalty It is expressed as follows: ； in, , and These represent the allowable ranges for joint position, velocity, and acceleration, respectively. , , and This corresponds to the penalty coefficient; The target reached reward It is expressed as follows: ； in, Rewards for reaching the goal The target error threshold.

9. The DDPG multi-objective trajectory planning method for robot fusion error feedforward as described in claim 8, characterized in that, In step S5, the DDPG trajectory planning agent adopts an Actor-Critic dual-network structure to reconstruct the state with error feedforward. As input to the Actor network, in continuous action As output, the Critic network is used to evaluate state-action pairs. The value of Q is determined by the fact that its input consists of a state vector and an action vector, and its output is a scalar Q-value. ； Furthermore, the DDPG trajectory planning agent uses soft updates to update the Actor network parameters and Critic network parameters: ； ； In the formula, This is the soft update coefficient. , These are the parameters for the Actor network and the Critic network, respectively. , These are the target Actor network parameters and the target Critic network parameters, respectively.

10. The DDPG multi-objective trajectory planning method for robot fusion error feedforward according to claim 9, characterized in that, In step S5, the DDPG trajectory planning agent is also trained by integrating BP neural network error feedforward. The specific method is as follows: a. First, initialize the Actor network, Critic network, target network, and experience replay pool, and load the already trained BP neural network error prediction model; b. Randomly generate or set the starting point and target point according to the task requirements to construct the initial state; c. The Actor network outputs continuous actions based on the current state and adds exploration noise to enhance the policy search capability. After the environment receives the actions, it updates the joint states and calculates the theoretical end pose. At the same time, it calls the BP network to predict the current end error and then reconstructs the state at the next moment. d. Calculate the current reward based on the multidimensional composite reward function, store the interaction samples in the experience replay pool, and then randomly sample a small batch of data from it to update the parameters of the Critic network and Actor network, while performing a soft update on the target network. e. Determine if the termination condition is met: If the condition is not met, proceed to the next round of interaction; If the conditions are met, the trained DDPG agent is output.

Images