A Dynamic Compensation Method for Cumulative Installation Error Trajectory in Industrial Robots

Abstract

This invention discloses a dynamic compensation method for cumulative installation error trajectories of industrial robots. The method includes: constructing a digital twin model containing inherent physical error parameters; establishing a high-fidelity virtual environment by generating error trajectories; training an initial pose compensation strategy network using the PPO algorithm for reinforcement learning within the virtual environment; aligning high-frequency joint states with low-frequency external visual errors using an asynchronous data fusion mechanism during actual operation, and reconstructing a comprehensive state input strategy network to achieve online dynamic compensation; collecting historical data and tracking deviations from actual operation, and using an offline reinforcement learning algorithm with behavioral cloning constraints to safely fine-tune the network. This invention effectively overcomes the shortcomings of traditional static calibration in handling dynamic time-varying errors, balances the physical security of algorithm deployment with long-term adaptive capability, and improves the trajectory tracking accuracy of the robot.

Description

Technical Field

[0001] This invention relates to the field of industrial robot motion control technology, and in particular to a method for dynamic compensation of installation cumulative error trajectory for industrial robots. Background Technology

[0002] With the continuous evolution of intelligent manufacturing, industrial robots have been widely used in high-precision processing scenarios such as aerospace and automotive assembly. However, during manufacturing, multi-joint assembly, and long-cycle operation, industrial robots inevitably experience machining tolerances, cumulative installation errors of joint links, and stiffness deformation, leading to deviations between their actual end-effector trajectory and the theoretical nominal trajectory. To improve the absolute positioning and trajectory tracking accuracy of robots, most research focuses on kinematic calibration and dynamic compensation. Traditional error compensation techniques typically rely on external high-precision measuring equipment to establish a static geometric error model based on standard kinematics through offline parameter identification. In recent years, some research has begun to introduce conventional data-driven or neural network algorithms, attempting to compensate for difficult-to-analyze nonlinear errors by fitting historical operating trajectories, which has played a certain role in improving trajectory accuracy under specific and fixed working conditions.

[0003] Despite some progress in existing technologies, they still fall short of meeting the long-term, high-precision requirements under complex and variable working conditions. First, traditional analytical modeling and static offline calibration often treat geometric parameter errors and dynamic characteristics separately, failing to dynamically eliminate time-varying cumulative errors caused by external load fluctuations and environmental disturbances. Second, existing pure neural network compensation methods heavily rely on massive trial-and-error sampling of physical entities, resulting in high costs for real data acquisition, the risk of equipment collision damage, and a lack of generalization and anti-interference capabilities for unknown states and sudden changes in working conditions. Furthermore, existing compensation control systems typically lack a virtual-real interactive verification environment and a self-evolution mechanism for strategies. Once deployed, the compensation algorithm remains fixed, unable to safely iterate and fine-tune offline based on the long-term degradation of the robot's mechanical performance. This makes the system highly susceptible to failure when facing unpredictable external dynamic errors, severely limiting its long-term stable application in complex industrial environments. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a dynamic compensation method for the cumulative installation error trajectory of industrial robots to solve the problems mentioned in the background art.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for dynamic compensation of installation cumulative error trajectory for industrial robots, comprising:

[0007] Construct a digital twin model corresponding to the physical industrial robot, the model integrating the robot's geometric error parameters and dynamic parameters;

[0008] In the virtual environment constructed by the digital twin model, a compensation policy network is trained using a reinforcement learning algorithm to obtain the ability to generate pose compensation based on the robot's real-time state and error information.

[0009] The trained compensation strategy network is deployed into the control loop. During online operation, based on the real-time acquired robot state and external dynamic error information, the compensation strategy network generates a real-time pose compensation amount, and the compensation amount is used to correct the robot's target trajectory command.

[0010] During the robot's performance of the compensation task, robot operation data and task effect data are collected, and the operation data is used to perform offline optimization and iteration on the compensation strategy network.

[0011] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0012] 1. This invention adopts a two-layer architecture that combines digital twins and reinforcement learning. All high-risk trial and error explorations are carried out in a high-fidelity digital twin model. After the policy network converges and the parameters are frozen, the inference application is seamlessly transferred to the physical machine, thereby ensuring the security of expensive physical devices.

[0013] 2. By introducing external visual features and employing a zero-order hold, this invention aligns low-frequency external dynamic errors with high-frequency joint control data and inputs them into the compensation strategy network, achieving millisecond-level online dynamic compensation within each interpolation cycle, which significantly improves the absolute trajectory accuracy of the robot under complex working conditions.

[0014] 3. In addition, to address the problem of residual deviations in virtual-to-real migration caused by wear and aging of robots during long-term operation, this invention introduces behavioral cloning regularization constraints and performs network fine-tuning based on massive historical safe operation data. This achieves adaptive learning of errors that evolve over time and strictly limits the boundaries of network output, ensuring the stability of long-term operation in industrial settings. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0016] Figure 1 This is a flowchart illustrating the overall process of a dynamic compensation method for cumulative installation error trajectory of industrial robots according to an embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0020] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0021] Example 1

[0022] Reference Figure 1 This is the first embodiment of the present invention, which provides a method for dynamic compensation of installation cumulative error trajectory for industrial robots, including:

[0023] S1. Construct a digital twin model corresponding to the physical industrial robot, which integrates the robot's geometric error parameters and dynamic parameters.

[0024] It should be noted that, in order to overcome the technical defects of static spatial calibration and dynamic real-time compensation being separated in traditional industrial robot control systems, the present invention provides a virtual training environment for reinforcement learning algorithms with zero trial and error cost, no collision risk, and strict physical laws by mapping a digital twin with flaws (i.e., error characteristics) and mechanical responses that are completely consistent with those of a real robot in virtual space.

[0025] Furthermore, in the actual assembly and on-site deployment of industrial robots, errors not only originate from the machining tolerances of the connecting links themselves, but are also affected by the tilt of the base on the ground and the installation eccentricity of the end effector (flange or tool). Therefore, this embodiment establishes a comprehensive error model that uniformly describes the parameter errors of each joint link of the robot, the mounting posture error of the base, and the posture error of the tool center point.

[0026] Specifically, based on the modified DH (MDH) kinematic modeling method, the end-effector pose transformation matrix in the nominal state is defined. Considering the small perturbation characteristics of the cumulative installation error, a comprehensive error model can be established by introducing the principle of differential kinematic mapping, and its mathematical expression is as follows: ; in, The vector represents the total six-degree-of-freedom pose deviation of the robot's end effector in Cartesian space (including three position deviations and three attitude deviations). Its physical meaning is the spatial topological difference between the robot's actual trajectory and its theoretical nominal trajectory. This represents the base mounting pose error vector (i.e., the translation and rotation error of the base coordinate system relative to the world coordinate system). Representing the The geometric error parameter vector of each joint link (usually including the link length, link torsion angle, joint offset, and joint angle offset deviation); The vector representing the pose error of the tool's center point; , , These represent the error Jacobian matrices of the corresponding error parameters relative to the end pose, respectively, characterizing the topology transfer sensitivity of the cumulative amplification of errors from each local installation node to the end. The total number of joints of the industrial robot (in this embodiment, ).

[0027] Furthermore, in a real-world scenario, the robot is controlled to move within its workspace to multiple preset configurations (in this example, preferably 50-100 attitude points distributed at the edge and center of the workspace, denoted as...). At each configuration point At this configuration point, the actual end-effector pose data is acquired using an external high-precision measuring device (in this embodiment, a laser tracker). The difference between this actual pose data and the theoretical nominal pose within the robot controller is then calculated to obtain the actual pose measurement deviation vector at that configuration point. By summing all deviation vectors at this configuration point, a multi-dimensional pose error measurement data matrix can be constructed. After obtaining this data matrix, in order to solve for the unknown error parameters in the comprehensive error model (i.e., the aforementioned base mounting pose error vector, joint link geometric error parameter vector, and tool center point pose error vector), an optimization objective function is constructed with the goal of minimizing the sum of squared residuals. Its mathematical expression is as follows: ; in, The objective function value represents the overall degree of matching between the prediction error generated by the identified digital twin model and the measurement error actually generated in the physical world; the smaller the objective function value, the closer the model is to the real robot. Representative at the The actual pose deviation vector measured by the laser tracker at each configuration point (i.e., the specific element column in the pose error measurement data matrix mentioned above). Representative at the At each configuration point, based on the established comprehensive error model (i.e., including the above...), , as well as The predicted pose deviation vector is calculated using the mapping equation of the Jacobian matrix. This represents the square of the second norm of a vector, which measures the square of the Euclidean geometric distance between the actual measurement deviation and the model prediction deviation in Cartesian space.

[0028] Furthermore, using the constructed pose error measurement data matrix as the algorithm input, the Levenberg-Marquardt (LM) nonlinear optimization algorithm is employed to optimize the objective function. The algorithm performs iterative dimensionality reduction to solve the problem. It continuously adjusts... , and The value of the objective function is such that the predicted pose deviation vector continuously approximates the actual pose deviation vector. When the objective function value converges to its minimum, the algorithm's iteration terminates, and the output parameters ( , and This refers to the inherent cumulative error value of the robot's installation, which is used to reproduce the robot's static error characteristics in the digital twin space.

[0029] Furthermore, when industrial robots perform high-speed trajectory compensation or are subjected to external dynamic loads (such as cutting and grinding), simply compensating for static geometric errors is far from sufficient. This is because inertial forces, centrifugal forces, and nonlinear frictional forces within the joint gearbox can induce elastic deformation and tracking lag in the robotic arm. Therefore, it is essential to identify the robot's dynamic parameters.

[0030] Specifically, based on the Euler-Lagrange equations of rigid body dynamics, a dynamic model of the industrial robot is established and reconstructed into a linear form with respect to the minimum set of inertial parameters: ; in, This represents the actual driving torque vector output by the motors at each joint. These represent the robot's joint angles, angular velocities, and angular acceleration vectors, respectively. The inertial mass matrix representing the robot reflects the resistance of the robot's spatial mass distribution to accelerated motion. The matrix representing the Coriolis force and centrifugal force embodies the coupled nonlinear forces during high-speed linkage of multiple joints; Represents the gravitational load vector; Representing the nonlinear frictional torque of the joint, in this embodiment, the Stribeck friction model, which includes Coulomb friction and viscous friction, is preferably used to characterize the error of the robot during the low-speed start-stop phase. This is the dynamic observation matrix; The vector of the robot's fundamental inertial parameters to be identified (including the mass, centroid coordinates, independent components of the inertial tensor, and friction coefficient of each link).

[0031] Furthermore, a set of periodic excitation trajectories composed of a finite number of Fourier series is designed to ensure that the velocities and accelerations of each joint of the robot fully cover their physical limits. During the operation of the excitation trajectory, encoder data (i.e., joint angles) and current loop data (i.e., mapping the actual driving torque vector to the actual torque) at the underlying servo driver are collected synchronously at a millisecond sampling rate.

[0032] Specifically, angular velocity and angular acceleration vectors are obtained by differential filtering of joint angles. The joint angles, angular velocity, angular acceleration vectors and actual driving torque vectors in the time series are used as input data pools. The overdetermined equations are solved by weighted least squares (WLS) to extract the accurate set of dynamic parameters, namely the basic inertial parameter set vector mentioned above.

[0033] Furthermore, the set of all kinematic geometric error parameters obtained from the solution is used as the bias constraint of the underlying topology, and the set vector of basic inertial parameters obtained from the solution is used as the physical property constraint. Both are then imported into the dynamic simulation engine.

[0034] It should be noted that the geometric error parameter reconstructs the skeletal distortion of the digital twin, determining the initial cumulative deviation of the end-effector trajectory in space (i.e., static characteristics); while the dynamic parameter endows the digital twin with realistic muscle mass and joint friction, determining the secondary dynamic tracking error (i.e., dynamic response characteristics) caused by the applied compensation velocity or acceleration during the follow-up compensation process. Coupled with these two parameters, the digital twin model, when inputted with any standard trajectory command in its virtual environment, can output a dynamically deviated trajectory with cumulative error that is highly consistent with that of the real robot, thus completing the construction of a high-fidelity virtual training environment.

[0035] S2. In the virtual environment constructed by the digital twin model, a compensation policy network is trained using a reinforcement learning algorithm to obtain the ability to generate pose compensation based on the robot's real-time state and error information.

[0036] It should be noted that traditional purely analytical compensation algorithms are prone to failure when faced with unmodeled dynamics such as temperature drift and sudden load changes in real industrial environments. Furthermore, training neural networks directly on physical entities faces extremely high time costs and the risk of collision damage. Therefore, this invention utilizes a constructed digital twin model as a training ground, allowing the compensation strategy network to undergo millions of trials in virtual space, thereby learning a robust compensation strategy.

[0037] Furthermore, in order to eliminate the virtual-real differences between the virtual environment and the real physical world, and to endow the compensation strategy network with the ability to generalize to unknown states and resist interference, this embodiment introduces domain randomization processing technology into the digital twin model.

[0038] Specifically, before each reinforcement learning training round, instead of using the fixed parameters identified in S1, random perturbations with Gaussian noise are introduced to sample the geometric error parameters, dynamic parameters, and pose deviations of the external workpiece. The mathematical expression is as follows: ; in, Represents the system parameters actually used by the digital twin model in the current training round; The reference parameters (including geometric error parameters and dynamic parameters) identified in S1 are represented. This indicates that the mean is 0 and the variance is 0. Gaussian distribution; The value is set based on the maximum physical drift range of each parameter in the actual industrial environment (e.g., the range of slight changes in the length of the connecting rod caused by the thermal expansion of the motor, and the range of positional tolerances for clamping different batches of workpieces).

[0039] It should be noted that by introducing domain randomization, the policy network is forced to face a polymorphic environment with massive parameters undergoing small changes during training. This makes the policy network no longer just memorize the error patterns of a robot in a specific state, but learn to extract the underlying physical laws that cause errors. As a result, when it is deployed to a robot that has undergone wear or thermal deformation in the future, it can still maintain high compensation accuracy.

[0040] Furthermore, a Markov Decision Process (MDP) is constructed, and the state, action, and reward function are defined. It should be noted that the core of reinforcement learning algorithms lies in the interaction between the agent and the environment. Therefore, this embodiment designs the MDP as follows for a scenario involving dynamic compensation for accumulated errors:

[0041] For the state space, a comprehensive state information vector is defined for the reinforcement learning algorithm. To enable the policy network to possess feedforward prediction capabilities that overcome robot dynamic hysteresis (i.e., the time difference between the motor receiving a command and the actual movement), the integrated state information vector of this invention not only includes the current state but also introduces a predictive window. The mathematical expression for this integrated state information vector is: ; in, For the robot at the current moment The expected target Cartesian pose; For the future The predicted trajectory points at each time step (in this embodiment) The introduction of this parameter enables the policy network to see upcoming sharp turns or high-speed straight sections, thereby calculating the pre-compensation amount to overcome inertial forces in advance. and These represent the robot's current joint angles and angular velocities, respectively, characterizing the robot's current body dynamic potential state. This is an online estimate of external dynamic errors (such as end load fluctuations or workpiece clamping offsets).

[0042] For the action space, the output of the policy network is an action vector. The motion vector is defined as the six-dimensional pose compensation amount that needs to be superimposed on the original trajectory during the current control cycle. : ; in, These represent the translational position compensation amounts of the robot's end effector along the X, Y, and Z axes in the base coordinate system (or world coordinate system), respectively. These represent the rotational attitude compensation amounts of the robot's end effector around the X, Y, and Z axes, respectively (usually expressed as fine-tuning values ​​for roll, pitch, and yaw angles).

[0043] It is important to emphasize that, to ensure the stability and safe operation of the robot servo system, the motion values ​​in the above six dimensions, after being output by the policy network, will be limited to the robot's physical safety threshold range (e.g., translation is limited to...). Within, rotation is limited to within).

[0044] To ensure the reward function can accurately compensate for errors without causing high-frequency oscillations that could damage the servo motor, this invention designs it as a composite reward function that includes a position tracking term and a motion smoothing term. ; in, The immediate reward value at the current time step; This refers to the actual end-effector pose of the robot in a digital twin environment after motion vector compensation. Weights are used to track error penalties; Weighting is used to compensate for jitter.

[0045] It should be explained that this reward function is mainly used to encourage the actual pose to approach the target pose infinitely (the first penalty tends to 0), while strongly penalizing drastic changes in compensation instructions within two adjacent control cycles (the second term), thereby forcing the policy network to output a continuous and accurate dynamic compensation trajectory.

[0046] Furthermore, in this embodiment, the Proximal Policy Optimization (PPO) algorithm, a reinforcement learning algorithm that performs well in the field of continuous motion control, is preferably used. Specifically, this algorithm internally comprises two cooperating neural networks:

[0047] One is the Actor network (i.e., the compensation policy network): responsible for outputting a compensation action vector based on the current state information vector. Its structure consists of: an input layer (with the same dimension as the state space), three fully connected hidden layers (256, 256, and 128 nodes respectively, using the ReLU activation function for non-linear feature extraction), and an output layer (6 dimensions, using the Tanh activation function to map the output to...). Then linearly scaled to the range of the aforementioned physical security threshold.

[0048] The other is the Critic network (i.e., the value evaluation network): it is responsible for evaluating the quality of the current state information vector (i.e., the long-term expected return), guiding the parameter updates of the Actor network. Its hidden layer structure is the same as that of the Actor network, except that the output layer of the Critic network is a single scalar neuron.

[0049] Furthermore, in the simulation engine of the digital twin, the typical processing trajectory of an industrial robot is continuously run at a certain control frequency (e.g., 100Hz):

[0050] (i) at each time step The Actor network observes the state information vector of the virtual environment, outputs compensated action vectors, and applies them to the underlying controller of the digital twin robot. After dynamic calculation, the virtual environment evolves to the next state information vector. It also provides a reward function. The complete interaction data set... Store in the experience replay pool.

[0051] (ii) Once the replay pool has collected a specified number of trajectory data (e.g., up to 2048 steps), the generalized advantage estimate (GAE) is calculated using the Critic network to assess how much better the current compensation action is than the average level.

[0052] (iii) Using the above data, maximize the cutoff objective function of the PPO algorithm using the gradient ascent method, and adjust the weight parameters of the Actor network. After updating, we get: ; in, Output the ratio of the probability of actions for the new and old strategies; The dominant function; To truncate the hyperparameter (usually set to 0.2).

[0053] It should be noted that this update method limits the magnitude of each policy network update through the clip function, avoiding catastrophic forgetting or policy collapse due to a single erroneous data sample, and ensuring the monotonically increasing nature of the learning process.

[0054] (iv) Repeat steps (i) to (iii) above, continuously clearing and resampling the replay pool data. As the number of training rounds increases, the accumulated reward value will gradually rise and tend to stabilize. At this point, it indicates that the compensation policy network has fully converged and has the ability to output the optimal pose compensation amount in real time according to complex dynamic conditions. Training terminates and the policy network model is saved.

[0055] S3. Deploy the trained compensation strategy network into the control loop. During online operation, based on the real-time acquired robot state and external dynamic error information, generate real-time pose compensation through the compensation strategy network, and use the compensation to correct the robot's target trajectory command.

[0056] It should be noted that, through massive trial and error and domain randomization training in a digital twin environment, the compensation policy network (i.e., the Actor network) has internalized the robot's nonlinear dynamics and error accumulation characteristics. Based on this, the main purpose of this step is to transfer the virtual-to-real world connection, that is, to remove the Critic network and exploration noise from the training phase, freeze the weight parameters of the Actor network, and embed it as a nonlinear feedforward compensator into the robot's underlying control loop, thereby achieving millisecond-level online dynamic correction.

[0057] Furthermore, during the virtual training phase of S2, the compensation policy network already possesses the ability to process state inputs. (i.e., the ability to estimate external dynamic errors online). When this network is deployed in real industrial settings (e.g., aerospace panel milling or automotive body welding), in addition to the inherent cumulative errors of the robot itself, the actual spatial position of the workpiece to be processed often deviates from the theoretically programmed position due to minor deviations in AGV docking, fixture thermal expansion, or machining tolerances from the previous process. Therefore, it is essential to obtain this information in real time through real physical means. This is to activate the corresponding compensation mechanism in the policy network.

[0058] Specifically, an external sensor (preferably a 3D vision camera in this embodiment) is installed at the robot's end flange or workstation fixed reference point. Before or during task execution, the 3D vision camera scans the feature points of the workpiece to obtain the actual pose matrix of the workpiece to be processed, denoted as... .

[0059] Furthermore, a theoretical pose matrix of the workpiece in the world coordinate system is preset within the robot's underlying controller, denoted as... By performing the inverse operation of the homogeneous transformation matrix, the six-dimensional Cartesian space pose deviation (i.e., ...) relative to the theoretical pose is calculated. Its mathematical expression can be converted into vector form: ; in, This represents the actual homogeneous transformation matrix of the workpiece measured by the sensor and calibrated to the robot's base coordinate system. Dimensions); This represents the pre-defined homogeneous transformation matrix of the workpiece theory during offline programming. The matrix representing the relative rigid body displacement transformation of the workpiece; This represents the mapping function that solves the homogeneous transformation matrix into Euler angles and translation vectors.

[0060] Furthermore, within each interpolation control cycle of the robot performing the processing task (e.g., the controller is set to a cycle of 4ms), it is necessary to synchronously collect internal and external states to construct the input tensor fed into the neural network.

[0061] Specifically, due to the inconsistent sampling frequencies of multiple sensors in industrial settings, this solution employs an asynchronous data fusion mechanism. For high-frequency data, the servo driver reads the feedback values ​​from each joint encoder in real time and performs differential calculations to obtain the current actual joint angle and angular velocity at high frequency during the control cycle. Simultaneously, it acquires the nominal target pose (i.e., ...) issued from the controller buffer. ) and the planned future Predicted trajectory points Meanwhile, for low-frequency data, namely the external dynamic error information calculated by the external 3D vision camera (whose update period is typically 33ms~100ms), a zero-order hold (ZOH) mechanism is used on the controller's internal data bus. That is, until the next frame of error data from the vision camera arrives, the external dynamic error information sampled in the previous sampling is kept unchanged in each high-frequency control cycle.

[0062] Furthermore, by concatenating the aligned high-frequency real-world data with the zero-order preserved external dynamic error information, an online integrated state information vector with the same dimension as during S2 training can be reconstructed. Subsequently, this integrated state information vector is fed into the compensation policy network with frozen parameters for forward propagation inference, yielding: ; in, This represents the forward computation process of the compensation policy network, which is obtained by a series of matrix multiplications and nonlinear activation functions (ReLU, Tanh). This represents the optimal network weights and bias parameter matrix that has been trained, converged, and solidified in S2. This is the real-time six-dimensional pose compensation output by the policy network.

[0063] It should be noted that, compared with traditional complex numerical iterative algorithms such as Jacobian matrix inversion, the forward inference process of the trained neural network after the above processing requires only extremely small computational power consumption (usually completed in the sub-millisecond level). Therefore, the algorithm can meet the stringent real-time requirements of industrial robots for the underlying controller and will not cause frame loss or oscillation in the control system communication due to computation timeout.

[0064] Furthermore, after obtaining the real-time six-dimensional pose compensation, since this compensation is a small offset defined in Cartesian space, it cannot be directly sent to the joint motors. It must be physically combined and inversely calculated with the original nominal target trajectory.

[0065] Specifically, to ensure that the non-commutative features of spatial rotation are not erroneous, the six-dimensional nominal target pose (i.e., ) and the compensation amount in six dimensions (i.e., Each of these is converted into a corresponding homogeneous transformation matrix, denoted as . and Then, a left multiplication (or right multiplication, depending on the actual reference coordinate system definition) composite operation is performed to generate the corrected target pose matrix. : ; Next, the robot controller's inverse kinematics (IK) solver is invoked to convert the corrected Cartesian space pose into control commands in the joint space.

[0066] Finally, the sequence of control commands containing the target angles of each joint is sent to the robot's underlying servo drive system.

[0067] It should be noted that in this step, the compensation output by the policy network is essentially a reverse-induced quantity. Due to the robot's cumulative installation error and dynamic hysteresis, directly following... Sending a command will cause the robot's end effector to actually reach a deviated position; and after incorporating compensation, it is essentially tricking the robot into moving towards a pseudo-target point that has been offset (i.e., The robot moves along its own physical imperfections and mechanical inertia. The actual trajectory of its end effector perfectly offsets the original combined error, thus accurately landing on the originally expected nominal target (i.e., ...) in the real environment. This enables trajectory tracking under dynamic operating conditions.

[0068] S4. During the robot's compensation task, collect robot operation data and task effect data, and use the operation data to perform offline optimization and iteration of the compensation strategy network.

[0069] It should be noted that after months or even years of high-intensity operation, industrial robots inevitably experience mechanical wear, grease aging, and minor plastic deformation of their joint gearboxes and structural components. This means that the discrepancy between the digital twin model established in S1 and the real physical world may widen over time. If the compensation policy network (Actor network) remains in the fixed state after training in S2, its compensation accuracy will gradually decrease. To address this, this embodiment proposes a policy self-evolution mechanism based on offline reinforcement learning, enabling the policy network to autonomously learn new error patterns from historical operating data without engaging in risky random trial and error on physical entities.

[0070] Furthermore, in the background where the robot is actually performing processing tasks (online operation phase), data is continuously recorded on the disk at a set sampling frequency (e.g., 100Hz) without interfering with the underlying real-time control loop.

[0071] Specifically, the collected data mainly includes three parts:

[0072] The first part is the state sequence. : That is, the online integrated state information vector of the robot during actual operation in S3, which includes the nominal target pose, the predicted trajectory points, the actual joint kinematic state, and the external dynamic error information;

[0073] The second part is the action sequence. : That is, the real-time six-dimensional pose compensation amount that is actually output by the compensation strategy network in S3 and has been applied to the servo system. );

[0074] The second part contains the final mission execution accuracy data. For scenarios with real-time high-precision external tracking measurement (laser tracker), the step tracking deviation between the actual machining trajectory and the theoretical trajectory in each control cycle is directly obtained; for scenarios where real-time measurement is not possible, after the machining of a single workpiece is completed, the global contour deviation is obtained using an online inspection device (e.g., an online coordinate measuring machine, CMM), and the global deviation is weighted and distributed to each time step using a reward reshaping algorithm based on trajectory curvature and acceleration allocation, and converted into an equivalent single-step tracking deviation.

[0075] Furthermore, in order to transform task execution accuracy into a reward signal that reinforcement learning can understand, this invention defines an offline real reward function. ,in, This represents the positive gain coefficient. Subsequently, the collected discrete data are aligned along the time axis and organized into transition tuples of a Markov decision process, thus constructing a massive offline training dataset. : ; in, This represents the total number of historical data samples collected.

[0076] It should be noted that this offline training dataset originates from a real physical processing site, implicitly recording the nonlinear dynamic characteristics of the robot in its latest aging state. Compared to the simulation data virtually generated in stage S2, this real data is the most valuable data asset reflecting the current health status and residual errors of the equipment.

[0077] Furthermore, after obtaining the offline training dataset, the computing power deployed on edge computing nodes or cloud servers is used to safely fine-tune the parameters of the compensation policy network using an offline reinforcement learning algorithm with behavioral cloning (BC) constraints (such as the TD3+BC algorithm).

[0078] It's important to explain that the reason this invention emphasizes offline operation and safety is because traditional online reinforcement learning requires the agent to interact with the environment and explore (i.e., attempt unknown compensatory actions). If this random exploration is performed on a robot, it's highly likely that the output will exceed the compensation limit, causing the robotic arm to collide with the workpiece or damage the motor, thus disrupting factory production. Offline reinforcement learning, however, only needs to rely on offline training datasets for parameter optimization, thus completely eliminating the risk of physical trial and error.

[0079] Specifically, the offline optimization iteration training process includes:

[0080] First, the network architecture for offline training is defined. The Actor network (compensation policy network) directly inherits the three-layer fully connected network structure (256→256→128 nodes) trained in stage S2. Furthermore, to address the Q-value (action value) overestimation problem that easily occurs in offline reinforcement learning, the Critic network (value evaluation network) in this embodiment adopts a double-Q network architecture, that is, constructing two fully connected hidden layer networks with the same structure as the Actor, denoted as... and The two have different initial weights. and .

[0081] Next, the value evaluation network is updated by randomly sampling batches of data from the offline training dataset. The dual-Q network is trained by minimizing the temporal difference (TD) error, enabling it to accurately assess the long-term real benefit of taking an action under the current machine aging state. When calculating the target Q-value, the minimum value of the dual network is taken, and the loss function is: ; in, ; This is a discount factor, representing the degree of concern regarding future errors; The parameters for the target Critic network are used to truncate gradients for stable training. The target Actor network generates actions for the next state, and truncated smooth Gaussian noise is added to them to avoid overfitting.

[0082] Finally, the parameters of the compensation policy network (Actor network) Safety fine-tuning is performed. To prevent network overfitting and to infer dangerous high-scoring actions that do not exist in the dataset (i.e., out-of-distribution action extrapolation errors), this invention introduces a Behavior Cloning (BC) regularization term to construct the objective function of the Actor network: ; in, The first term is the number of samples in a single training batch; the second term aims to maximize the Q value under the current policy (i.e., to find a better compensation amount that can improve processing accuracy); the third term is the behavior cloning constraint term, which is to calculate the mean square error between the new output action of the Actor network and the historical safe actions in the dataset. This is a balancing coefficient used to dynamically adjust the weight ratio between exploring better compensation and conservatively replicating historical actions.

[0083] It should be noted that, through the design of the loss function and objective function described above, the policy network can learn how to eliminate residual errors while its output actions are forcibly pulled back to the vicinity of a known and safe historical operation distribution. This ensures that after fine-tuning and updating, the policy network can both adapt to the degradation of the robot's mechanical performance and never produce abrupt jump commands, achieving extremely smooth and safe policy self-evolution.

[0084] Furthermore, although the aforementioned offline fine-tuning mechanism can cope with a certain range of mechanical performance degradation, when the robot experiences severe physical changes (such as slight bending of the links due to an accidental collision, or replacement of the end effector with a different specification), the underlying physical topology undergoes a qualitative change, and relying solely on parameter fine-tuning of the policy network is clearly insufficient to achieve convergence. Therefore, this embodiment also provides a calibration step, which is mainly used to achieve control throughout the robot's entire lifecycle.

[0085] Specifically, the effectiveness of offline optimization iterations is evaluated periodically (e.g., weekly or monthly) in the background. The average trajectory tracking error (i.e., the average absolute profile deviation of the most recent processing batch) is extracted from the feedback of the online inspection equipment. A preset threshold based on the process tolerance limit (e.g., 0.05 mm) is established. The extracted average trajectory tracking error is compared with this preset threshold. If the extracted average trajectory tracking error does not exceed the preset threshold, it indicates that the current offline strategy fine-tuning is sufficient to cope with mechanical attenuation, and the above processes S3 and S4 will continue. If it is determined that the extracted average trajectory tracking error exceeds the preset threshold, it indicates that the underlying architectural deviation between the physical world and the digital twin model has seriously exceeded the limit. At this time, an alarm signal will be triggered, forcibly interrupting the current strategy fine-tuning process and automatically calling S1. The robot's geometric error parameters and dynamic parameters are re-identified and calibrated extensively using a laser tracker, thereby completely updating the digital twin model and resetting the performance baseline of the entire system.

[0086] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamic compensation of cumulative installation error trajectory for industrial robots, characterized in that, include: Construct a digital twin model corresponding to the physical industrial robot, the model integrating the robot's geometric error parameters and dynamic parameters; In the virtual environment constructed by the digital twin model, a compensation policy network is trained using a reinforcement learning algorithm to obtain the ability to generate pose compensation based on the robot's real-time state and error information. The trained compensation strategy network is deployed into the control loop. During online operation, based on the real-time acquired robot state and external dynamic error information, the compensation strategy network generates a real-time pose compensation amount, and the compensation amount is used to correct the robot's target trajectory command. During the robot's performance of the compensation task, robot operation data and task effect data are collected, and the operation data is used to perform offline optimization and iteration on the compensation strategy network.

2. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 1, characterized in that, Constructing the digital twin model includes: Identify the robot's full kinematic geometric error parameters and dynamic parameters, and integrate the geometric error parameters and dynamic parameters into the digital twin model so that the geometric error parameters and dynamic parameters reproduce the robot's static error characteristics and dynamic response characteristics.

3. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 2, characterized in that, Identifying the full kinematic geometric error parameters includes: A comprehensive error model is established to uniformly describe the errors of the robot's joint link parameters, the base mounting pose error, and the tool center point pose error. The end-effector pose data of the robot under multiple different configurations are obtained through external measurement equipment, and the parameter values ​​of the comprehensive error model are solved.

4. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 1, characterized in that, A compensation policy network is trained using a reinforcement learning algorithm, including: In the digital twin model, the geometric error parameters, dynamic parameters, and pose deviations of the external workpiece are subjected to domain randomization to generate diverse training scenarios containing uncertainties.

5. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 1 or 4, characterized in that, When training the compensation policy network, a state information is defined for the reinforcement learning algorithm, the state information including: The robot's current target trajectory point and predicted trajectory points for multiple future time steps, the robot's current joint angles and angular velocities, and online estimates of external dynamic errors.

6. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 1, characterized in that, Before generating the real-time pose compensation, the following is also included: The actual pose of the workpiece to be processed is perceived in real time using external sensors, and the six-dimensional pose deviation of the actual pose relative to the theoretical pose is calculated. The deviation is used as part of the external dynamic error information.

7. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 1 or 6, characterized in that, The step of generating real-time pose compensation and using the compensation to correct the robot's target trajectory includes: In each control cycle, the current state and nominal target pose of the robot are acquired. The current state and nominal target pose, together with the external dynamic error information, are used as inputs. A six-dimensional pose compensation amount is calculated through the compensation strategy network. The compensation amount is then combined with the nominal target pose to generate the final motion control command issued to the robot servo system.

8. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 1, characterized in that, The robot's operational data and task performance data are collected, including: The robot's state sequence during the task, the action sequence output by the compensation strategy network, and the final task execution accuracy data evaluated by online detection equipment; The state sequence, the action sequence, and the accuracy data are organized into an offline training dataset.

9. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 8, characterized in that, The compensation strategy network is optimized and iterated offline, including: An offline reinforcement learning algorithm is used to safely update and fine-tune the parameters in the compensation policy network using the offline training dataset without direct interaction with the robot environment.

10. The method for dynamic compensation of cumulative installation error trajectory for industrial robots as described in claim 1, characterized in that, The method further includes a calibration step, which includes: The effects of the offline optimization iterations and the performance deviation between the physical world and the digital twin model are analyzed periodically. When the deviation exceeds a preset threshold, the geometric error parameters and dynamic parameters of the digital twin model are re-identified and recalibrated.

Images