Kinematic parameter adaptive identification system for mechanical arm based on binocular vision guidance

Abstract

The application relates to the technical field of mechanical arms and discloses a mechanical arm kinematic parameter self-adaptive identification system based on binocular vision guidance, which comprises the following modules: a modeling module obtains joint angles and temperatures to construct an augmented parameter set, calculates an end theoretical pose, a joint Jacobian matrix and a task Jacobian matrix; a vision module obtains measurement parameters through a binocular camera and calculates a visual confidence matrix; a monitoring module constructs a Hessian matrix based on the Jacobian matrix and the confidence matrix and sends a micro-activation instruction when a condition number exceeds a standard; a control module generates joint micro-offsets in combination with a tolerance band and triggers the binocular camera to collect actual measurement poses according to torque derivatives; and an updating module updates the augmented parameter set by using the residual errors of the actual measurement poses and the theoretical poses in combination with the confidence matrix. The application utilizes binocular vision guidance and stable torque triggering, overcomes the coupling of structural thermal deformation and error parameters, realizes high-precision mechanical arm kinematic parameter self-adaptive identification, and improves calculation stability.

Description

Technical Field

[0001] This invention relates to the field of robotic arm technology, specifically to an adaptive identification system for the kinematic parameters of a robotic arm based on binocular vision guidance. Background Technology

[0002] Multi-joint robotic arms are widely used in precision assembly and other tasks, and the spatial positioning accuracy of their end effectors directly determines the quality of the work. Due to manufacturing tolerances and assembly clearances, there are deviations between the actual parameters and theoretical values ​​of the robotic arm. Therefore, adaptive identification of the robotic arm's kinematic parameters is required to calibrate the model and compensate for pose errors.

[0003] Binocular vision-based non-contact measurement techniques are commonly used to obtain the three-dimensional coordinates of the end effector of a robotic arm. Traditional methods typically involve using a binocular camera to acquire images of the end effector target after the robotic arm has docked, calculating the actual pose, and then using a basic optimization algorithm to solve for the residual between the measured pose and the theoretical pose, thereby updating the basic geometric parameters of the robotic arm.

[0004] Existing identification methods primarily rely on static geometric models, failing to consider the thermal expansion and deformation of connecting rods caused by equipment heating during continuous operation, leading to a decrease in model accuracy after prolonged use. Secondly, under specific joint configurations, the impact of parameter errors in adjacent connecting rods on end-effector pose is prone to overlap. This parameter coupling can cause ill-conditioned solutions to the identification equations, reducing computational stability. Furthermore, traditional systems often use fixed-time delays to trigger camera exposure, making it difficult to accurately reflect the steady-state torque of the underlying joint servo actuators. Directly acquiring images when there are minor residual vibrations in the mechanical structure introduces non-negligible dynamic observation errors.

[0005] Therefore, this invention proposes an adaptive identification system for the kinematic parameters of a robotic arm based on binocular vision guidance to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an adaptive identification system for the kinematic parameters of a robotic arm based on binocular vision guidance. This system solves the problems of decreased model accuracy due to failure to consider thermal deformation of the linkage, ill-conditioned solution matrix caused by high parameter coupling, and dynamic observation errors easily introduced by relying on fixed-delay camera triggering.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an adaptive identification system for the kinematic parameters of a robotic arm based on binocular vision guidance, comprising:

[0008] The modeling module is used to obtain the joint angles and temperature of the robotic arm, construct an augmented parameter set containing geometric errors and thermal expansion coefficients, and calculate the theoretical pose of the end effector, the joint Jacobian matrix, and the task Jacobian matrix.

[0009] The vision module is used to acquire measurement parameters through a binocular camera and calculate the inverse of the covariance matrix as the visual confidence matrix.

[0010] The monitoring module is used to construct a Hessian matrix based on the joint Jacobian matrix and the visual confidence matrix, and to issue a micro-excitation command when the condition number of the Hessian matrix is ​​greater than the parameter coupling critical threshold.

[0011] The control module is used to receive the micro-excitation command, combine the task Jacobian matrix and tolerance band to generate joint micro-offset, and trigger the binocular camera to acquire the measured pose according to the torque derivative;

[0012] The update module is used to update the augmented parameter set by combining the residual between the measured pose and the theoretical pose of the end effector with the visual confidence matrix.

[0013] Preferably, the modeling module extracts real-time sequence data from the temperature sensor integrated in the absolute encoder as the temperature, and constructs the augmented parameter set containing the geometric error and the coefficient of thermal expansion based on the temperature and the initial parameters of the robotic arm linkage; the modeling module calculates the theoretical end-effector pose of the robotic arm based on the forward kinematic mapping function, using the joint angles of the robotic arm, the temperature, and the augmented parameter set.

[0014] Preferably, the modeling module obtains the joint Jacobian matrix by taking the partial derivative of the end-effector theoretical pose with respect to the augmented parameter set, and the modeling module obtains the task Jacobian matrix by taking the partial derivative of the end-effector theoretical pose with respect to the robot arm joint angles.

[0015] Preferably, the measurement parameters include the physical baseline length of the binocular camera, the spatial depth of the target, the sum of squared reprojection errors, and the local gradient of the radial distortion field of the image plane; the vision module calculates the covariance matrix of the three-dimensional spatial coordinates based on the error propagation law, combined with the pixel observation variance on the two-dimensional image plane and the visual observation Jacobian matrix, and uses the singular value decomposition algorithm to obtain the inverse matrix of the covariance matrix and uses the obtained inverse matrix as the visual confidence matrix.

[0016] Preferably, the monitoring module multiplies the transpose of the joint Jacobian matrix, the visual confidence matrix, and the joint Jacobian matrix to obtain the Hessian matrix; the monitoring module calculates the first eigenvalue and the last non-zero eigenvalue in the Hessian matrix after sorting them in descending order of numerical value, and calculates the ratio of the first eigenvalue to the last eigenvalue to obtain the condition number.

[0017] Preferably, the control module extracts the eigenvector corresponding to the last eigenvalue of the Hessian matrix as the ideal excitation direction, and constructs a quadratic programming problem by combining the task Jacobian matrix and the tolerance band; the control module generates the joint micro-bias by solving the quadratic programming problem, and sends the joint micro-bias to the underlying controller of the robotic arm.

[0018] Preferably, during the process of the robotic arm performing the joint micro-biasing, the control module collects the real-time feedback torque of all joint servo drives, calculates the first-order time derivative of the real-time feedback torque to obtain the torque derivative; when the absolute value of the torque derivative of all joints is less than the dead zone threshold, the control module sends a synchronous trigger exposure signal to the binocular camera, and the binocular camera collects the measured pose of the robotic arm according to the synchronous trigger exposure signal.

[0019] Preferably, the update module calculates the residual by subtracting the theoretical end pose from the measured pose; the update module extracts the visual confidence matrix corresponding to all sampling points, and arranges the visual confidence matrix along the diagonal to construct a global heteroscedasticity block diagonal weight matrix.

[0020] Preferably, the update module vertically concatenates the joint Jacobian matrices of all sampling points along the vertical axis to form a global-scale joint Jacobian matrix, and vertically concatenates the residuals of all sampling points along the vertical axis to form a global residual column vector; the update module combines the transpose of the global-scale joint Jacobian matrix, the global heteroscedasticity block diagonal weight matrix, and the global-scale joint Jacobian matrix to calculate the Hessian matrix used for global optimization.

[0021] Preferably, the update module constructs a regularized normal equation based on the Hessian matrix used for global optimization and solves for the unknown vector to obtain the update step size; the update module adds the update step size to the historical parameters to complete the update of the augmented parameter set; when the infinite norm of the update step size is less than the preset convergence tolerance, the update module determines that the iteration has converged.

[0022] This invention provides an adaptive kinematic parameter identification system for a robotic arm based on binocular vision guidance. It offers the following advantages:

[0023] 1. This invention acquires joint angle and temperature data of a robotic arm to construct an augmented parameter set including geometric errors and the coefficient of thermal expansion, providing a more accurate physical model foundation for adaptive identification of the robotic arm's kinematic parameters. Based on this, a data loop is formed by utilizing theoretical calculation data and subsequent experimental steps guided by binocular vision. This mechanism effectively compensates for the minute structural deformations caused by temperature changes during long-term operation, solves the problem of large errors in traditional single geometric models, and improves the accuracy of basic modeling and the system's adaptability to environmental fluctuations.

[0024] 2. This invention uses the inverse of the covariance matrix as the confidence level and combines it with the Jacobian matrix to construct a Hessian matrix to assess the degree of parameter coupling. When the condition number exceeds the limit, it actively generates micro-bias instructions to optimize the measurement space based on binocular vision guidance. This local micro-excitation mechanism can effectively separate highly coupled error terms, making the adaptive identification process of robotic arm kinematic parameters more robust to noise. It solves the problem that the identification equation is prone to getting trapped in ill-conditioned matrices under certain configurations, and significantly improves the numerical stability of the parameter solution algorithm.

[0025] 3. This invention precisely triggers the binocular camera to acquire measured poses by monitoring the derivative of the underlying joint servo torque, ensuring a high degree of synchronization between binocular vision-guided image acquisition and the device's physical steady-state docking. Subsequently, a global heteroscedasticity weight matrix is ​​constructed using the visual confidence matrix, and the augmented parameter set is globally updated using the measured residuals. This iterative mechanism based on steady-state physical triggering and confidence weighting ensures the closed-loop convergence of the robotic arm's kinematic parameter adaptive identification under complex working conditions, effectively improving the reliability of overall pose error compensation. Attached Figure Description

[0026] Figure 1 This is a diagram illustrating the architecture of the binocular vision-guided adaptive kinematic parameter identification system for a robotic arm according to the present invention.

[0027] Figure 2 This is a flowchart of the adaptive identification method for kinematic parameters of a robotic arm based on binocular vision guidance according to the present invention.

[0028] Figure 3 This is a flowchart illustrating the micro-stimulus execution and synchronous triggering exposure of the present invention;

[0029] Figure 4 This is a graph showing the condition number monitoring and micro-excitation triggering of the present invention;

[0030] Figure 5 This is a global residual convergence curve diagram of the present invention.

[0031] Among them, 100 is the modeling module; 200 is the vision module; 300 is the monitoring module; 400 is the control module; and 500 is the update module. Detailed Implementation

[0032] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] See attached document Figure 1 The present invention provides an adaptive identification system for kinematic parameters of a robotic arm based on binocular vision guidance, comprising: a modeling module 100, a vision module 200, a monitoring module 300, a control module 400, and an update module 500.

[0034] The modeling module 100 acquires the current joint angles of the robotic arm and the temperature of each joint. The modeling module 100 extracts real-time sequence data from the temperature sensor integrated in the absolute encoder as the temperature. The modeling module 100 constructs an augmented parameter set containing geometric errors and thermal expansion coefficients based on the acquired temperature and the initial parameters of the robotic arm links. The modeling module 100 calculates the theoretical end-effector pose of the robotic arm based on the forward kinematic mapping function, using joint angles, temperature, and the augmented parameter set. The modeling module 100 obtains the joint Jacobian matrix by taking the partial derivative of the theoretical end-effector pose with respect to the augmented parameter set. The modeling module 100 obtains the task Jacobian matrix by taking the partial derivative of the theoretical end-effector pose with respect to the joint angles.

[0035] The vision module 200 is connected to a binocular camera. The vision module 200 acquires the measurement parameters of the target at the end of the robotic arm through the binocular camera. The measurement parameters include the physical baseline length of the binocular camera, the spatial depth of the target, the sum of squared reprojection errors, and the local gradient of the radial distortion field of the image plane. The vision module 200 calculates the covariance matrix of the three-dimensional spatial coordinates based on the error propagation law. The vision module 200 inverses the covariance matrix and uses the resulting inverse matrix as the visual confidence matrix.

[0036] The monitoring module 300 receives the joint Jacobian matrix output by the modeling module 100 and the visual confidence matrix output by the vision module 200. The monitoring module 300 multiplies the transpose of the joint Jacobian matrix, the visual confidence matrix and the joint Jacobian matrix to obtain the Hessian matrix. The monitoring module 300 calculates the first eigenvalue and the last non-zero eigenvalue in the Hessian matrix after sorting them in descending order of numerical value. The monitoring module 300 calculates the ratio of the first eigenvalue and the last eigenvalue to obtain the condition number. When the condition number is greater than the preset parameter coupling critical threshold, the monitoring module 300 determines that the linkage deformation has caused kinematic parameter coupling and generates a micro-excitation command.

[0037] The control module 400 receives micro-excitation commands from the monitoring module 300. It extracts the eigenvector corresponding to the last eigenvalue of the Hessian matrix as the ideal excitation direction. Combining the task Jacobian matrix output by the modeling module 100 and the preset process tolerance zone, the control module 400 constructs a quadratic programming problem. By solving the quadratic programming problem, the control module 400 generates joint micro-biases and sends them to the robot arm's underlying controller. During the execution of the joint micro-biases, the control module 400 collects the real-time feedback torque from all joint servo drives. It calculates the first-order time derivative of the real-time feedback torque to obtain the torque derivative. When the absolute value of the torque derivative of all joints is less than the dead-zone threshold, the control module 400 sends a synchronous trigger exposure signal to the binocular camera. The binocular camera then acquires the measured pose of the robot arm based on the synchronous trigger exposure signal.

[0038] The update module 500 receives the measured pose and the theoretical pose of the end effector. The update module 500 calculates the residual by subtracting the theoretical pose of the end effector from the measured pose. The update module 500 extracts the visual confidence matrix corresponding to all sampling points and arranges the visual confidence matrix along the diagonal to construct a global heteroscedasticity block diagonal weight matrix. The update module 500 combines the Hessian matrix, the joint Jacobian matrix, the global heteroscedasticity block diagonal weight matrix, and the residual, and uses a weighted nonlinear iterative algorithm to calculate the update step size of the augmented parameter set. The update module 500 accumulates the update step size into the historical parameters to complete the update of the augmented parameter set.

[0039] See attached document Figure 2 This invention provides an adaptive identification method for the kinematic parameters of a robotic arm based on binocular vision guidance, comprising the following steps:

[0040] S1. Obtain the joint angles and temperature of the robotic arm, construct an augmented parameter set containing geometric errors and thermal expansion coefficients, and calculate the theoretical pose of the end effector, the joint Jacobian matrix, and the task Jacobian matrix.

[0041] S2. Obtain measurement parameters through a binocular camera, and calculate the inverse of the covariance matrix as the visual confidence matrix;

[0042] S3. Construct a Hessian matrix based on the joint Jacobian matrix and the visual confidence matrix, and issue a micro-excitation command when the condition number of the Hessian matrix is ​​greater than the parameter coupling critical threshold.

[0043] S4. After receiving the micro-excitation command, combine the task Jacobian matrix and tolerance zone to generate joint micro-offset, and trigger the binocular camera to collect the measured pose according to the torque derivative.

[0044] S5. Update the augmented parameter set by using the residual between the measured pose and the theoretical pose of the end effector, combined with the visual confidence matrix.

[0045] The adaptive identification method for kinematic parameters of a robotic arm based on binocular vision guidance and the adaptive identification system for kinematic parameters of a robotic arm based on binocular vision guidance belong to the same inventive concept. Each logical module in the identification system is configured to execute the corresponding step in the identification method. Specifically, the modeling module 100 in the identification system is used to execute step S1; the vision module 200 is used to execute step S2; the monitoring module 300 is used to execute step S3; the control module 400 is used to execute step S4; and the update module 500 is used to execute step S5. The identification system relies on the sequential interactive operation of each module to achieve a computational closed loop from low-level data acquisition and dynamic constraint optimization to kinematic parameter closed-loop reconstruction.

[0046] To further clarify the implementation of each technical aspect of the present invention, the following will provide a detailed description of the implementation of each functional module involved above and its internal processing flow.

[0047] See attached document Figure 1 and Figure 2 In this embodiment, the modeling module 100 is executed through the following sub-steps:

[0048] S101, the modeling module 100 acquires the current joint angles of the robotic arm and the temperature of each joint. In actual industrial applications, each joint servo motor is generally equipped with an absolute encoder, and the circuit board of such absolute encoder or the stator winding of the servo motor has a built-in thermistor element. To achieve high-precision state perception, the modeling module 100 extracts the real-time sequence data of the temperature sensor integrated in the absolute encoder as the temperature. In specific implementation, the modeling module 100 reads the position register data of the absolute encoder through the underlying industrial communication bus and converts it into the current joint angle vector of the robotic arm. ,in The joint degrees of freedom of the robotic arm are defined. The feedback voltage of the thermistor is synchronously read and converted into a temperature sequence vector using the Steinhart-Hart equation. This is referred to here as a temperature sequence vector rather than a simple temperature vector, because the industrial robotic arm comprises multiple motion joints connected in series according to a spatial topology. This vector not only contains temperature values, but the arrangement of its internal elements maps to the topological sequence of the robotic arm's links from the base to the end effector. This voltage-to-temperature conversion is a conventional and well-known technique in the field of sensors. For reading data from the industrial control bus and parsing data from the underlying registers, those skilled in the art can use standard bus communication protocols. The underlying communication and encoding / decoding rules are well-known techniques in this field and will not be elaborated upon here.

[0049] Unifying the expression of static and dynamic variables is fundamental to achieving adaptive identification. In S102, the modeling module 100 constructs an augmented parameter set containing geometric errors and the coefficient of thermal expansion based on the acquired temperature and initial parameters of the robotic arm links. As a preferred approach, based on the Denavit-Hartenberg parametric model, the nominal link length, link offset, link torsion angle, and joint zero-position deviation are extracted as basic variables to form the geometric error vector. ,in This represents the number of parameters for geometric errors. In this process, the Denavit-Hartenberg parametric model is a standard existing modeling technique in the field of robot kinematics. Specifically, the extraction method is as follows: from the robot arm's factory 3D drawings or a pre-set machine tool parameter table, read the nominal values ​​of the four standard dimensional parameters corresponding to each link under this model; due to machining tolerances in actual manufacturing, the unknown deviations corresponding to these nominal values ​​are used as the basic variables to be identified, arranged sequentially from the robot arm base to the end effector, thus forming the geometric error vector. Based on the material properties of the connecting rod castings, the first-order linear thermal expansion coefficients of each connecting rod are extracted to form a thermal expansion coefficient vector. ,in This represents the number of coefficients of thermal expansion. The first-order linear coefficient of thermal expansion is a constant characterizing the linear elongation per unit length of a solid material as the temperature increases by one degree Celsius. In practice, based on the specific material properties of the connecting links of the robotic arm, such as cast aluminum or cast iron, the physical nominal value of the corresponding first-order linear coefficient of thermal expansion is obtained by consulting standard material property handbooks (i.e., authoritative reference books or national industrial standard documents that are well-known in the engineering field and specifically record the standard performance parameters of various basic solid materials, including their physical, mechanical, and thermodynamic properties). This value is used as the initial estimated variable, and the coefficients of thermal expansion are arranged longitudinally according to the link sequence to form a vector. The modeling module 100 will use the geometric error vector With the vector of thermal expansion coefficient Mathematically, they are concatenated into a unified column vector. That is, satisfying superscript This represents the transpose operation of a matrix or vector. Specifically, this concatenation process is a matrix stacking operation, where the matrix of dimension 1 is stacked in blocks. geometric error vector Placed in the upper half of the matrix, with dimension thermal expansion coefficient vector Placed in the lower half of the matrix, this combination generates a new multidimensional vertical array. This unified column vector That is, the augmented parameter set, whose total dimension is An augmented parameter set refers to a set of variables to be identified that integrates error variables characterizing static geometric deformation and coefficient variables characterizing dynamic thermodynamic deformation within the same mathematical space.

[0050] S103, After establishing a complete parameter set, it is necessary to further construct mapping relationships to predict the end-effector state. The modeling module 100 calculates the theoretical pose of the robotic arm's end-effector based on the forward kinematic mapping function, using joint angles, temperature, and the augmented parameter set. In this embodiment, the acquired temperature sequence vector... relative to ambient reference temperature The difference yields the temperature rise vector. Ambient reference temperature This refers to the initial ambient temperature of the robotic arm before cold start-up, when no thermal deformation has occurred. It is typically collected by an external ambient temperature sensor during system initialization. To ensure the validity of the dimensions in algebraic operations, the modeling module 100 pre-constructs a dimension-temperature sequence vector before performing the aforementioned difference operation. A completely consistent reference temperature vector, in which all element values ​​are uniformly assigned the ambient reference temperature. Then, the temperature rise vector is calculated by subtracting the reference temperature vector from the temperature sequence vector, element by element. The temperature rise vector is then... With augmented parameter set The coefficients of thermal expansion in the parameters are multiplied accordingly to calculate the real-time thermal deformation elongation of each link. This thermal deformation elongation is then used as a compensation value and added to the augmented parameter set. Based on the geometric error parameters, obtain the dynamic link parameters under the current thermal equilibrium state. Then, compare these dynamic link parameters with the currently acquired joint angle vector. Substituting into the homogeneous transformation matrix multiplication model, the output is a six-dimensional spatial vector containing three-dimensional translation coordinates and three-dimensional rotation Euler angles. The calculation formula is: ;

[0051] In the formula, This represents the positive kinematic mapping function that incorporates thermodynamic variables. The calculated output is a six-dimensional space vector. This refers to the theoretical pose of the end effector, which is the spatial position and attitude of the end effector in the base coordinate system, derived solely from the feedback values ​​of the internal encoder and the current mathematical mapping model without relying on actual measurements by external optical instruments.

[0052] S104. To quantify the impact of various errors on pose, a corresponding sensitivity matrix needs to be constructed. Modeling module 100 obtains the joint Jacobian matrix by taking the partial derivative of the theoretical end-effector pose with respect to the augmented parameter set. This is then applied to the aforementioned positive kinematic mapping function. The six spatial dimension variables of its output are respectively applied to the augmented parameter set. Each independent geometric error and thermal expansion coefficient in the equation is used to calculate the first-order partial derivative, forming a dimension of... matrix The calculation formula is: ;

[0053] In the formula, Represents the partial derivative differential operator; The differential variable represents the positive kinematic mapping function in the multivariable space; This represents the differential factor of the matrix of variables in the augmented parameter set. The output matrix is ​​calculated. This is the joint Jacobian matrix, which is a sensitivity matrix based on the static link length mapping and dynamic temperature mapping. Its internal elements quantify the influence weights of the underlying manufacturing and assembly tolerances and the small deformations caused by temperature rise on the final end spatial pose.

[0054] S105, simultaneously, it is also necessary to evaluate the sensitivity mapping relationship of the joint movement itself. Modeling module 100 calculates the partial derivative of the theoretical end-effector pose with respect to the joint angle to obtain the task Jacobian matrix. During the mathematical calculations, the augmented parameter set... With temperature sequence vector Treating it as a constant that does not change instantaneously with time, the theoretical pose of the end effector is... The six spatial dimension variables respectively affect the joint angle vector In Taking the first-order partial derivatives of each rotation angle variable constitutes a dimension of... matrix The calculation formula is:

[0055] ;

[0056] In the formula, This represents the differential factors of the joint angle variable matrix. The output matrix is ​​calculated. This is the task Jacobian matrix, which is a differential mapping matrix that reflects the small linear and angular displacements of the end effector in the Cartesian operating space caused by small rotation angle changes of each joint under the current spatial configuration.

[0057] See attached document Figure 1 and Figure 2 In this embodiment, the vision module 200 specifically executes the following sub-steps:

[0058] S201, the vision module 200 is connected to a binocular camera. The vision module 200 acquires measurement parameters of the target at the end of the robotic arm through the binocular camera. In specific implementation, a visual target containing a high-contrast geometric pattern is rigidly fixed to the end flange of the robotic arm. The binocular camera synchronously captures images of the physical space where the visual target is located and extracts its image feature pixels. The aforementioned binocular camera refers to an optical measurement device composed of two industrial cameras with fixed relative positions and epipolar correction. It acquires the three-dimensional geometric information of the object by simulating the principle of human binocular parallax. The measurement parameters include the physical baseline length of the binocular camera, the spatial depth of the target, the sum of squared reprojection errors, and the local gradient of the radial distortion field of the image plane. The physical baseline length refers to the absolute straight-line distance between the optical centers of the left and right cameras in a stereo camera in physical space. This value is usually obtained through calibration using an external calibration plate at the camera's factory. The spatial depth of the target refers to the vertical distance between the physical center of the target and the plane containing the stereo camera coordinate system, calculated from the disparity of the stereo image. The sum of squares of reprojection errors refers to the sum of the squares of the Euclidean distances between the theoretical pixel coordinates and the actual extracted observation pixel coordinates when the three-dimensional observation point is remapped back to the two-dimensional image plane through the camera projection model. This quantifies the local pixel noise level during feature extraction. The local gradient of the radial distortion field of the image plane refers to the rate of change of distortion displacement in the local pixel region where the target feature point is located when the image exhibits nonlinear distortion due to the physical deformation of the camera lens. The gradient near the image edge usually has a larger magnitude. For camera internal parameter calibration and sub-pixel extraction of image feature points, those skilled in the art can use the Zhang Zhengyou calibration method and edge detection algorithms. The underlying image coordinate system to world coordinate system transformation model is a well-known technology in this field and will not be elaborated here.

[0059] S202, accurately mapping the pixel extraction error of the two-dimensional image plane to three-dimensional physical space is a crucial step in evaluating the quality of visual observation. The vision module 200 calculates the covariance matrix of the three-dimensional spatial coordinates based on the error propagation law. As a preferred method, the vision module 200 utilizes the sum of squared reprojection errors and the local gradient of the radial distortion field of the image plane to construct the pixel observation variance on the two-dimensional image plane. Since camera lens distortion leads to uneven pixel noise distribution in different regions, a linear combination of these two factors can accurately reflect the local observation quality of the current target location in the image. The formula for calculating the pixel observation variance is:

[0060] ;

[0061] In the formula, Represents the pixel observation variance in a two-dimensional plane; This represents the sum of squared reprojection errors; This represents the local gradient of the radial distortion field in the image plane; The second norm of a vector; and For example, the preset constant weighting coefficients. The possible value is 1.0. The possible value is 0.5, and its specific value can be obtained from the statistical regression of historical camera calibration data.

[0062] Furthermore, based on the triangulation principle of binocular vision, there is a non-linear mapping relationship between the 3D coordinates and their pixel coordinates in the left and right images. Taking the first-order partial derivative of this non-linear mapping function with respect to the pixel coordinates of the left and right cameras yields the visual observation Jacobian matrix. The aforementioned nonlinear mapping function and its partial derivatives based on the binocular triangulation model belong to the classic 3D reconstruction algorithm in the field of computer vision. Specifically, it utilizes the intrinsic parameter matrices of the left and right cameras and the extrinsic translation and rotation matrices between the cameras to establish a perspective projection equation from the 3D spatial coordinates to the four 2D pixel coordinate components of the left and right images (the horizontal and vertical coordinates of the left and right cameras), and then calculates the partial derivatives with respect to the four pixel coordinate variables. This mathematical derivation process is a well-known existing technique in the field of visual observation. The Jacobian matrix in this visual observation... In the specific derivation and construction, the amplitude of its internal matrix elements is directly determined by the physical baseline length and the spatial depth of the target in the measurement parameters. The greater the spatial depth or the smaller the physical baseline length, the larger the value of the depth partial derivative element in this matrix. This is then combined with the derived visual observation Jacobian matrix. And the pixel observation variance, to calculate the covariance matrix. The calculation formula is:

[0063] ;

[0064] In the formula, This represents the output covariance matrix; The dimensions are strictly corresponding to the four pixel coordinate variables of the left and right cameras in a 4×4 unit diagonal matrix to ensure the validity of the dimensions in the matrix multiplication; superscript This represents the transpose of the matrix. The calculated covariance matrix is ​​a symmetric square matrix in a multidimensional random variable space, used to characterize the measurement variances of the three-dimensional spatial coordinates (X, Y, Z directions) of the robotic arm's end effector and the error correlations between different directions. In essence, it is a mathematical quantification of the dispersion and uncertainty of optical observation data in real three-dimensional space.

[0065] S203, to reasonably weight observation data of different qualities in subsequent optimal parameter estimation, the matrix representing the error magnitude needs to be converted into a confidence level of the measurement data. The vision module 200 inverts the covariance matrix and uses the resulting inverse matrix as the visual confidence matrix. In the specific numerical calculation, to avoid the algorithm dead zone caused by the covariance matrix becoming singular due to excessive observation distance, the vision module 200 uses a singular value decomposition algorithm to transform the covariance matrix... Perform the inversion operation, and if any singular value obtained from the decomposition is less than the preset machine precision floating-point tolerance (e.g., 10), then... -7 When ), its reciprocal is forcibly truncated and set to zero to obtain the numerically stable generalized inverse matrix. The calculation formula is:

[0066] ;

[0067] In the formula, This represents the inverse matrix obtained; the superscript -1 indicates the matrix inverse operation. Calculate the output matrix. This is the visual confidence matrix. The visual confidence matrix is ​​a weighted matrix used to measure the reliability of the 3D coordinate data output by the binocular camera at the current moment. The larger the eigenvalue of the matrix in a certain spatial direction, the smaller the measurement error in that direction, and the more reliable the observation data. Conversely, if the target is at the edge of the camera's field of view, causing a larger local gradient, or is far from the camera, causing an increase in spatial depth, the weight value of the visual confidence matrix in the corresponding spatial dimension will adaptively decrease through a formula, thereby effectively reducing the interference of inferior observation data in subsequent data fusion calculations.

[0068] See attached document Figure 1 and Figure 2 In this embodiment, the monitoring module 300 specifically executes the following sub-steps:

[0069] S301, the monitoring module 300 receives the joint Jacobian matrix output by the modeling module 100 and the visual confidence matrix output by the vision module 200. To construct a quadratic objective function for state estimation, the monitoring module 300 multiplies the transpose of the joint Jacobian matrix, the visual confidence matrix, and the joint Jacobian matrix to calculate the Hessian matrix. As a preferred approach, considering the aforementioned joint Jacobian matrix... The dimension is Corresponding to six-dimensional spatial pose, but if the original visual confidence matrix only corresponds to three-dimensional spatial position, it needs to be expanded into a 6×6 diagonal block matrix before performing multiplication. The sub-matrix blocks corresponding to the pose are set as minimum constant diagonal matrices or assigned values ​​based on the actual pose observation accuracy to ensure strict matching of matrix multiplication dimensions. The calculation formula is:

[0070] ;

[0071] In the formula, This represents the calculated Hessian matrix; This represents the received joint Jacobian matrix; This represents the visual confidence matrix after dimensional expansion and alignment. This represents the transpose of the joint Jacobian matrix. The transpose of the joint Jacobian matrix is ​​a mathematical matrix formed by interchanging the row and column elements of the original matrix. The calculated Hessian matrix is ​​a square matrix of the second-order partial derivatives of the target equation with respect to each parameter to be identified. In a physical sense, it represents the information matrix after incorporating the uncertainty of visual measurement under the current pose of the robotic arm. The distribution of elements within this matrix directly determines whether each physical parameter can be solved independently.

[0072] S302, For the information matrix constructed above, it is necessary to further quantify the stability of its numerical structure. Monitoring module 300 calculates the first eigenvalue and the last non-zero eigenvalue of the Hessian matrix after sorting by numerical value in descending order. In specific implementation, due to the constructed Hessian matrix... Mathematically, the Hessian matrix is ​​a symmetric positive semi-definite matrix. Monitoring module 300 uses singular value decomposition (SVD) to decompose its eigenvalues, obtaining all non-negative eigenvalues. The magnitude of these eigenvalues ​​physically maps to the observable information of each combination of parameters to be identified under the current configuration. After obtaining all eigenvalues, they are sorted in descending order of magnitude. The largest eigenvalue is extracted as the first eigenvalue, and the sequence is traversed downwards, skipping invalid zero values ​​that are close to the computer's floating-point precision. The smallest eigenvalue in the remaining sequence is then extracted as the last non-zero eigenvalue.

[0073] S303, the monitoring module 300 calculates the condition number by the ratio of the first eigenvalue to the last eigenvalue. When the condition number is greater than a preset parameter coupling critical threshold, the monitoring module 300 determines that the link deformation has triggered kinematic parameter coupling and generates a micro-excitation command. The calculation formula is:

[0074] ;

[0075] In the formula, This represents the calculated condition number; Indicates the first eigenvalue; This represents the last eigenvalue. During this process, when multiple internal error parameters cause linearly correlated interference to the last observed data, the last data in the aforementioned eigenvalue sequence will approach zero, leading to a significant increase in the ratio. The preset parameter coupling critical threshold refers to a safety boundary set in advance to ensure that the subsequent equation solution process does not diverge. The parameter coupling critical threshold is determined as follows: Before the identification system officially runs, the robotic arm is controlled to traverse multiple sets of random non-singular poses within its workspace under no-load conditions, collecting condition number samples when all augmented parameters successfully converge independently under historical standard verification conditions; the statistical average of all successfully converged condition number samples is calculated. with standard deviation According to statistics, 3 The criterion sets the preset parameter coupling critical threshold as follows: For example, it can be set to 10. 3 Up to 10 4 A real constant between orders of magnitude. If the actual calculated ratio exceeds this boundary, it indicates that the data collected under the current static configuration has redundancy and aliasing. At this time, the monitoring module 300 will generate a corresponding marker signal. This marker signal is the micro-excitation command, which is a Boolean control signal issued by the monitoring program to the actuator to trigger the robotic arm to deviate from the current fixed working point to obtain observation data in the new dimension. Conversely, when the calculated condition number is less than or equal to the preset parameter coupling critical threshold, the monitoring module 300 determines that the kinematic parameters to be identified under the current spatial configuration are independent and have good observability. At this time, the monitoring module 300 does not generate a micro-excitation command, but directly triggers the control module 400 to perform data sampling according to the normal process and simultaneously sends commands to the binocular camera to obtain the measured pose, thereby ensuring that the system can smoothly enter the subsequent parameter update closed loop under the excellent state of parameter non-coupling.

[0076] See attached document Figures 1-3 In this embodiment, the control module 400 specifically executes the following sub-steps:

[0077] S401, the control module 400 receives the micro-excitation command sent by the monitoring module 300. After receiving the command, it needs to evaluate the effective observation dimensions in the current parameter space. The control module 400 extracts the eigenvector corresponding to the last eigenvalue of the Hessian matrix as the ideal excitation direction. In specific implementation, since the constructed Hessian matrix is ​​a symmetric positive semi-definite matrix, the control module 400 uses the Jacobi method to diagonalize it and solve for the basic eigenvector set corresponding to the matrix. Then, it extracts the basic eigenvector that strictly corresponds to the last eigenvalue and normalizes the basic eigenvector using the L2 norm. This normalization process divides each element of the basic eigenvector by its own Euclidean length to calculate a unit column vector with unit length. This unit column vector is the ideal excitation direction. The ideal excitation direction refers to the spatial search vector in the multidimensional parameter space that can eliminate the linear correlation between unobservable parameters with the shortest physical path. It accurately indicates the combination of error parameters with the least information under the current static configuration.

[0078] S402, after determining the search vector, it needs to be mapped into an executable physical motion. The control module 400, combined with the task Jacobian matrix output by the modeling module 100 and the preset process tolerance zone, constructs a quadratic programming problem. In the mathematical modeling stage, since the ideal excitation direction is in the parameter error space, the control module 400 maps it to the desired displacement of the end-effector's operating space using the joint Jacobian matrix; simultaneously, it uses the actual end-effector displacement obtained by multiplying the task Jacobian matrix by the increment of the joint angle to be solved, and performs residual approximation with the aforementioned desired displacement, thereby constructing a least-squares objective function. To avoid additional actions interfering with the predetermined processing task, boundary inequality constraints are introduced. The calculation formula for the quadratic programming problem is:

[0079] ;

[0080] ;

[0081] In the formula, This represents the joint angle micro-bias vector to be solved; Represents the proportional gain constant; This represents the task Jacobian matrix output by modeling module 100; Represents the joint Jacobian matrix; This represents the extracted feature vector; Represents the square of the L2 norm; Represents the infinite norm; This indicates the preset process tolerance zone. The preset process tolerance zone refers to the maximum permissible three-dimensional spatial positional deviation boundary value specified on the engineering drawings based on the specific machining task currently being performed by the robotic arm; for example, it might be set to 0.1mm in a precision assembly scenario. Proportional gain constant. The excitation step size is determined by extracting the product of the duration of the underlying control cycle and the maximum safe linear velocity of the robotic arm end effector as a reference value, and setting it to a real number between 0.01 and 0.05 to ensure that the displacement caused by a single micro-excitation is within the millimeter-level micro-range.

[0082] S403, the control module 400 generates joint micro-biases by solving a quadratic programming problem and sends the joint micro-biases to the underlying controller of the robotic arm. In a preferred embodiment, the control module 400 uses the interior-point method to numerically solve the above quadratic programming problem. The specific solution process is as follows: the control module 400 introduces Lagrange multipliers, transforms the inequality constraints corresponding to the preset process tolerance zone into a logarithmic barrier function, and integrates this logarithmic barrier function into the original objective function to construct an unconstrained augmented objective equation; it calculates the gradient vector and Hessian approximation matrix of the augmented objective equation with respect to the joint variables to be solved, thereby determining the Newton descent search direction for the current iteration step; it updates the joint angle variables step by step along the Newton descent search direction until the absolute value of the calculated gradient vector is less than the preset convergence tolerance. The final converged solution output at this point is the joint micro-bias. It should be noted that the above solution process for the quadratic programming problem with inequality constraints and its underlying interior-point method optimization algorithm are known prior art in the fields of mathematical optimization and robot control. Joint micro-offset refers to the additional small angular displacement command superimposed on the current nominal target angle of each joint to break the parameter coupling state. The underlying controller refers to the servo control hardware motherboard responsible for parsing position commands and executing motor current closed-loop and speed closed-loop control. Its communication parsing and internal drive mechanism are well-known technologies in this field and will not be elaborated here.

[0083] S404, during the joint micro-biasing process of the robotic arm, the control module 400 collects the real-time feedback torque of all joint servo drives. Specifically, the control module 400 establishes synchronous periodic communication with all joint servo drives via an industrial Ethernet fieldbus. Within each fixed low-level communication cycle (e.g., 1ms), the control module 400 directly reads the current loop actual output register value mapped in the object dictionary of each joint servo drive and multiplies this current value by the motor's rated torque constant, thus converting it into real-time feedback torque in the physical world. To evaluate the dynamic response trend within the transmission chain, the control module 400 calculates the first-order time derivative of the real-time feedback torque to obtain the torque derivative. To avoid interference from high-frequency electromagnetic noise in derivative extraction, discrete difference calculation based on a historical sampling window is used. The calculation formula is:

[0084] ;

[0085] In the formula, This represents the calculated derivative of the torque; This represents the real-time feedback torque during the current sampling period; This represents the real-time feedback torque from the previous sampling period; This indicates the data communication refresh cycle of the servo driver.

[0086] When the absolute values ​​of the torque derivatives of all joints are less than the dead-band threshold, the control module 400 sends a synchronous trigger exposure signal to the binocular camera. The binocular camera then acquires the measured pose of the robotic arm based on the synchronous trigger exposure signal. The dead-band threshold here refers to the upper limit boundary of torque fluctuation noise when the joints overcome motion friction and the elastic deformation of the internal flexible transmission mechanism reaches steady-state equilibrium. This dead-band threshold is determined by consulting the static noise floor technical manual of a specific servo driver model to obtain the factory reference value, for example, a value between 0.5 N·m / s and 1.0 N·m / s. When the above torque derivative condition is met, it indicates that the linkages have filtered out motion aftershocks and are in a relatively absolutely rigid static state. At this time, sending the synchronous trigger exposure signal via network message ensures that the measured pose acquired by the binocular camera is free from motion blur interference, providing high-confidence observation samples for subsequent calculations. Conversely, when the absolute value of the torque derivative of any joint is greater than or equal to the dead-band threshold, it indicates that there are still residual elastic oscillations or incompletely released motion inertia within the robotic arm's transmission chain, and it has not yet reached a static stable state suitable for high-precision visual observation. At this time, the control module 400 temporarily suspends the issuance of the synchronous trigger exposure signal and drives the control flow back, maintaining the real-time feedback torque acquisition and derivative calculation process of the current cycle to continuously monitor the torque change trend of the underlying servo system until the static judgment condition that the absolute value of the torque derivative of all the aforementioned joints is less than the dead-band threshold is fully satisfied. This branch logic constructs a rigorous waiting closed loop on the underlying algorithm, effectively avoiding the risk of image motion blur and observation data failure caused by premature exposure triggering due to the robotic arm not being completely stationary.

[0087] See attached document Figure 1 and Figure 2 In this embodiment, the update module 500 is executed through the following sub-steps:

[0088] S501, the update module 500 receives the measured pose and the theoretical pose of the end effector. Here, the aforementioned pose data collectively refers to the position and orientation information of the robotic arm's end effector in three-dimensional space. Specifically, the measured pose refers to the actual three-dimensional state of the robotic arm's end effector in the physical world, extracted and calculated by the binocular camera based on visual features. Mathematically, it includes a three-dimensional measured position vector and a measured rotation matrix representing the absolute rotational relationship in space. Correspondingly, the theoretical pose of the end effector refers to the theoretical state of the robotic arm's end effector, derived forward from the underlying controller under the currently set nominal kinematic parameters. It also includes a three-dimensional theoretical position vector and a theoretical rotation matrix. After obtaining the above basic data, the update module 500 subtracts the theoretical pose from the measured pose to calculate the residual. In the specific algebraic operations, since the position variables satisfy the linear addition and subtraction rules of Euclidean space, the update module 500 can directly subtract the theoretical position vector from the measured position vector to obtain the position deviation. For attitude components that do not satisfy the conventional additive closure property, those skilled in the art can use Lie groups and Lie algebra mapping algorithms to perform matrix multiplication of the transpose of the measured rotation matrix and the theoretical rotation matrix to obtain the relative rotation matrix, and then convert it into a three-dimensional Lie algebra vector as the attitude deviation through logarithmic mapping. After obtaining the two types of deviations, the update module 500 vertically aligns and concatenates the position deviation vector and the attitude deviation vector along the longitudinal direction to form a six-dimensional column vector of the current observation point. This six-dimensional column vector is the residual. The residual refers to the mathematical difference between the end state predicted by the theoretical kinematic model and the physical observation result of the binocular camera under the current nominal parameter configuration, which physically quantifies the inaccuracy of the current model.

[0089] S502, after collecting observation data from multiple spatial locations within the workspace, the reliability of the data at each measurement point needs to be comprehensively considered. The update module 500 extracts the visual confidence matrix corresponding to all sampling points and arranges the visual confidence matrices diagonally to construct a global heteroscedasticity block diagonal weight matrix. During implementation, this is done by traversing the total number of records kept by the identification system. For each valid sampling point, a 6×6 visual confidence matrix is ​​extracted sequentially according to the data acquisition time series. To perform global-scale optimization calculations, the update module 500 pre-allocates a 6×6 dimension matrix within the recognition system. A large sparse square matrix. The extracted... Visual confidence matrices are sequentially filled into the main diagonal blocks of this large sparse matrix, while all elements in the remaining off-diagonal blocks are forced to zero. The resulting matrix is ​​the global heteroscedasticity block diagonal weight matrix. The global heteroscedasticity block diagonal weight matrix is ​​a weighting operator used in the multi-measurement joint identification equation to differentiate the contribution of residual data at different spatial locations to the final parameter update. It ensures that high-quality observation data dominates the optimization process.

[0090] S503, after assembling the global observation data, it is necessary to establish an inverse mapping relationship from operational space error to joint parameter error. The update module 500, combining the Hessian matrix, joint Jacobian matrix, global heteroscedasticity block diagonal weight matrix, and residuals, uses a weighted nonlinear iterative algorithm to calculate the update step size of the augmented parameter set. As a preferred approach, this step uses the Levenberg-Marquardt algorithm as the core iterative solver. In this process, to achieve data fusion from multiple isolated sampling points, the update module 500 vertically concatenates the joint Jacobian matrices of all sampling points (i.e., stacking them in a row-wise expansion dimension) to form a global-scale joint Jacobian matrix, and vertically concatenates the residuals of all sampling points to form a global residual column vector. After concatenation, the update module 500, combining the previously generated basic mathematical expression, calculates the Hessian matrix used for global optimization by multiplying the transpose of the concatenated joint Jacobian matrix and the global heteroscedasticity block diagonal weight matrix with the concatenated joint Jacobian matrix. To clarify the underlying logic of the above matrix operations, the calculation formula is disclosed as follows:

[0091] ;

[0092] In the formula, This represents the calculated Hessian matrix used for global optimization. This represents the joint Jacobian matrix after vertically concatenating the above components along the vertical axis. This represents the diagonal weight matrix of the global heteroscedasticity block; This represents the transpose of the concatenated joint Jacobian matrix. Through the above global matrix multiplication operations, the identification system can simultaneously integrate the local error sensitivity and observation uncertainty of all spatial sampling points.

[0093] Based on this, a regularized normal equation is constructed and the unknown vector is solved. The calculation formula is:

[0094] ;

[0095] In the formula, This represents the update step size of the augmented parameter set to be solved; This represents the Hessian matrix calculated above and used for global optimization; This represents the damping factor used to balance the gradient descent method and the Gauss-Newton method; Represents an identity matrix aligned with the dimension of the total number of parameters to be identified; This represents the joint Jacobian matrix after vertical splicing along the longitudinal direction; This represents the transpose of the concatenated joint Jacobian matrix; This represents the constructed global heteroscedasticity block diagonal weight matrix; This represents the global residual column vector formed by vertically splicing along the longitudinal direction.

[0096] Due to the complexity of nonlinear systems, the damping factor The selection of the matrix directly determines whether the algorithm can successfully avoid matrix singularities and achieve convergence. It is determined as follows: in the first iteration, the globally optimized Hessian matrix is ​​extracted. The maximum value of the main diagonal elements, multiplied by a preset constant (e.g., 10). -3 The initial damping factor is used as the initial damping factor. In each subsequent iteration, the update module 500 dynamically adjusts the damping factor based on the ratio of the actual decrease in the sum of squared residuals after the current update step size is substituted into the objective function to the predicted decrease in the linear approximation model (i.e., the Marquardt proportionality constant). If the ratio is greater than zero, it indicates that the model is well approximated, and the damping factor is reduced to accelerate convergence. If the ratio is less than or equal to zero, it indicates that the current step size leads to an increase in error, and the update is rejected, and the damping factor is increased exponentially, forcing the search direction to approach the stable gradient descent direction.

[0097] For numerical solutions to this system of linear equations, those skilled in the art can use the Cholesky decomposition method to obtain a stable solution vector. This solution vector is the update step size of the augmented parameter set. The update step size of the augmented parameter set refers to the algebraic correction amount of each error parameter to be identified, calculated within the current iteration period, in order to reduce the quadratic objective function of the overall observation error.

[0098] S504, after obtaining the above correction amount, it needs to be applied to the closed loop of the system's physical mathematical model. The update module 500 accumulates the update step size into the historical parameters to complete the update of the augmented parameter set. The calculation formula is:

[0099] ;

[0100] In the formula, This represents the updated augmented parameter set; This represents the historical parameters at the start of the current iteration cycle (in the first iteration: it refers to the augmented parameter set initially established by the robotic arm, i.e., the basic model parameters built based on the initial factory parameters of the link and the current temperature; in subsequent iterations: it refers to the augmented parameter set that has just been updated after the previous iteration calculation). Updating the augmented parameter set means compensating the parameter deviations calculated by the optimization back into the original kinematic derivation model, so that the corrected theoretical model can more realistically reflect the actual manufacturing tolerances and assembly offsets of the link. After completing the parameter accumulation calculation, the decrease in the sum of squares of the residuals in this iteration is calculated. When this decrease or the infinite norm of the update step is less than the preset convergence tolerance, the update module 500 determines that the iteration has converged and outputs the final identification parameters. The aforementioned preset convergence tolerance refers to the numerical boundary where the parameter adjustment is determined to be close to the global minimum and no further computational effort is required. The convergence tolerance is determined as follows: based on the repeatability accuracy parameters of the robotic arm at the factory, it is converted into a microscopic equivalent change in joint angle or link length, and the value range can be set within 10. -6 Up to 10 -8 On the other hand, when the magnitude of the decrease or the infinite norm of the update step is greater than or equal to the preset convergence tolerance, it indicates that the current optimization process has not yet reached the global minimum region. At this time, the update module 500 uses the updated parameter set as the input benchmark value for the next iteration, and drives the control flow back to continue the recalculation of the Jacobian matrix and the residual evaluation process for subsequent cycles.

[0101] This embodiment uses a 6-DOF industrial robotic arm on a precision assembly line as an example. After continuous high-load operation, the temperature of the connecting motor and castings rises, leading to a decrease in the absolute positioning accuracy of the end effector. To restore positioning accuracy, an adaptive kinematic parameter identification system for the robotic arm, guided by binocular vision, is activated.

[0102] Phase 1: During system initialization, the modeling module 100 acquires the current joint angles and temperatures of each joint of the robotic arm. Specifically, the modeling module 100 extracts real-time sequence data from the temperature sensor integrated into the absolute encoder as the temperature, measuring the current real-time temperature of each joint of the robotic arm to be between 42℃ and 48℃. Subsequently, the modeling module 100 constructs an augmented parameter set containing geometric errors and the coefficient of thermal expansion based on the acquired temperatures and the initial parameters of the robotic arm links. On this basis, the modeling module 100 calculates the theoretical end-effector pose of the robotic arm using the joint angles, temperatures, and the augmented parameter set, based on the forward kinematic mapping function. Simultaneously, to establish the mathematical correlation for error propagation, the modeling module 100 calculates the partial derivatives of the theoretical end-effector pose with respect to the augmented parameter set to obtain the joint Jacobian matrix, and calculates the partial derivatives of the theoretical end-effector pose with respect to the joint angles to obtain the task Jacobian matrix.

[0103] The second stage: When the robotic arm moves to the initial sampling point, the vision module 200 connects to the binocular camera, and the vision module 200 acquires the measurement parameters of the target at the end of the robotic arm through the binocular camera. In this embodiment, the physical baseline length of the binocular camera is 120mm, the resolved spatial depth of the target is 850mm, and the corresponding sum of squared reprojection errors and the local gradient of the radial distortion field of the image plane are extracted simultaneously. To quantify the observation quality of the current spatial position, the vision module 200 calculates the covariance matrix of the three-dimensional spatial coordinates based on the error propagation law. Then, the vision module 200 inverses the covariance matrix and uses the resulting inverse matrix as the visual confidence matrix. This matrix mathematically reduces the weight of poor-quality observation data at the edge of the field of view in subsequent calculations.

[0104] The third stage: The monitoring module 300 receives the joint Jacobian matrix output by the modeling module 100 and the visual confidence matrix output by the vision module 200. To evaluate parameter discriminability, the monitoring module 300 multiplies the transpose of the joint Jacobian matrix, the visual confidence matrix, and the joint Jacobian matrix to calculate the Hessian matrix. Then, the monitoring module 300 calculates the first eigenvalue and the last non-zero eigenvalue in the Hessian matrix, arranged in descending order of numerical value. The monitoring module 300 calculates the ratio of the first eigenvalue to the last eigenvalue to obtain the condition number; the calculated condition number for the current configuration is 8500. In this embodiment, the preset parameter coupling critical threshold is set to 5000. Since the current condition number 8500 is greater than the preset parameter coupling critical threshold 5000, the monitoring module 300 determines that the link deformation triggers kinematic parameter coupling and generates a micro-excitation command. (See attached...) Figure 4 As shown, Figure 4 The data points on the solid black line represent the condition number calculated by the monitoring module 300 for each sampling sequence, while the dark gray horizontal dashed line represents the preset parameter coupling threshold. When a solid line data point crosses the horizontal dashed line upwards, it visually indicates that the current condition number is greater than the preset boundary, and the system generates a micro-excitation command at this moment. After the action is executed, the condition number of subsequent sampling points quickly falls back below the dashed line, indicating that the parameter coupling state has been successfully decoupled.

[0105] In the fourth stage, after receiving the micro-excitation command from the monitoring module 300, the control module 400 extracts the eigenvector corresponding to the last eigenvalue of the Hessian matrix as the ideal excitation direction. To ensure that the additional action does not disrupt the current processing task, the control module 400 constructs a quadratic programming problem by combining the task Jacobian matrix output by the modeling module 100 and the preset process tolerance zone. In this embodiment, the preset process tolerance zone is set to 0.1mm. The control module 400 generates the joint micro-bias by solving the quadratic programming problem and sends the joint micro-bias to the underlying controller of the robotic arm. During the execution of the joint micro-bias by the robotic arm, the control module 400 collects the real-time feedback torque of all joint servo drives. To determine whether the robotic arm is truly stationary, the control module 400 calculates the first-order time derivative of the real-time feedback torque to obtain the torque derivative. In this embodiment, the dead zone threshold is set to 0.8N·m / s. When the absolute values ​​of the torque derivatives of all joints are less than the dead-band threshold, the control module 400 sends a synchronous trigger exposure signal to the binocular camera. The binocular camera then acquires the measured pose of the robotic arm based on the synchronous trigger exposure signal. This process is executed cyclically within the robotic arm's workspace, acquiring a total of 20 sets of valid spatial measurement point data.

[0106] Phase 5: After data acquisition, the update module 500 receives the measured pose and the theoretical end-effector pose. The update module 500 subtracts the theoretical end-effector pose from the measured pose to calculate the residual. To fuse multi-point data, the update module 500 extracts the visual confidence matrix corresponding to all sampling points and arranges the visual confidence matrix along the diagonal to construct a global heteroscedasticity block diagonal weight matrix. Based on this, the update module 500 combines the Hessian matrix, the joint Jacobian matrix, the global heteroscedasticity block diagonal weight matrix, and the residuals, using a weighted nonlinear iterative algorithm to calculate the update step size of the augmented parameter set. During this process, the residuals of all sampling points are vertically concatenated along the longitudinal direction to form a global residual column vector. In this embodiment, by constructing and solving a regularized normal equation, after four nonlinear iterations, the decrease in the sum of squared residuals is less than the preset convergence tolerance of 10. -6 The final update module 500 adds the update step size to the historical parameters to complete the update of the augmented parameter set. The underlying control system calls the updated augmented parameter set to replace the original model. Independent verification shows that after updating the parameters, the absolute value of the residual corresponding to the absolute positioning of the robotic arm's end effector decreased from the initial 1.2mm to 0.07mm, achieving a leap in absolute positioning accuracy from millimeter to sub-millimeter levels. This verifies the effectiveness of the binocular vision-guided adaptive kinematic parameter identification system for the robotic arm. (See attached...) Figure 5 As shown, Figure 5The black solid line marked with an asterisk represents the approximate trajectory of the absolute value of the positional deviation of the global residual column vector during the process of update module 500 calling the nonlinear iterative algorithm to calculate the update step size of the augmented parameter set. In the initial 0th iteration, the absolute value of the residual is approximately 1.2 mm; as the iteration progresses, the absolute value of the residual shows a monotonically decreasing trend, and tends to level off after the 4th iteration.

[0107] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A binocular vision-guided adaptive kinematic parameter identification system for a robotic arm, characterized in that, include: The modeling module is used to obtain the joint angles and temperature of the robotic arm, construct an augmented parameter set containing geometric errors and thermal expansion coefficients, and calculate the theoretical pose of the end effector, the joint Jacobian matrix, and the task Jacobian matrix. The vision module is used to acquire measurement parameters through a binocular camera and calculate the inverse of the covariance matrix as the visual confidence matrix. The monitoring module is used to construct a Hessian matrix based on the joint Jacobian matrix and the visual confidence matrix, and to issue a micro-excitation command when the condition number of the Hessian matrix is ​​greater than the parameter coupling critical threshold. The control module is used to receive the micro-excitation command, combine the task Jacobian matrix and tolerance band to generate joint micro-offset, and trigger the binocular camera to acquire the measured pose according to the torque derivative; The update module is used to update the augmented parameter set by combining the residual between the measured pose and the theoretical pose of the end effector with the visual confidence matrix.

2. The adaptive kinematic parameter identification system for a robotic arm based on binocular vision guidance according to claim 1, characterized in that, The modeling module extracts real-time sequence data from the temperature sensor integrated with the absolute encoder as the temperature, and constructs the augmented parameter set containing the geometric error and the coefficient of thermal expansion based on the temperature and the initial parameters of the robotic arm linkage. The modeling module calculates the theoretical end-effector pose of the robotic arm based on the forward kinematic mapping function, using the joint angles of the robotic arm, the temperature, and the augmented parameter set.

3. The adaptive identification system for kinematic parameters of a robotic arm based on binocular vision guidance according to claim 2, characterized in that, The modeling module calculates the partial derivative of the end-effector theoretical pose with respect to the augmented parameter set to obtain the joint Jacobian matrix, and the modeling module calculates the partial derivative of the end-effector theoretical pose with respect to the robot arm joint angle to obtain the task Jacobian matrix.

4. The adaptive identification system for kinematic parameters of a robotic arm based on binocular vision guidance according to claim 1, characterized in that, The measurement parameters include the physical baseline length of the binocular camera, the spatial depth of the target, the sum of squared reprojection errors, and the local gradient of the radial distortion field of the image plane. The vision module calculates the covariance matrix of the three-dimensional spatial coordinates based on the error propagation law, the pixel observation variance on the two-dimensional image plane, and the visual observation Jacobian matrix. It then uses the singular value decomposition algorithm to obtain the inverse matrix of the covariance matrix and uses the obtained inverse matrix as the visual confidence matrix.

5. The adaptive kinematic parameter identification system for a robotic arm based on binocular vision guidance according to claim 1, characterized in that, The monitoring module calculates the Hessian matrix by multiplying the transpose of the joint Jacobian matrix, the visual confidence matrix, and the joint Jacobian matrix; the monitoring module calculates the first eigenvalue and the last non-zero eigenvalue in the Hessian matrix after sorting them in descending order of numerical value, and calculates the ratio of the first eigenvalue to the last eigenvalue to obtain the condition number.

6. The adaptive kinematic parameter identification system for a robotic arm based on binocular vision guidance according to claim 1, characterized in that, The control module extracts the eigenvector corresponding to the last eigenvalue of the Hessian matrix as the ideal excitation direction, and constructs a quadratic programming problem by combining the task Jacobian matrix and the tolerance band; the control module generates the joint micro-bias by solving the quadratic programming problem, and sends the joint micro-bias to the underlying controller of the robotic arm.

7. The adaptive identification system for kinematic parameters of a robotic arm based on binocular vision guidance according to claim 6, characterized in that, During the process of the robotic arm performing the joint micro-biasing, the control module collects the real-time feedback torque of all joint servo drivers and calculates the first-order time derivative of the real-time feedback torque to obtain the torque derivative; when the absolute value of the torque derivative of all joints is less than the dead zone threshold, the control module sends a synchronous trigger exposure signal to the binocular camera, and the binocular camera collects the measured pose of the robotic arm according to the synchronous trigger exposure signal.

8. The adaptive kinematic parameter identification system for a robotic arm based on binocular vision guidance according to claim 1, characterized in that, The update module calculates the residual by subtracting the theoretical end pose from the measured pose; the update module extracts the visual confidence matrix corresponding to all sampling points, and arranges the visual confidence matrix along the diagonal to construct a global heteroscedasticity block diagonal weight matrix.

9. The adaptive identification system for kinematic parameters of a robotic arm based on binocular vision guidance according to claim 8, characterized in that, The update module vertically concatenates the joint Jacobian matrices of all sampling points along the vertical axis to form a global-scale joint Jacobian matrix, and vertically concatenates the residuals of all sampling points along the vertical axis to form a global residual column vector; the update module combines the transpose of the global-scale joint Jacobian matrix, the global heteroscedasticity block diagonal weight matrix, and the global-scale joint Jacobian matrix to calculate the Hessian matrix used for global optimization.

10. The adaptive identification system for kinematic parameters of a robotic arm based on binocular vision guidance according to claim 9, characterized in that, The update module constructs a regularized normal equation based on the Hessian matrix used for global optimization and solves for the unknown vector to obtain the update step size; the update module adds the update step size to the historical parameters to complete the update of the augmented parameter set; when the infinite norm of the update step size is less than the preset convergence tolerance, the update module determines that the iteration has converged.

Images