Shaking compensation control method for underwater manipulator based on spatio-temporal heterogeneous matching of sensors

Abstract

The application discloses a kind of underwater mechanical arm shake compensation control methods based on sensor space-time heterogeneous matching.The method comprises: first, the eye in hand camera of mechanical arm tail end is used to realize the estimation of mechanical arm base shake;While using inertial measurement unit to directly measure the mechanical arm base shake;Then, the multi-sensor space-time heterogeneous matching algorithm based on Kalman filtering and cross-correlation method is used to realize the fusion and optimal estimation of two-way shake measurement data;Finally, the joint control amount of mechanical arm is obtained through inverse kinematics solution, so as to effectively compensate the rotational shake of mechanical arm base and maintain the stability of mechanical arm tail end posture.The method solves the problem that the control accuracy may be damaged and the operation reliability may be reduced due to base shake during the operation of mobile platform mechanical arm, especially underwater vehicle-mechanical arm system, and can be combined with visual servo system or remote operation mechanical arm control system to improve the reliability and practicability of operation.

Description

Technical Field

[0001] This invention relates to a sway compensation control method for an underwater robotic arm, which relates to the field of robotic arm motion control, and specifically to a sway compensation control method for an underwater robotic arm based on spatiotemporal heterogeneous matching of sensors. Background Technology

[0002] In the field of marine engineering, underwater vehicle-manipulator systems (UVMS) play a crucial role, commonly used in marine resource exploration, underwater archaeology, and the inspection and maintenance of marine facilities. However, underwater manipulator operations have limitations. Currently, most mainstream UVMS systems still rely on master-slave remote control. The complex underwater environment, influenced by strong nonlinear motion coupling and unsteady flow disturbances, causes UVMS to exhibit swaying with random variations in direction, amplitude, and period, leading to decreased control accuracy at the manipulator's end effector. To stabilize the end effector, operators may need to expend considerable effort frequently making corrections; furthermore, for certain frequencies of swaying, the human operator's reaction speed may not meet the requirements for sway compensation control, resulting in human-machine coupling oscillations that severely compromise the reliability and practicality of underwater manipulator operations. Summary of the Invention

[0003] To address the problems existing in the background technology, this invention provides a sway compensation control method for underwater robotic arms based on spatiotemporal heterogeneous matching of sensors. Specifically, this invention is a compensation control method for the decreased end-effector control accuracy caused by platform sway during the operation of mobile platform multi-joint electric robotic arms or underwater vehicle-robotic arm systems (UVMS). To improve the stability and reliability of underwater robotic arm operations, this invention designs a sway compensation control method. Both camera-deployed VSLAM algorithms and IMUs can sense the sway of the robotic arm, but each sensor has its advantages and disadvantages: cameras can directly measure the pose changes of their own sway, but require certain excitation conditions to maintain the accuracy of attitude estimation, and scale drift problems will occur during long-term operation; IMUs can operate stably for a long time, and the measurement results are relatively accurate, but the angle and displacement are measured by integrating angular velocity and acceleration, and the gradual accumulation of errors will also lead to measurement drift. Therefore, the sway compensation control method designed in this invention uses both a camera and an IMU to sense the sway of the robotic arm base. The two sensors complement each other, and the sway is accurately measured based on spatiotemporal heterogeneous matching of multiple sensors, which improves the measurement stability. On this basis, it compensates for the impact of sway on the control accuracy of the robotic arm end effector, thereby improving the reliability of underwater robotic arm operation.

[0004] The technical solution adopted in this invention is: The underwater robotic arm sway compensation control method based on sensor spatiotemporal heterogeneous matching of the present invention includes: First, the kinematic model of the robotic arm is established using the DH parameter method to obtain the coordinate transformation matrix of the underwater robotic arm. At the same time, the distortion coefficients of the eye-hand monocular camera are calibrated.

[0005] Step 1: Underwater images are acquired using an eye-to-hand monocular camera at the end of the underwater robotic arm, thereby obtaining motion information of the robotic arm end relative to the camera coordinate system of the eye-to-hand monocular camera. This motion information of the robotic arm base relative to the camera coordinate system of the eye-to-hand monocular camera is then calculated. Simultaneously, the inertial measurement unit (IMU) on the robotic arm base obtains motion information of the robotic arm base relative to the IMU coordinate system, enabling the observation of the rotation and swaying of the robotic arm base.

[0006] Step 2: Perform coordinate system heterogeneous matching between the motion information of the robotic arm base relative to the camera coordinate system of the eye-to-hand monocular camera and the motion information of the robotic arm base relative to the coordinate system of the inertial measurement unit (IMU) to obtain the motion information of the robotic arm base after spatial matching.

[0007] Step 3: Perform time heterogeneous matching on the motion information of the robotic arm base relative to the world coordinate system after spatial matching to obtain the motion information of the robotic arm base after time delay matching.

[0008] The fourth step involves using a dual-loop Kalman filter to fuse the motion information of the robotic arm base after time delay matching, thereby obtaining an estimate of the sway angle of the robotic arm base. Based on the estimate of the sway angle of the robotic arm base, the target joint position of the underwater robotic arm is calculated, and then the operation of the underwater robotic arm is controlled to achieve sway compensation of the underwater robotic arm.

[0009] In the first step, based on the underwater images acquired by the eye-on monocular camera, the Visual Simultaneous Localization and Mapping (VSLAM) algorithm is used to obtain the six-degree-of-freedom motion estimate of the eye-on monocular camera, thereby obtaining the motion information of the robotic arm's end effector relative to the camera coordinate system of the eye-on monocular camera. This refers to the pose of the robotic arm's end effector, which in turn provides motion information about the robotic arm's base relative to the camera coordinate system of the eye-to-hand monocular camera. as follows: in, The coordinate transformation matrix from the robot arm base to the end effector is obtained through the robot arm's forward kinematics model.

[0010] The rotational motion of the robotic arm base is directly measured by an inertial measurement unit (IMU), thereby obtaining the motion information of the robotic arm base relative to the IMU coordinate system. .

[0011] In the second step, the camera coordinate system of the eye-on-hand monocular camera or the IMU coordinate system is used as the world coordinate system. That is, the measurement in one of the coordinate systems is used as the reference to achieve reference coordinate matching between the two measurements. The measurement usually starts before the shake compensation control is turned on, and the measurement at this time is used as the reference. In this way, the motion information of the robotic arm base relative to the camera coordinate system of the eye-on-hand monocular camera is obtained. Motion information of the robotic arm base relative to the inertial measurement unit (IMU) coordinate system The mean space is matched to the world coordinate system as follows: in, and These are the first and second motion information of the robotic arm base after spatial matching, i.e., the motion information of the robotic arm base after spatial matching; This is the transformation matrix from the world coordinate system to the camera coordinate system; This is the transformation matrix from the world coordinate system to the inertial measurement unit (IMU) coordinate system.

[0012] In the third step, the initial time period after the measurement begins is obtained. The motion information of the robotic arm base after spatial matching is obtained, and then the relative time delay between the eye and the hand monocular camera and the inertial measurement unit (IMU) is obtained through the cross-correlation function. By using feedforward compensation to make up for time delay, motion information of the robotic arm base after time delay matching is obtained, including the first motion information of the robotic arm base after time delay matching. Second motion information .

[0013] In the fourth step, the first motion information of the robotic arm base after time delay matching is first... Second motion information The inner loop Kalman filter is used to fuse the values ​​to obtain an estimate of the swaying angular velocity of the robotic arm base. Then, the estimated value of the swaying angular velocity of the robotic arm base is processed by the outer loop Kalman filter to obtain an estimate of the swaying angle of the robotic arm base. This achieves the effect of joint optimization estimation of the inner and outer loops and eliminates interference such as high-frequency noise.

[0014] In the fourth step, during the sway compensation control process, the estimated value of the sway angle of the robotic arm base is... and the preset reference signal of the underwater robotic arm input by the operator The control law for robotic arm sway compensation is obtained by superimposing the results. as follows: in, This is the transformation matrix from the world coordinate system to the initial pose of the robotic arm base; Estimated value of the sway angle of the robotic arm base The robotic arm base pose before sway compensation is considered stationary. To the robotic arm base position after sway compensation The transformation matrix; the preset reference signal of the underwater robotic arm. The target pose of the underwater robotic arm's end effector relative to the world coordinate system is defined as the coordinate transformation matrix from the world coordinate system to the end effector; the robotic arm sway compensation control law is also defined. The coordinate transformation matrix of the robotic arm required for sway compensation.

[0015] The robotic arm sway compensation control law After performing inverse kinematics solution, the compensation control quantity is obtained as the joint target position and sent to the joint motor of the underwater robotic arm for control, thereby realizing sway compensation.

[0016] The beneficial effects of this invention are: The method of this invention can accurately estimate the sway of the robotic arm base by using an IMU and an eye-to-hand camera at the end of the robotic arm. The sway of the base is treated as a virtual joint and incorporated into the inverse kinematics calculation of the robotic arm to obtain the compensation control quantity of each joint. This enables compensation control of uncontrollable sway of the base, improving the reliability and practicality of the robotic arm operation.

[0017] The method of this invention can effectively compensate for the rotational sway of the robotic arm base and maintain the stability of the robotic arm end posture. It solves the problem of reduced control accuracy and decreased operational reliability caused by base sway in mobile platform robotic arms, especially underwater vehicle-robotic arm systems, during operation. It can be combined with a vision servo system or a remotely operated robotic arm control system to improve the reliability and practicality of the operation. Attached Figure Description

[0018] Figure 1 This is a block diagram of the control system for an underwater electric robotic arm without the introduction of sway compensation control. Figure 2 This is a block diagram of the underwater robotic arm sway compensation control method based on sensor spatiotemporal heterogeneous matching designed in this invention; Figure 3 This is a diagram of the base shaking trajectory of the multi-sensor spatiotemporal heterogeneous matching algorithm used in the experimental verification of this invention. Figure 4 This is a graph showing the end-effector sway compensation in the experimental verification of this invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] like Figure 2 As shown, the specific implementation process of the underwater robotic arm sway compensation control method based on sensor spatiotemporal heterogeneous matching of the present invention is as follows: 1. Establish a kinematic model of the electric robotic arm, and use the DH parameter method to establish a link description of the robotic arm, thereby obtaining the transformation matrix of each joint, and the overall coordinate transformation matrix from the robotic arm base to the end effector. .

[0021] 2. Measure the coordinate transformation matrix between the inertial measurement unit (IMU) and the robotic arm base, and the coordinate transformation matrix between the eye-to-hand camera and the robotic arm end effector.

[0022] 3. Using a handheld monocular camera, parameter calibration was performed based on a pinhole imaging model. Zhang Zhengyou's calibration method was used to take multiple images of the checkerboard calibration board and calibrate the camera's intrinsic parameters and distortion coefficients.

[0023] 4. Deploy visual real-time localization and mapping (VSLAM) algorithms (such as ORB_SLAM2 algorithm), and use the environmental images obtained by the eye-to-hand monocular camera to obtain the end effector of the robotic arm. Relative to its world coordinates The pose information, i.e., the motion data of the robotic arm's end effector, is denoted as... .

[0024] 5. Activate the inertial measurement unit (IMU) to measure the rotational motion of the robotic arm base directly, thereby obtaining the data of the robotic arm base. Relative Inertial Measurement Unit (IMU) World Coordinates The pose information, i.e., the motion data of the robotic arm base, is denoted as... .

[0025] 6. Based on the camera coordinate system of the robotic arm's end effector relative to the eye-handled monocular camera. motion information The camera coordinate system of the robotic arm base relative to the eye-to-hand monocular camera can be obtained by solving the coordinate transformation matrix of the robotic arm's forward kinematics model. motion information This allows for the observation of the rotation and swaying of the robotic arm's base using an eye-to-hand camera. Specifically: Underwater images are acquired using an eye-on monocular camera mounted on the end effector of an underwater robotic arm. A Visual Real-Time Localization and Mapping (VSLAM) algorithm is then used to obtain six degrees of freedom motion estimates from the eye-on monocular camera, thereby acquiring motion information of the robotic arm's end effector relative to the camera's coordinate system. This refers to the end-effector pose, and then the motion information of the robot arm base relative to the camera coordinate system of the eye-in-hand monocular camera is obtained through robot arm kinematics calculation. as follows: in, The coordinate transformation matrix from the robot arm base to the end effector is obtained through the robot arm's forward kinematics model.

[0026] 7. Before activating sway compensation control, collect two channels of pose information data. Note that both channels acquire motion data of the robotic arm base, but the two sensors have spatial heterogeneity and refer to different world coordinate systems. Therefore, refer to the following coordinate systems respectively: and Therefore, the obtained motion data are also different in numerical value.

[0027] 8. When sway compensation control is enabled, coordinate system heterogeneity matching needs to be performed between the motion information of the robotic arm base relative to the camera coordinate system of the eye-to-hand monocular camera and the motion information of the robotic arm base relative to the inertial measurement unit (IMU) coordinate system, so that the world coordinate systems on which the two motion data are based are matched to the same world coordinate system. The camera coordinate system of the eye-in-the-hand monocular camera or the coordinate system of the inertial measurement unit (IMU) is used as the world coordinate system. The measurement in one of the coordinate systems is used as the reference to achieve reference coordinate matching between the two measurements. The measurement usually starts before the shake compensation control is turned on, and the measurement at this time is used as the reference. You can determine this yourself; for convenience, you can let... or .

[0028] It can be solved , The motion information of the robotic arm base relative to the camera coordinate system of the eye-mounted monocular camera. Motion information of the robotic arm base relative to the inertial measurement unit (IMU) coordinate system The motion information of the robotic arm base is obtained by spatially matching the coordinates to the world coordinate system. , ,as follows: in, and These are the first and second motion information of the robotic arm base after spatial matching, i.e., the motion information of the robotic arm base after spatial matching; This is the transformation matrix from the world coordinate system to the camera coordinate system; This is the transformation matrix from the world coordinate system to the inertial measurement unit (IMU) coordinate system.

[0029] In practical implementation, if the IMU coordinate system is chosen as the world coordinate system, then... , Given an identity matrix, before enabling sway compensation, we have: This serves as the initial value and will change in real time. The measurement obtained here is the measurement of the two sensors in the same world coordinate system after spatial matching, rather than the measurement in their respective coordinate systems, which is consistent with the aforementioned... , They are different. This enables adaptive spatial matching between the two sensors.

[0030] 9. In addition to spatial heterogeneity, the camera and IMU sensors also exhibit temporal heterogeneity, with different sampling frequencies and time delays, necessitating temporal matching. After spatial matching of the two sensors, temporal matching of the two data streams should be performed to obtain a unified base sway measurement result. The initial period after the start of measurement should be considered. The motion information of the robotic arm base after spatial matching, i.e., the measurement results of two sensors (three rotation axes decoupled), is cross-correlation function calculated, i.e., two-channel data convolution. The maximum value of the convolution result is the relative time delay between the eye-to-hand monocular camera and the inertial measurement unit (IMU). .have: By using feedforward compensation to make up for time delay, motion information of the robotic arm base after time delay matching is obtained, including the first motion information of the robotic arm base after time delay matching. Second motion information .

[0031] Because the camera sensor delay is significantly greater than that of the inertial measurement unit (IMU), compensation is performed to address the camera delay and achieve time heterogeneous matching between the two sensors. At this point, for any... ,due: 10. For two motion data streams from different sensors (camera and inertial measurement unit), sensor fusion is achieved using the Kalman filter algorithm. The Kalman filter algorithm is as follows: 1) Initialization parameters: Initial posterior error covariance The estimate from the previous optimization step ,k The prediction step value. The estimator to be optimized. In the inner loop, it represents angular velocity; in the outer loop, it represents angle.

[0032] 2) Prediction step: Predicted value: , Differentiate the value to be estimated. It is noise.

[0033] Prior error covariance: , This represents the noise variance of the prediction model.

[0034] 3) Update step: Kalman gain: in, and These represent the noise variances of the inertial measurement unit (IMU) and the eye-to-hand monocular camera, respectively.

[0035] Optimal estimate: , These are measured values.

[0036] in, and These are measurements obtained independently from an eye-to-hand monocular camera and an inertial measurement unit (IMU), respectively.

[0037] Posterior error covariance: .

[0038] 4) Repeat steps 2)-3) during the iteration process. By continuously increasing the value, the optimal estimated value of the three-axis rotational motion attitude angle is obtained.

[0039] The dual-loop Kalman filter designed based on the above Kalman filtering algorithm has an inner loop value to be estimated as the differential of the rotation vector (angular velocity), i.e.: in, , and These are the components of the base sway angular velocity measured by the camera along the x, y, and z axes, respectively. , and These are the components of the base sway angular velocity measured by the IMU along the x, y, and z axes, respectively.

[0040] The outer ring to be estimated is a rotation vector (angle), that is: in, , and These are the components of the base sway angle measured by the camera along the x, y, and z axes, respectively. , and These are the components of the base sway angle measured by the IMU along the x, y, and z axes, respectively.

[0041] After the inner loop angular velocity is optimally estimated using Kalman filtering, it is used in the kinematic prediction model of the outer loop to improve prediction accuracy and further filter out noise. In specific implementation, there are... .

[0042] Based on this Kalman filter algorithm, this invention designs a dual-loop Kalman filter. The dual-loop Kalman filter is used to fuse the motion information of the robotic arm base after time delay matching to obtain an estimate of the robotic arm base's sway angle. Based on this estimate, the joint target position of the underwater robotic arm is calculated, thereby controlling the underwater robotic arm's operation and achieving sway compensation. Specifically: First, the first motion information of the robotic arm base after time delay matching is... Second motion information The inner loop Kalman filter is used to fuse the values ​​to obtain an estimate of the swaying angular velocity of the robotic arm base. Then, the estimated value of the swaying angular velocity of the robotic arm base is processed by the outer loop Kalman filter to obtain an estimate of the swaying angle of the robotic arm base. This achieves the effect of joint optimization estimation of the inner and outer loops and eliminates interference such as high-frequency noise.

[0043] 11. From step 9, the sway estimation of the robotic arm base can be obtained through multi-sensor spatiotemporal heterogeneous matching based on Kalman filtering. Let the transformation matrix from world coordinates to robotic arm end-effector coordinates be... The transformation matrix from the robot arm base to the robot arm end effector is: It can be calculated through the forward kinematics model of the robotic arm; the transformation matrix from world coordinates to the initial pose of the robotic arm base is: The robotic arm base wobbles. .

[0044] During the sway compensation control process, the estimated value of the sway angle of the robotic arm base is... and the preset reference signal of the underwater robotic arm input by the operator The control law for robotic arm sway compensation is obtained by superimposing the results. as follows: in, This is the transformation matrix from the world coordinate system to the initial pose of the robot arm base.

[0045] Estimated value of the sway angle of the robotic arm base The robotic arm base pose before sway compensation is considered stationary. To the robotic arm base position after sway compensation The transformation matrix; the preset reference signal of the underwater robotic arm. The target pose of the underwater robotic arm's end effector relative to the world coordinate system is defined as the coordinate transformation matrix from the world coordinate system to the end effector; the robotic arm sway compensation control law is also defined. The coordinate transformation matrix of the robotic arm required for sway compensation.

[0046] The robotic arm sway compensation control law After performing inverse kinematics solution, the compensation control quantity is obtained as the joint target position and sent to the joint motor of the underwater robotic arm for control, thereby realizing sway compensation.

[0047] like Figure 1 The diagram shows the control system of an underwater electric robotic arm without sway compensation control. Without sway compensation control, the end effector motion is the sum of the base sway and the robotic arm's own motion. ,Right now: The control objective is to maintain the pose of the robotic arm's end effector, i.e., eliminate the effects of base sway. The operator-inputted reference signal is superimposed, and then inverse kinematics calculations are performed to obtain the target joint position. The aforementioned sway compensation control law essentially adds an uncontrollable virtual joint to the robotic arm's base. Through motion sensing and multi-sensor spatiotemporal heterogeneous matching algorithms, as described above, the observation of this virtual joint is achieved.

[0048] like Figure 3 The image shows the base sway trajectory of the multi-sensor spatiotemporal heterogeneous matching algorithm used in the experimental verification of this invention. It includes the base sway curve (rotation vector value) directly measured by the IMU, the base sway curve calculated using the transformation matrix of the eye-in-the-hand monocular camera and the robotic arm, and the base sway estimation curve obtained after fusing the two sensors. Figure 4 The figure shows the end-effector sway compensation in the experimental verification of this invention, including the base sway curve and the end-effector sway curve. When a sway excitation is input to the base without operator input, the measured end-effector sway amplitude is reduced relative to the base, verifying the effectiveness of the sway compensation control law designed in this invention.

[0049] The above content is merely a technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for sway compensation control of an underwater robotic arm based on spatiotemporal heterogeneous matching of sensors, characterized in that, include: Step 1: Underwater images are acquired using an eye-to-hand monocular camera at the end of the underwater robotic arm, thereby obtaining motion information of the robotic arm's end relative to the camera coordinate system of the eye-to-hand monocular camera. This motion information of the robotic arm base relative to the camera coordinate system of the eye-to-hand monocular camera is then calculated. Simultaneously, the inertial measurement unit (IMU) on the robotic arm base obtains motion information of the robotic arm base relative to the IMU coordinate system, enabling the observation of the rotation and sway of the robotic arm base. Step 2: Perform coordinate system heterogeneous matching between the motion information of the robotic arm base relative to the camera coordinate system of the eye-to-hand monocular camera and the motion information of the robotic arm base relative to the coordinate system of the inertial measurement unit (IMU) to obtain the motion information of the robotic arm base after spatial matching. Step 3: Perform time heterogeneous matching on the motion information of the robotic arm base relative to the world coordinate system after spatial matching to obtain the motion information of the robotic arm base after time delay matching; The fourth step is to use a dual-loop Kalman filter to fuse the motion information of the robotic arm base after time delay matching, obtain an estimate of the sway angle of the robotic arm base, and calculate the joint target position of the underwater robotic arm based on the estimate of the sway angle of the robotic arm base, thereby controlling the operation of the underwater robotic arm and realizing the sway compensation of the underwater robotic arm. In the third step, the time is obtained. The motion information of the robotic arm base after spatial matching is obtained, and then the relative time delay between the eye and the hand monocular camera and the inertial measurement unit (IMU) is obtained through the cross-correlation function. By using feedforward compensation to make up for time delay, motion information of the robotic arm base after time delay matching is obtained, including the first motion information of the robotic arm base after time delay matching. Second motion information ; In the fourth step, the first motion information of the robotic arm base after time delay matching is first... Second motion information The estimated value of the swaying angular velocity of the robotic arm base is obtained by fusing the inner loop Kalman filter, and then the estimated value of the swaying angular velocity of the robotic arm base is processed by the outer loop Kalman filter to obtain the estimated value of the swaying angle of the robotic arm base. In the fourth step, during the sway compensation control process, the estimated value of the sway angle of the robotic arm base is... and the preset reference signal of the underwater robotic arm The control law for robotic arm sway compensation is obtained by superimposing the results. as follows: ； in, This is the transformation matrix from the world coordinate system to the initial pose of the robotic arm base; Estimated value of the sway angle of the robotic arm base The robotic arm base pose before sway compensation To the robotic arm base position after sway compensation The transformation matrix; the preset reference signal of the underwater robotic arm. The target pose of the underwater robotic arm's end effector relative to the world coordinate system; The robotic arm sway compensation control law After performing inverse kinematics solution, the compensation control quantity is obtained as the joint target position and sent to the joint motor of the underwater robotic arm for control, thereby realizing sway compensation.

2. The underwater robotic arm sway compensation control method based on sensor spatiotemporal heterogeneous matching according to claim 1, characterized in that: In the first step, based on the underwater images acquired by the eye-on-hand monocular camera, the Visual Real-Time Localization and Mapping (VSLAM) algorithm is used to obtain the six-degree-of-freedom motion estimation of the eye-on-hand monocular camera, thereby obtaining the motion information of the robotic arm's end effector relative to the camera coordinate system of the eye-on-hand monocular camera. This refers to the pose of the robotic arm's end effector, which in turn provides motion information about the robotic arm's base relative to the camera coordinate system of the eye-to-hand monocular camera. as follows: ； in, This is the coordinate transformation matrix from the robot arm base to the end effector; The rotational motion of the robotic arm base is directly measured by an inertial measurement unit (IMU), thereby obtaining the motion information of the robotic arm base relative to the IMU coordinate system. .

3. The underwater robotic arm sway compensation control method based on sensor spatiotemporal heterogeneous matching according to claim 1, characterized in that: In the second step, the camera coordinate system of the eye-on-hand monocular camera or the IMU coordinate system is used as the world coordinate system, thereby transmitting the motion information of the robotic arm base relative to the camera coordinate system of the eye-on-hand monocular camera. Motion information of the robotic arm base relative to the inertial measurement unit (IMU) coordinate system The mean space is matched to the world coordinate system as follows: ； ； in, and These are the first and second motion information of the robotic arm base after spatial matching, i.e., the motion information of the robotic arm base after spatial matching; This is the transformation matrix from the world coordinate system to the camera coordinate system; This is the transformation matrix from the world coordinate system to the inertial measurement unit (IMU) coordinate system.

Images