A self-calibration system for a rigid-link-flexible-continuum hybrid robot arm and a method of implementing the same

Abstract

This invention discloses a self-calibration system and implementation method for a rigid-link-flexible continuum hybrid robotic arm, wherein the real-time pose transpose matrix of the hybrid robotic arm end effector is combined with the joint angle dataset {θ} of the robotic arm. k}, Bending deformation angle dataset of flexible continuum {θ l}, obtain the real-time pose matrix, and compensate for it to obtain the compensated pose matrix {P}. r}, will {θ k}、{θ l}、{P r By combining these elements, we obtain the hybrid visual servoing configuration space-operation space dataset X. t X t After standardization, we get X′ t , X′ t Hand-eye transposition calculation error ΔM f Gravity attitude error ΔM g As input to the recurrent neural network, after training iterations, the hand-eye transpose composite error prediction model M is obtained. R , by M R Obtain the joint angle correction dataset {θ′ k}, Bending deformation angle correction dataset {θ′ l}, the {θ′ k}、{θ′ l The data is aligned using timestamp information and then sent separately to the rigid linkage robotic arm and the multi-rope driven flexible continuum for calibration and adjustment. This invention can significantly improve the calibration accuracy and speed of the rigid linkage-flexible continuum hybrid robotic arm.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots, in particular to a self-calibration system of a rigid link-flexible continuum hybrid manipulator and an implementation method thereof. BACKGROUND

[0002] Flexible continuum has the characteristics of flexibility and continuous configuration, which can realize complex motion mode and attitude adjustment, provide higher flexibility and control ability, and can adapt to complex and unstructured scenes. Compared with traditional rigid arms, the dynamic model of flexible continuum cannot be accurately measured and calibrated due to the nonlinear and multi-degree-of-freedom characteristics of the traditional rigid body dynamics, because of the continuous deformation characteristics. The motion data of flexible continuum is dependent on multi-source and large amount of sensor data input, which leads to the fact that the existing traditional calibration method of manipulator cannot be used for the calibration of flexible continuum.

[0003] The rigid link-flexible continuum hybrid manipulator retains the high adaptability and flexibility of the flexible continuum while providing accurate positioning and control capability through the rigid link, that is, it expands the working space of the manipulator and can also operate in a narrow or complex environment. When modeling the dynamics and kinematics characteristics of the rigid link and the flexible continuum, the rigid link-flexible continuum hybrid manipulator needs to process the sensor data of the hybrid manipulator in real time, which leads to high real-time calculation requirement. At the same time, the calibration of the hybrid manipulator needs to be calibrated separately for each part, and the relative position and attitude between the parts need to be coordinated. This process is complex and time-consuming, and in high-precision applications, the calibration accuracy of the hybrid manipulator is required to be higher. As a necessary link before operation, the fast and autonomous calibration of the hybrid manipulator can help to improve its use efficiency, reduce the use bottleneck of the hybrid manipulator, and to a certain extent, reduce the accuracy error of the hybrid manipulator. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application provides a self-calibration system of a rigid link-flexible continuum hybrid manipulator and an implementation method thereof.

[0005] The present application achieves the above technical purpose by the following technical means.

[0006] A self-calibration system of a rigid link-flexible continuum hybrid manipulator, comprising:

[0007] An ArUco combined marker module, comprising n sets of flexible continuum ArUco marker modules and an arm external reference ArUco marker module, the flexible continuum ArUco marker modules are sequentially pasted to the surface of the integrated joint of the multi-cable driven flexible continuum, and the arm external reference ArUco marker module is arranged within the capture range of the field of view of the depth vision camera in the "eye outside hand" configuration.

[0008] The mixed visual perception architecture module includes a "hand-in-eye" configuration depth vision camera and a "hand-out-eye" configuration depth vision camera, the "hand-in-eye" configuration depth vision camera is vertically installed above the end effector, and the "hand-out-eye" configuration depth vision camera is installed close to the rigid link-flexible continuum hybrid robot arm, the "hand-in-eye" configuration depth vision camera is used to complete the identification and positioning of the arm-out-reference ArUco marker module, and the "hand-out-eye" configuration depth vision camera is used to complete the identification and positioning of the flexible joint continuum ArUco marker module and the arm-out-reference ArUco marker module respectively.

[0009] The control computer communicates with the "hand-in-eye" configuration depth vision camera, the "hand-out-eye" configuration depth vision camera, and the rigid link-flexible continuum hybrid robot arm.

[0010] In the technical scheme, the rigid link-flexible continuum hybrid robot arm is composed of a rigid link robot arm, a multi-cable driven flexible continuum, and an end effector connected in sequence, the rigid link robot arm is composed of a plurality of rigid link joint modules connected in series, and the multi-cable driven flexible continuum is composed of n sets of integrated joints connected in series by driving ropes.

[0011] An implementation method of a self-calibration system of a rigid link-flexible continuum hybrid robot arm is provided.

[0012] The "hand-in-eye" configuration depth vision camera is used to collect position and attitude information of a plurality of arm-out-reference ArUco marker modules, and the "hand-out-eye" configuration depth vision camera is used to collect position and attitude information of a plurality of flexible continuum ArUco marker modules, the position and attitude information are obtained by a "hand-in-hand-out" mixed visual synchronous measurement method, and a rigid link-flexible continuum hybrid robot arm end real-time pose transpose matrix P is obtained. h P;

[0013] A plurality of cable-driven flexible continuum bending deformation angle data sets {θ l} are obtained from real-time position and attitude information of the next period flexible continuum ArUco marker module, a plurality of rigid link robot arm joint angle data sets {θ k} are obtained from joint angle information of the next period rigid link robot arm, the plurality of rigid link robot arm joint angle data sets {θ k}, the plurality of cable-driven flexible continuum bending deformation angle data sets {θ l} are input into the P, and a rigid link-flexible continuum hybrid robot arm end real-time pose matrix {P h} is obtained. e ;

[0014] A hand-eye transpose error compensation threshold Δn t is input into the {Pe} and compensating the position information in the position information, and then adding a driving rope compensation torque τ r to the driving rope control torque to obtain a rigid link-flexible continuum hybrid manipulator end compensation pose matrix {P r};

[0015] The {θ k}, {θ l}, and {P r} are combined to obtain a hybrid visual servo configuration space-operation space data set X t , and the data set is structurally standardized to obtain a hybrid visual servo configuration space-operation space data matrix X′ t ;

[0016] The X′ t , an eye-in-hand transpose calculation error ΔM f , and a gravity pose error ΔM g are taken as inputs of a recurrent neural network model, and the recurrent neural network is iterated ε times to obtain an eye-in-hand transpose compound error prediction model M R ; X′ t of the next time period is input into the M R , and joint angle correction data set {θ′ k} of the rigid link manipulator and bending deformation angle correction data set {θ′l} of the multi-rope driven flexible continuum are output.

[0017] The {θ′k} and {θ′l} are aligned with timestamp information, and are then respectively sent to the rigid link manipulator and the multi-rope driven flexible continuum. The rigid link manipulator continuously adjusts the joint angle according to the corresponding time sequence, and the multi-rope driven flexible continuum continuously adjusts the bending deformation angle according to the corresponding time sequence, so that the driving rope is stretched / tensioned to realize bending angle correction adjustment, and finally the rigid link-flexible continuum hybrid manipulator is rapidly and autonomously calibrated.

[0018] Further, before the rigid link-flexible continuum hybrid manipulator is calibrated, forward and inverse kinematics models of the rigid-flexible hybrid manipulator are established, including first establishing a rigid link manipulator kinematics model K , a flexible continuum kinematics model K soft , and then establishing a rigid link-flexible continuum hybrid manipulator multivariable kinematics model K com according to the installation mode of the rigid link manipulator and the multi-rope driven flexible continuum.

[0019]

[0020] where C is a cosine function, S is a sine function, θ kFor the Z-axis of the rigid link joint module K-1 k-1 Axis, from X k-1 Rotate to X k Angle; α k For X k Axis, from Z k-1 Rotate to Z k Angle; a i For rigid link joint module K along X k Axis, from Z k-1 Move to Z k distance; d k For rigid link joint module K along Z k-1 The axis, from the origin O k-1 The distance to the common perpendicular; k is the number of rigid link joint modules, L i Let ω be the length of the multi-rope driven flexible continuum. i θ is the deflection angle of a multi-rope driven flexible continuum. i The bending angle of a multi-rope driven flexible continuum.

[0021] Furthermore, by employing a hybrid visual synchronous measurement sub-method of "hand-to-hand" and "external-hand" approaches, the real-time pose transpose matrix of the end effector of the rigid link-flexible continuum hybrid robotic arm is obtained. h P, specifically:

[0022] h P = w T e · n T e · b T e · w T K · C T h -1

[0023] in, w T e The pose matrix of the ArUco marker module, an external reference for the arm, acquired by a depth vision camera, is configured for the "eye outside the hand" configuration. n T e The pose matrix of the ArUco marker module, a flexible continuum acquired by a depth vision camera, is configured for the "eye outside the hand" configuration. b T e This is a transpose matrix model between rigid link joint modules. w T K The pose transpose quasi-matrix of the base coordinate system of the "eye outside the hand" manipulator is configured with a depth vision camera and a rigid link-flexible continuum hybrid manipulator. C T hThe pose matrix of the arm-external reference ArUco marker module identified by the "eye-in-hand" configuration depth vision camera.

[0024] Further, the hand-eye transpose computation error ΔM f is:

[0025]

[0026] wherein, is the transformation matrix from the robot base coordinate system to the arm-external reference ArUco marker module, is the transformation matrix from the robot base coordinate system to the "eye-in-hand" configuration depth vision camera, is the transformation matrix from the "eye-in-hand" configuration depth vision camera to the arm-external reference ArUco marker module.

[0027] Further, the gravity attitude error ΔM g is:

[0028]

[0029] wherein, K is the driving rope stiffness coefficient of the multi-rope driven flexible continuum, J T is the Jacobian matrix of the multi-rope driven flexible continuum, g is the gravity acceleration.

[0030] Further, the hand-eye transpose error compensation threshold Δn t is:

[0031]

[0032] wherein, σ f are the error mean and standard deviation of the hand-eye transpose computation error ΔM f , respectively, and δ is a constant coefficient.

[0033] Further, the driving rope compensation torque τ r is:

[0034] τ r = K g · ΔM g

[0035]

[0036] wherein, K g is the torque control gain matrix, k ij represents the influence coefficient of the i-th component of the gravity attitude error on the j-th driving rope torque.

[0037] Further, the hybrid visual servoing configuration space-operation space dataset Xt For:

[0038] X t = [{theta k}‖{theta l}‖{{P r}}]

[0039] Wherein, || represents data separation.

[0040] The present application has the beneficial effects that: the present application solves two key technical difficulties of rigid link-flexible continuum hybrid manipulator structure multi-dimensional calculation cumbersome and rigid-flexible hybrid manipulator model precision insufficient, realizes "hand on-hand out" hybrid visual synchronous measurement through hybrid visual perception architecture module, greatly improves the rigid link-flexible continuum hybrid manipulator end real-time pose transpose matrix calculation precision; through calculating hand-eye transpose calculation error and gravity attitude error, establishing hand-eye transpose error compensation threshold and driving rope compensation torque, reducing the attitude error under the gravity disturbance of the kinematic model transpose calculation error; through constructing the hand-eye transpose compound error prediction model based on recurrent neural network, directly obtaining the joint angle correction data set of the rigid link manipulator and the bending deformation angle correction data set of the multi-rope driven flexible continuum, aligning through timestamp information, the joint angle of the rigid link-flexible continuum hybrid manipulator can be directly corrected, and the rapid autonomous calibration of the rigid link-flexible continuum hybrid manipulator is realized. The present application can greatly improve the rigid link-flexible continuum hybrid manipulator calibration precision and speed. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 It is the self-calibration system structure schematic diagram of the rigid link-flexible continuum hybrid manipulator of the present application;

[0042] Figure 2 It is the local structure schematic diagram of the multi-rope driven flexible continuum of the present application;

[0043] Figure 3 It is the deformation bending angle schematic diagram of the multi-rope driven flexible continuum of the present application;

[0044] Figure 4 It is the self-calibration implementation method logic diagram of the rigid link-flexible continuum hybrid manipulator of the present application;

[0045] Figure 5 It is the ArUco mark code diagram of the rigid link-flexible continuum hybrid manipulator of the present application;

[0046] Figure 6 It is the rigid link-flexible continuum hybrid manipulator hand-eye transpose calculation error prediction recurrent neural network structure diagram of the present application;

[0047] In the figure, 1. rigid link mechanical arm, 2. multi-rope driven flexible continuum, 3. ArUco combined marker module, 4. hybrid visual perception architecture module, 5. end effector, 6. rigid link joint module, 7. flexible continuum ArUco marker module, 8. "eye on hand" configuration depth vision camera, 9. arm external reference ArUco marker module, 10. "eye off hand" configuration depth vision camera. DETAILED DESCRIPTION

[0048] The application will be further described below in conjunction with the drawings and specific embodiments, but the scope of protection of the application is not limited thereto.

[0049] As shown in the figure, the self-calibration system of a rigid link-flexible continuum hybrid manipulator according to the application comprises an ArUco combined marker module 3, a hybrid visual perception architecture module 4 and a control computer. Figure 1 The rigid link-flexible continuum hybrid manipulator therein is composed of a rigid link mechanical arm 1, a multi-rope driven flexible continuum 2 and an end effector 5 connected in sequence; the rigid link mechanical arm 1 is composed of a plurality of rigid link joint modules 6 connected in series; the multi-rope driven flexible continuum 2 is composed of n groups of integrated joints connected in series by driving ropes, as shown in the figure.

[0050] Figure 2

[0051] The hybrid visual perception architecture module 4 comprises a "eye on hand" configuration depth vision camera 8 and a "eye off hand" configuration depth vision camera 10; the "eye on hand" configuration depth vision camera 8 is installed vertically above the end effector 5, and the installation height is d cam ; the "eye off hand" configuration depth vision camera 10 is installed close to the rigid link-flexible continuum hybrid manipulator.

[0052] The ArUco combined marker module 3 comprises n groups of flexible continuum ArUco marker modules 7 and an arm external reference ArUco marker module 9; the flexible continuum ArUco marker modules 7 are sequentially pasted to the surfaces of the integrated joints of the multi-rope driven flexible continuum 2; the arm external reference ArUco marker module 9 is arranged within the capture range of the field of view of the "eye off hand" configuration depth vision camera 10; the "eye on hand" configuration depth vision camera 8 is used to complete the identification and positioning of the arm external reference ArUco marker module 9; the "eye off hand" configuration depth vision camera 10 is used to complete the identification and positioning of the flexible joint continuum ArUco marker modules 7 and the arm external reference ArUco marker module 9, respectively. The ArUco marker codes of the continuum ArUco marker modules 7 and the arm external reference ArUco marker module 9 are shown in the figure. Figure 5

[0053] ​​​The control computer communicates with the "eye on the hand" equipped with a depth vision camera 8, the "eye outside the hand" equipped with a depth vision camera 10, and the rigid link-flexible continuum hybrid robotic arm.

[0054] like Figure 4 As shown, a self-calibration method for a rigid link-flexible continuum hybrid robotic arm includes the following steps:

[0055] Step 1: The rigid link-flexible continuum hybrid robotic arm is powered on and started. The rigid link-flexible continuum hybrid robotic arm and the hybrid vision perception architecture module 4 are initialized.

[0056] Step 2: In the hybrid vision perception architecture module 4, the "eye on the hand" configuration depth vision camera 8 acquires multiple sets of position and orientation information of the arm-external reference ArUco marker module 9, and the "eye outside the hand" configuration depth vision camera 10 acquires multiple sets of position and orientation information of the flexible continuum ArUco marker module 7. This position and orientation information is transmitted to the control computer, and the real-time pose transpose matrix of the rigid link-flexible continuum hybrid robotic arm end effector is obtained using the "hand-outside-hand" hybrid vision synchronous measurement sub-method. h P;

[0057] Step 3: Using a depth vision camera 10, the real-time position and attitude information of the ArUco marker module 7 for the next time period of the flexible continuum is acquired and converted to the multi-rope driven flexible continuum base coordinate system. This information is then used to determine the position and attitude based on the change in the length of the driving rope, ΔL. i By performing inverse kinematics solutions, the bending deformation angle dataset {θ} of the multi-rope driven flexible continuum 2 is obtained. l The control computer directly acquires the joint angle information of the rigid linkage manipulator 1 for the next time period and organizes it into a joint angle dataset {θ} for the rigid linkage manipulator 1. k}; The joint angle dataset {θ} of the rigid linkage robotic arm 1 k Data set of bending deformation angles of multi-rope driven flexible continuum 2 {θ} l Import the real-time pose transpose matrix of the rigid link-flexible continuum hybrid robotic arm end effector from step two. h P, obtains the real-time pose matrix of the end effector of the rigid link-flexible continuum hybrid robotic arm {P}. e};

[0058] Step 4: Calculate the hand-eye transpose calculation error ΔM f The mean error (generated during data acquisition, calculation, and calibration by the rigid link-flexible continuum hybrid robotic arm) With standard deviation σ f The hand-eye transpose error compensation threshold Δn is set by adding δ times the standard deviation to the mean error. t; define the torque control gain matrix K g , the driving rope compensation torque τ r is obtained by multiplying the torque control gain matrix K g and the gravity attitude error ΔM g ; the hand-eye transpose error compensation threshold Δn t is input into the rigid link-flexible continuum hybrid manipulator end real-time pose matrix {P e}, the position information in it is compensated, and the end pose matrix at this time is {P′ e}, and the driving rope compensation torque τ r is superimposed into the driving rope control torque for compensation, and the end pose matrix at this time becomes {P″ e}, that is, the rigid link-flexible continuum hybrid manipulator end compensation pose matrix {P r};

[0059] Step five: combine the joint angle data set {θ k} of the rigid link manipulator 1, the bending deformation angle data set {θ l} of the multi-rope driven flexible continuum 2, and the rigid link-flexible continuum hybrid manipulator end compensation pose matrix {P r} to obtain the hybrid visual servo configuration space-operation space data set X t , and complete the structure standardization of the data set to obtain the hybrid visual servo configuration space-operation space data matrix X′ t ;

[0060] Step six: use the hand-eye transpose calculation error predictor method based on the recurrent neural network to input the hybrid visual servo configuration space-operation space data matrix X′ t , the hand-eye transpose calculation error ΔM f , and the gravity attitude error ΔM g into the recurrent neural network model, train for iterations ε times, obtain the hand-eye transpose compound error prediction model M R based on the recurrent neural network, input the next period of hybrid visual servo configuration space-operation space data matrix X′ t into the hand-eye transpose compound error prediction model M R based on the recurrent neural network, and output the joint angle correction data set {θ′ k} of the rigid link manipulator 1 and the bending deformation angle correction data set {θ′ l} of the multi-rope driven flexible continuum 2;

[0061] Step seven: combine the joint angle correction data set {θ′ kThe bending deformation angle correction data set {θ'1} of the multi-rope driven flexible continuum 2 is aligned with the time stamp information, the two types of data are time synchronized, and then the control computer sends the two types of data to the rigid link mechanical arm 1 and the multi-rope driven flexible continuum 2 respectively. The rigid link mechanical arm 1 performs continuous data correction adjustment of the joint angle in the corresponding time sequence, and the multi-rope driven flexible continuum 2 performs continuous data correction adjustment of the bending deformation angle in the corresponding time sequence, so that the driving rope performs tension / extension and contraction, the bending angle correction adjustment is realized, and finally the rapid autonomous calibration of the rigid link-flexible continuum hybrid mechanical arm is realized.

[0062] Before self-calibration of the rigid link-flexible continuum hybrid mechanical arm, a rigid-flexible hybrid mechanical arm forward and inverse kinematics model is established, specifically a rigid link mechanical arm kinematics model is first established A flexible continuum kinematics model K soft Then, according to the installation mode of the rigid link mechanical arm 1 and the multi-rope driven flexible continuum 2, a rigid link-flexible continuum hybrid mechanical arm multivariable kinematics model K is established com The multivariable kinematics model is calculated by the following formula:

[0063]

[0064] Wherein, C is the cosine function cos; S is the sine function sin; θ k is the angle of rotation around the Z k-1 axis of the rigid link joint module K-1 from X k-1 to X k ; α k is the angle of rotation around the X k axis from Z k-1 to Z k ; a i is the distance of the rigid link joint module K along the X k axis from Z k-1 to Z k ; d k is the distance of the rigid link joint module K along the Z k-1 axis from the origin O k-1 to the common perpendicular; k is the number of rigid link joint modules; L i is the length of the multi-rope driven flexible continuum 2, and L i =n·Δl i , n is the number of integrated joints, and Δl i is the length of the integrated joint; ω i is the deflection angle of the multi-rope driven flexible continuum 2, and θ i is the deformation bending angle of the multi-rope driven flexible continuum 2. The deformation bending angle of the multi-rope driven flexible continuum is shown in Figure 3 .

[0065] As Figure 1 shown, the "hand-on-hand-off" hybrid vision synchronization measurement sub-method, in particular:

[0066] The "eye-on-hand" configured depth vision camera 8 identifies the pose matrix of the exo-reference ArUco marker module 9 acquired C T h , point-multiply the real-time pose of the rigid-link-flexible-continuum hybrid manipulator end-effector transpose matrix h P, to get the exo-reference ArUco marker module 9 to rigid-link-flexible-continuum hybrid manipulator pose transpose matrix A P;

[0067] The "eye-off-hand" configured depth vision camera 10 acquires the pose matrix of the exo-reference ArUco marker module 9 w T e , the "eye-off-hand" configured depth vision camera 10 acquires the pose matrix of the flexible-continuum ArUco marker module 7 n T e , joint the transpose matrix model between the rigid-link joint module 6 b T e , point-multiply the three, to get the exo-reference ArUco marker module 9 to rigid-link-flexible-continuum hybrid manipulator pose transpose matrix B P;

[0068] From the exo-reference ArUco marker module 9 to rigid-link-flexible-continuum hybrid manipulator pose transpose matrix A P of "hand-on" vision and the exo-reference ArUco marker module 9 to rigid-link-flexible-continuum hybrid manipulator pose transpose matrix B P of "hand-off" vision, get the rigid-link-flexible-continuum hybrid manipulator end-effector real-time pose transpose matrix h P calculation process as follows:

[0069] A P= C T h · h P

[0070] B P= w T e · n T e · b T e

[0071] A P= B P· w T K

[0072] h P= w T e · n T e · b T e · w T K · C T h -1

[0073] wherein, w T K is the pose transpose pseudo-matrix of the transformation matrix of the depth vision camera 10 in the "eye-in-hand" configuration to the rigid link-flexible continuum hybrid robot base coordinate system.

[0074] In step four, the control computer acquires the transformation matrix of the robot base coordinate system to the ArUco marker module 9 outside the arm minus the transformation matrix of the robot base coordinate system to the depth vision camera 8 in the "eye-in-hand" configuration and the transformation matrix of the depth vision camera 8 in the "eye-in-hand" configuration to the ArUco marker module 9 outside the arm product, obtaining the hand-eye transpose calculation error ΔM f ;

[0075] The depth vision camera 10 in the "eye-in-hand" configuration acquires the real-time position and attitude information of the flexible continuum ArUco marker module 7 in real time, and according to the driving rope length change amount ΔL i solves the inverse kinematics to obtain the bending deformation angle θ of the multi-rope driven flexible continuum 2 l , combined with the length L of the multi-rope driven flexible continuum 2 i , the driving rope stiffness coefficient of the multi-rope driven flexible continuum 2, to obtain the gravity attitude error ΔM g ;

[0076] The hand-eye transpose calculation error ΔM f and the gravity attitude error ΔM g are calculated by the following formula:

[0077]

[0078] wherein K is the driving rope stiffness coefficient of the multi-rope driven flexible continuum 2, J T is the Jacobian matrix of the multi-rope driven flexible continuum 2, and g is the gravitational acceleration.

[0079] The error mean f and the standard deviation σ of the hand-eye transpose calculation error ΔM are calculated.f , the hand-eye transformation error compensation threshold Δn is set by adding the error mean value to the standard deviation multiplied by δ t , the calculation process is as follows:

[0080]

[0081] The driving rope compensation torque τ r is obtained by multiplying the torque control gain matrix K g and the gravity attitude error ΔM g , wherein the torque control gain matrix K g is multiplied by the driving rope compensation torque τ r , which is calculated by the following formula:

[0082]

[0083] τ r = K g · ΔM g

[0084] wherein k ij represents the influence coefficient of the i-th component of the gravity attitude error on the j-th driving rope torque.

[0085] In step five, three types of data are combined to establish a hybrid visual servoing configuration space-operation space data set X t , and standardized to obtain a hybrid visual servoing configuration space-operation space data matrix X' t , which is calculated by the following formula:

[0086] X t = [{θ k}‖{θ l}‖{{P r}}]

[0087]

[0088] wherein || represents data separation, μ k is the data matrix mean value, and σ k is the data matrix standard deviation.

[0089] In step six, the hand-eye transformation calculation error predictor method based on the recurrent neural network is specifically: the hybrid visual servoing configuration space-operation space data matrix X' after standardization structure transformation t is taken as the input information A1 of the recurrent neural network model, the hand-eye transformation calculation error ΔM f and the gravity attitude error ΔM g are taken as the input information A2, and the hand-eye transformation calculation error ΔM Figure 6As shown, the recurrent neural network model inputs information A1, A2 according to the time stamp to complete the time sequence annotation in different time states, and takes α% in the data matrix X' t as the training set, β% in the data matrix X' t as the verification set, and γ% in the data matrix X' t as the test set, trains for ε times, and obtains the hand-eye transpose compound error prediction model M R based on the recurrent neural network; the mixed visual servoing configuration space-operation space data matrix X′ t of the next period is input into the hand-eye transpose compound error prediction model M R based on the recurrent neural network, and output information B1-the joint angle correction data set {θ′ k} of the rigid link mechanical arm 1, and output information B2-the bending deformation angle correction data set {θ′ l} of the multi-rope driven flexible continuum 2. See Figure 6 .

[0090] The embodiments are preferred embodiments of the present application, but the present application is not limited to the above embodiments, and any obvious improvements, replacements or modifications made by those skilled in the art without departing from the essential content of the present application shall fall within the protection scope of the present application.

Claims

1. An implementation method of a self-calibration system of a rigid-link-flexible-continuum hybrid robotic arm, characterized in that, The self-calibration system comprises: The ArUco combined marker module (3) comprises n groups of flexible continuum ArUco marker modules (7) and an arm-external reference ArUco marker module (9), the flexible continuum ArUco marker modules (7) are sequentially and orderly pasted to the integrated joint surfaces of the multi-rope driven flexible continuum (2), and the arm-external reference ArUco marker module (9) is arranged within the field-of-view capture range of the "eye-out-of-hand" configuration depth vision camera (10); The mixed visual perception architecture module (4) comprises a "eye-on-hand" configuration depth vision camera (8) and a "eye-out-of-hand" configuration depth vision camera (10), the "eye-on-hand" configuration depth vision camera (8) is vertically installed above the end effector (5), and the "eye-out-of-hand" configuration depth vision camera (10) is installed close to the rigid link-flexible continuum hybrid robot arm, the "eye-on-hand" configuration depth vision camera (8) is used for completing the identification and positioning of the arm-external reference ArUco marker module (9), and the "eye-out-of-hand" configuration depth vision camera (10) is used for completing the identification and positioning of the flexible joint continuum ArUco marker module (7) and the arm-external reference ArUco marker module (9) respectively; The rigid link-flexible continuum hybrid robot arm is composed of a rigid link robot arm (1), a multi-rope driven flexible continuum (2) and an end effector (5) connected in sequence. The implementation method is: The position and posture information of the plurality of arm external reference ArUco marker modules (9) is collected by the depth vision camera (8) configured as "eye on hand", and the position and posture information of the plurality of flexible continuum ArUco marker modules (7) is collected by the depth vision camera (10) configured as "eye off hand". The above-mentioned position and posture information is obtained through the "hand-on-hand-off" hybrid vision synchronous measurement sub-method, and the real-time pose transformation matrix of the rigid link-flexible continuum hybrid manipulator is obtained h P, specifically: h P= 0.0001 w T e · n T e · b T e · w T K · C T h -1 wherein, w T e is the pose matrix of the extrinsic reference ArUco marker module (9) acquired by the depth vision camera (10) configured "eye-in-hand", n T e is the pose matrix of the flexible continuum ArUco marker module (7) acquired by the depth vision camera (10) configured "eye-in-hand", b T e is the transpose matrix model between the rigid link joint modules (6), w T K is the pose transpose pseudo-matrix of the depth vision camera (10) configured "eye-in-hand" and the rigid link-flexible continuum hybrid robot base coordinate system, C T h is the pose matrix of the extrinsic reference ArUco marker module (9) recognized acquired by the depth vision camera (8) configured "eye-in-hand"; The real-time position and posture information of the next period flexible continuum ArUco marker module (7) is obtained, and the bending deformation angle data set {θ l} of the multi-rope driven flexible continuum (2) is obtained; the joint angle information of the next period rigid link mechanical arm (1) is obtained, and the joint angle data set {θ k} of the rigid link mechanical arm (1) is obtained; the joint angle data set {θ k} of the rigid link mechanical arm (1) and the bending deformation angle data set {θ l} of the multi-rope driven flexible continuum (2) are imported into the h P, and the real-time pose matrix {P e} of the rigid link-flexible continuum hybrid mechanical arm end is obtained; Compensate the hand-eye transformation error threshold Δn t Input to the {P e}, compensate the position information in it, and then superimpose the driving rope compensation torque τ r Into the driving rope control torque to compensate, get the rigid link-flexible continuum hybrid manipulator end compensation pose matrix {P r}; {θ k}、{θ l}、{P r} are combined to obtain a hybrid visual servoing configuration space-operation space dataset X t , and the structure standardization of the dataset is completed to obtain a hybrid visual servoing configuration space-operation space data matrix X' t ; X′ t , hand-eye transformation calculation error ΔM f and gravity attitude error ΔM g As the input of the recurrent neural network model, the training iteration is performed for ε times to obtain a hand-eye transformation compound error prediction model M based on the recurrent neural network R ; then input X′ t of the next period into the M R , and output the joint angle correction data set {θ′ k} of the rigid link robot arm (1) and the bending deformation angle correction data set {θ′l} of the multi-rope driven flexible continuum (2). The {θ′ k}、{θ′ l} are aligned with the timestamp information, and are respectively sent to the rigid link mechanical arm (1) and the multi-rope driven flexible continuum (2). The rigid link mechanical arm (1) performs continuous data correction adjustment of the joint angle in the corresponding time sequence, and the multi-rope driven flexible continuum (2) performs continuous data correction adjustment of the bending deformation angle in the corresponding time sequence, so that the driving rope performs tension / extension and retraction, realizes bending angle correction adjustment, and finally realizes rapid autonomous calibration of the rigid link-flexible continuum hybrid mechanical arm.

2. The implementation method of claim 1, wherein, Before self-calibration, the rigid-flexible hybrid manipulator establishes forward and inverse kinematics model, including establishing rigid link manipulator kinematics model Flexible continuum kinematics model K soft According to the installation mode of rigid link manipulator (1) and multi-rope driven flexible continuum (2), the rigid-flexible hybrid manipulator multi-variable kinematics model K is established com ; where C is a cosine function, S is a sine function; θ k is the angle of rotation of the rigid link joint module K-1 around the Z k-1 axis from X k-1 to X k ; a k is the angle of rotation of the rigid link joint module K around the X k axis from Z k-1 to Z k ; a i is the distance of the rigid link joint module K along the X k axis from Z k-1 to Z k ; d k is the distance of the rigid link joint module K along the Z k-1 axis from the origin O k-1 to the common perpendicular; k is the number of rigid link joint modules, L i is the length of the multi-rope driven flexible continuum (2), ω i is the deflection angle of the multi-rope driven flexible continuum (2), θ i is the deformation bending angle of the multi-rope driven flexible continuum (2).

3. The implementation method of claim 1, wherein, The hand-eye transformation calculation error AM f is: wherein, T is the transformation matrix from the robot base coordinate system to the ArUco marker module (9) outside the robot arm, T is the transformation matrix from the robot base coordinate system to the depth vision camera (8) in the eye-in-hand configuration, T is the transformation matrix from the depth vision camera (8) in the eye-in-hand configuration to the ArUco marker module (9) outside the robot arm.

4. The implementation method of claim 1, wherein, The gravity attitude error AM g is: where K' is the drive cable stiffness coefficient of the multi-cable driven flexible continuum (2), J T is the Jacobian matrix of the multi-cable driven flexible continuum (2), and g is the acceleration due to gravity.

5. The implementation method of claim 3, wherein, The hand-eye transpose error compensation threshold Δn t is: wherein, σ f are the mean and standard deviation of the error of the hand-eye transpose computation error ΔM f respectively, and δ is a constant factor.

6. The implementation method of claim 4, wherein, The driving rope compensating torque τ r is: τ r = K g · ΔM g where K g is the moment control gain matrix, k ij denotes the influence coefficient of the i-th component of the gravity attitude error on the j-th driving rope moment.

7. The implementation method of claim 2, wherein, The hybrid visual servoing configuration space-operation space dataset X t is: X t = [{θ k}‖{θ l}‖{{P r}}] Wherein, || represents data separation.

8. The implementation method of claim 2, wherein, Further comprising a control computer, the control computer communicates with the "eye-on-hand" configuration depth vision camera (8), the "eye-out-of-hand" configuration depth vision camera (10) and the rigid link-flexible continuum hybrid robot arm.

9. The implementation method of claim 2, wherein, The rigid link robot arm (1) is composed of a plurality of rigid link joint modules (6) connected in series, and the multi-rope driven flexible continuum (2) is composed of n groups of integrated joints connected by driving ropes.

Images