Calibration method and system for split surgical robot

Abstract

The application discloses a kind of calibration methods and systems of split type surgical robot, belong to robot technical field.The method comprises: S1: obtaining laparoscope internal parameter, calibrates laparoscope internal parameter, so that its re-projection error is within a certain range;S2: obtaining the pose information of surgical tool arm, based on laparoscope internal parameter and surgical tool arm pose, test laparoscope external parameter precision, ensure that laparoscope external parameter precision meets the requirement;S3: move surgical tool arm and endoscope arm end position and attitude, collect data through calibrated laparoscope;S4: calculate and obtain calibration quantity, i.e.the positional relationship between endoscope arm end and surgical tool arm end.The application is calibrated by laparoscope, encoder and calibration plate of split type surgical robot itself, does not need to set up camera additionally, is simpler and more flexible, and is more suitable for operating room environment.

Description

Technical Field

[0001] This disclosure relates to the field of robotics, and in particular to a calibration method and system for a split-type surgical robot. Background Technology

[0002] Currently, most laparoscopic surgical robots are integrated surgical robots. Integrated surgical robots integrate all robotic arms into a single device, and the relative poses between the base coordinate systems of the different robotic arms are fixed. Therefore, they can ensure that the movement direction of the surgeon's hand and the end effector of the surgical instruments remains consistent with the monitor image during operation. However, these systems are bulky, occupy a large amount of space, and are not very flexible.

[0003] In recent years, many newly developed laparoscopic surgical robots have adopted a modular design. A modular surgical robot is a system composed of several independent robotic arms, each with its own carriage. Surgeons can place different robotic arms at different positions beside the operating table according to surgical needs. Modular surgical robots allow for more flexible deployment in the operating room and are more convenient for clinical application. However, in collaborative operation, the controller needs to precisely know the relative positions of the different robots to ensure that the surgeon's hand and the end effector of the surgical instruments move in a consistent direction relative to the monitor image.

[0004] Split-type surgical robots can position their individual robotic arms using an optical positioning system. However, this positioning method has significant drawbacks in a real operating room environment. When using an optical positioning system, the robotic arms must be placed within a specific range to be accurately identified by the camera, which limits the flexibility of the robotic arm placement. The camera requires a large amount of space and cannot be moved; if the camera is moved during the operation, complex recalibration is required. Summary of the Invention

[0005] In view of this, the present disclosure provides a calibration method and system for a split-type surgical robot to at least partially solve the problems existing in the prior art.

[0006] In a first aspect, embodiments of this disclosure provide a calibration method for a split-type surgical robot, including:

[0007] S1: Obtain laparoscopic parameters and calibrate them to ensure that the reprojection error is within a preset range;

[0008] S2: Obtain the position information of the surgical tool arm, and test the accuracy of the laparoscopic external parameters based on the laparoscopic internal parameters and the position of the surgical tool arm to ensure that the accuracy of the laparoscopic external parameters meets the requirements;

[0009] S3: Move the surgical tool arm and the end of the laparoscopic arm to determine their position and orientation, and collect data using a calibrated laparoscope;

[0010] S4: Calculate the calibration quantity to determine the positional relationship between the end of the laparoscopic arm and the end of the surgical tool arm.

[0011] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0012] Define six parameters: A1, A2, B, X1, X2, and Z.

[0013] A1: The pose of the end flange of the surgical tool arm relative to the base is calculated by reading the encoder of the surgical tool arm robot;

[0014] A2: The pose of the end flange of the laparoscopic arm relative to the base is calculated by reading the encoder of the laparoscopic arm robot;

[0015] B: External parameters of laparoscopy, obtained by image analysis of laparoscopic images;

[0016] X1: Position of the calibration plate relative to the end flange of the surgical tool arm;

[0017] X2: Position of the laparoscope relative to the flange at the end of the laparoscope arm;

[0018] Z: Position of the surgical tool arm base relative to the laparoscopic arm base.

[0019] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0020] Fix the robotic arm so that the relative positions of the surgical tool arm and the laparoscopic arm base remain unchanged;

[0021] Zhang's calibration method was used to correct lens image distortion and calibrate laparoscopic internal parameters to ensure that reprojection error was within a preset range.

[0022] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0023] S41: Based on the previously calibrated laparoscopic distortion parameters and internal references, obtain a set of data (A1,B) before the end-effector pose of the surgical tool arm is moved and a set of data (A′1,B’) after the movement.

[0024] S42: After repeating step S41 k times, check the mean squared error (RMS) of the difference between the rotation angle of the laparoscopic external parameter and the rotation angle of the surgical tool arm tip.

[0025]

[0026] In the formula, A represents1i The rotation matrix, Represents A′ 1i The rotation matrix, B 1i The rotation matrix, Indicates B′ 1i The rotation matrix.

[0027] S43: If the RMS is within the preset accuracy range, it means that the laparoscopic parameters are accurate enough and the test is complete; otherwise, the internal parameter calibration should be performed again.

[0028] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0029] While ensuring that the calibration plate can be seen through the laparoscope, move the surgical tool arm and the end position of the end end of the laparoscopic arm, record the end flange position A1 of the surgical tool arm, the end flange position A2 of the laparoscopic arm, and the laparoscopic external parameter B, and collect n sets of data.

[0030] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0031] Calculated using the least squares method:

[0032]

[0033] Obtain the unknown standard quantity The LM algorithm is used to estimate the standardization quantitation. Immediately:

[0034]

[0035]

[0036] Choose any set of initial values ​​X (1,0) ,X (2,0) Z0

[0037] make Where I is the identity matrix

[0038]

[0039] Obtain a set of calibration values ​​that make L smaller.

[0040] A calibrated quantity that satisfies the conditions is obtained through sufficient iteration.

[0041] Secondly, embodiments of this disclosure provide a calibration system for a split-type surgical robot, comprising:

[0042] Surgical tool arm, used to hold surgical instruments;

[0043] Laparoscopic arm, used to hold the laparoscope;

[0044] The calibration plate, fixed to the surgical tool arm that holds the surgical instruments, is used to correct lens distortion and determine the conversion relationship between the spatial position of the robotic arm and pixels;

[0045] Laparoscopy allows for multi-angle observation of the abdominal cavity during laparoscopic surgery.

[0046] A computing device that performs calculations on calibration data to implement the calibration method for the split-type surgical robot in any of the aforementioned aspects or the first aspect.

[0047] Thirdly, embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the calibration method of the split-type surgical robot in the first aspect or any implementation thereof.

[0048] Fourthly, embodiments of this disclosure also provide a computer program product, which includes a computing program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the calibration method for the split-type surgical robot in the first aspect or any implementation thereof.

[0049] The calibration scheme for the split-type surgical robot in this embodiment includes: S1: acquiring laparoscopic intra-parameters and calibrating them to ensure their reprojection error is within a preset range; S2: acquiring the pose information of the surgical tool arm and testing the accuracy of the laparoscopic extrinsic parameters based on the laparoscopic intra-parameters and the surgical tool arm pose to ensure that the accuracy of the laparoscopic extrinsic parameters meets the requirements; S3: moving the position and posture of the surgical tool arm and the endoscope arm, and collecting data through the calibrated laparoscope; S4: calculating the calibration value and determining the positional relationship between the endoscope arm and the endoscope arm. This invention uses the laparoscope, encoder, and calibration plate integrated into the split-type surgical robot for calibration, eliminating the need for an additional camera, making it simpler, more flexible, and more suitable for the operating room environment. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of a split-type surgical robot calibration system provided in an embodiment of the present disclosure;

[0052] Figure 2 A flowchart illustrating the calibration method for a split-type surgical robot provided in this embodiment of the present disclosure;

[0053] Figure 3 This is a schematic diagram of the end-effector position of the surgical tool arm provided in an embodiment of this disclosure. Detailed Implementation

[0054] See Figure 1 , Figure 2 and Figure 3 The present invention discloses a calibration method for a split-type surgical robot, comprising the following steps:

[0055] S1: Obtain laparoscopic parameters and calibrate them to ensure that the reprojection error is within a preset range;

[0056] S2: Obtain the position information of the surgical tool arm, and test the accuracy of the laparoscopic external parameters based on the laparoscopic internal parameters and the position of the surgical tool arm to ensure that the accuracy of the laparoscopic external parameters meets the requirements;

[0057] S3: Move the surgical tool arm and the end of the laparoscopic arm to determine their position and orientation, and collect data using a calibrated laparoscope;

[0058] S4: Calculate the calibration quantity to determine the positional relationship between the end of the laparoscopic arm and the end of the surgical tool arm.

[0059] Specifically, a split-type surgical robot system consists of two or more robotic arms, such as... Figure 1 As shown, in the system, one robotic arm holds a laparoscope (hereinafter referred to as the laparoscopic arm), and one or more robotic arms hold surgical tools and have calibration plates fixed on the robotic arms (hereinafter referred to as surgical tool arms; there can be multiple surgical tool arms, but only one is used as an example in the figure).

[0060] 1. Surgical tool arm: A high-performance multi-axis robotic arm used to hold surgical tools;

[0061] 2. Laparoscopic arm: A high-performance multi-axis robotic arm used to hold the laparoscope;

[0062] 3. Calibration plate: Fixed to the surgical tool arm that holds the surgical instruments, used to correct lens distortion and determine the conversion relationship between the spatial position of the robotic arm and pixels;

[0063] 4. Laparoscope: A medical device equipped with a miniature camera, which allows for multi-angle observation of the abdominal cavity during laparoscopic surgery.

[0064] Hand-eye coordination

[0065] The concept of hand-eye coordination is crucial in the field of split-type surgical robots. It aims to establish a precise positional correspondence between the surgical tool arm and the laparoscopic arm, and to unify them within the same coordinate system for coordinated operation. The spatial transformation formula for hand-eye coordination is as follows:

[0066]

[0067] In the formula, δP1 represents the displacement direction in the surgical tool arm coordinate system, and δP 2，des R represents the desired displacement direction under laparoscopy. Z Denotes the rotation matrix of Z. Describe the rotation matrix of A2. Let X2 be the rotation matrix.

[0068] As shown in the formula, to obtain the spatial transformation relationship between the laparoscope and surgical tools and achieve correct movement, it is necessary to know Z, A2, and X2. A2 can be calculated from the robot's encoder readings. X2 and Z need to be obtained using the split-type surgical robot calibration method proposed in this invention.

[0069] This invention provides a calibration method for a split-type surgical robot, solving the problems of existing hand-eye calibration cameras being immobile, having high requirements for placement, cumbersome calibration steps, and insufficient flexibility in robotic arm movement. The method includes the following steps:

[0070] S1: Obtain laparoscopic parameters and calibrate them to ensure that the reprojection error is within a certain range;

[0071] S2: Obtain the position information of the surgical tool arm, and test the accuracy of the laparoscopic external parameters based on the laparoscopic internal parameters and the position of the surgical tool arm to ensure that the accuracy of the laparoscopic external parameters meets the requirements;

[0072] S3: Move the surgical tool arm and the end of the laparoscopic arm to determine their position and orientation, and collect data using a calibrated laparoscope;

[0073] S4: Calculate the calibration quantity, that is, the positional relationship between the end of the laparoscopic arm and the end of the surgical tool arm.

[0074] Setting parameters

[0075] Set six parameters: A1, A2, B, X1, X2, and Z.

[0076] 1) A1: The pose of the end flange of the surgical tool arm relative to the base (known), which may be calculated by reading the encoder of the surgical tool arm robot;

[0077] 2) A2: The pose of the end flange of the laparoscopic arm relative to the base (known), calculated by reading the encoder of the laparoscopic arm robot;

[0078] 3)B: Laparoscopic external parameters (position of the calibration plate relative to the laparoscope) (known), obtained by image analysis of laparoscopic images;

[0079] 4)X1: Position of the calibration plate relative to the end flange of the surgical tool arm (unknown);

[0080] 5)X2: Position of the laparoscope relative to the flange at the end of the laparoscope arm (unknown);

[0081] 6) Z: The pose of the surgical tool arm base relative to the laparoscopic arm base (unknown).

[0082] Calibration internal reference

[0083] 1) Fix the robotic arm so that the relative positions of the surgical tool arm and the laparoscopic arm base remain unchanged;

[0084] 2) Use Zhang's calibration method to correct lens image distortion and calibrate laparoscopic internal parameters so that the reprojection error is within a certain range.

[0085] Testing the accuracy of external parameters in laparoscopy

[0086] 1) Position of the distal end of the moving surgical tool arm:

[0087] Based on the previously calibrated laparoscopic distortion parameters and internal references, a set of data (A1, B) before the movement of the surgical tool arm end pose and a set of data (A′1, B′) after the movement were obtained;

[0088] 2) Repeat step 1) k times (k generally needs to be ≥5);

[0089] 3) Check the mean squared error (RMS) of the difference between the rotation angle of the laparoscopic external reference and the rotation angle of the surgical tool arm end.

[0090]

[0091] In the formula, A represents 1i The rotation matrix, Represents A′ 1i The rotation matrix,

[0092] B 1i The rotation matrix, Indicates B′ 1i The rotation matrix.

[0093] 4) If the RMS value is within a certain accuracy range, it indicates that the laparoscopic parameters are accurate enough, and the test is complete. Otherwise, return to the previous state. Calibration Internal Reference The next step is to recalibrate the internal parameters.

[0094] Data collection

[0095] While ensuring the calibration plate is visible to the laparoscope, move the surgical tool arm and the endplate of the laparoscopic arm to record the endplate flange position A1 of the surgical tool arm, the endplate flange position A2 of the laparoscopic arm, and the laparoscopic external parameter B. Collect n sets of data (n≥18).

[0096] calculate

[0097] Calculated by least squares

[0098] Obtain the unknown standard quantity

[0099] Here, the LM (Levenberg-Marquardt) algorithm can be used to estimate the scaling factor. Immediately:

[0100]

[0101]

[0102] Choose any set of initial values ​​X (1,0) ,X (2,0) Z0

[0103] make (where I is the identity matrix)

[0104]

[0105] This yields a set of calibration values ​​that make L smaller.

[0106] A calibrated quantity that meets the conditions can be obtained through sufficient iteration.

[0107] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0108] Define six parameters: A1, A2, B, X1, X2, and Z.

[0109] A1: The pose of the end flange of the surgical tool arm relative to the base is calculated by reading the encoder of the surgical tool arm robot;

[0110] A2: The pose of the end flange of the laparoscopic arm relative to the base is calculated by reading the encoder of the laparoscopic arm robot;

[0111] B: External parameters of laparoscopy, obtained by image analysis of laparoscopic images;

[0112] X1: Position of the calibration plate relative to the end flange of the surgical tool arm;

[0113] X2: Position of the laparoscope relative to the flange at the end of the laparoscope arm;

[0114] Z: Position of the surgical tool arm base relative to the laparoscopic arm base.

[0115] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0116] Fix the robotic arm so that the relative positions of the surgical tool arm and the laparoscopic arm base remain unchanged;

[0117] Zhang's calibration method was used to correct lens image distortion and calibrate laparoscopic internal parameters to ensure that reprojection error was within a preset range.

[0118] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0119] S41: Based on the previously calibrated laparoscopic distortion parameters and internal references, obtain a set of data (A1,B) before the end-effector pose of the surgical tool arm is moved and a set of data (A′1,B’) after the movement.

[0120] S42: After repeating step S41 k times, check the mean squared error (RMS) of the difference between the rotation angle of the laparoscopic external parameter and the rotation angle of the surgical tool arm tip.

[0121]

[0122] In the formula, A represents 1i The rotation matrix, Represents A′ 1i The rotation matrix, B 1i The rotation matrix, Indicates B′ 1i The rotation matrix.

[0123] S43: If the RMS is within the preset accuracy range, it means that the laparoscopic parameters are accurate enough and the test is complete; otherwise, the internal parameter calibration should be performed again.

[0124] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0125] While ensuring that the calibration plate can be seen through the laparoscope, move the surgical tool arm and the end position of the end end of the laparoscopic arm, record the end flange position A1 of the surgical tool arm, the end flange position A2 of the laparoscopic arm, and the laparoscopic external parameter B, and collect n sets of data.

[0126] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0127] Calculated using the least squares method:

[0128]

[0129] Obtain the unknown standard quantity The LM algorithm is used to estimate the standardization quantitation. Immediately:

[0130]

[0131]

[0132] Choose any set of initial values ​​X (1,0) ,X (2,0) Z0

[0133] make Where I is the identity matrix

[0134]

[0135] Obtain a set of calibration values ​​that make L smaller.

[0136] A calibrated quantity that satisfies the conditions is obtained through sufficient iteration.

[0137] Corresponding to the above method embodiments, the present invention also provides a calibration system for a split-type surgical robot, comprising:

[0138] Surgical tool arm, used to hold surgical instruments;

[0139] Laparoscopic arm, used to hold the laparoscope;

[0140] The calibration plate, fixed to the surgical tool arm that holds the surgical instruments, is used to correct lens distortion and determine the conversion relationship between the spatial position of the robotic arm and pixels;

[0141] Laparoscopy allows for multi-angle observation of the abdominal cavity during laparoscopic surgery.

[0142] A computing device that performs calculations on calibration data, thereby enabling the method described in the foregoing embodiments.

[0143] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A calibration method for a split-type surgical robot, characterized in that, include: S1: Obtain laparoscopic parameters and calibrate them to ensure that the reprojection error is within a preset range; S2: Obtain the position information of the surgical tool arm, and test the accuracy of the laparoscopic external parameters based on the laparoscopic intra-parameters and the position information of the surgical tool arm to ensure that the accuracy of the laparoscopic external parameters meets the requirements. S3: Move the surgical tool arm and the end of the laparoscopic arm to determine their position and orientation, and collect data using a calibrated laparoscope; S4: Calculate and obtain the calibration quantity to determine the positional relationship between the end of the laparoscopic arm and the end of the surgical tool arm; The method further includes: set up The six parameters are Z, among which The pose of the end flange of the surgical tool arm relative to the base is calculated by reading the robot encoder; The pose of the end flange of the laparoscopic arm relative to the base is calculated by reading the robot's encoder; B: External parameters for laparoscopy are obtained by reading endoscopic images and using image analysis methods; The orientation of the calibration plate relative to the end flange of the surgical tool arm; The position of the laparoscope relative to the flange at the end of the laparoscope arm; Z: Position of the surgical tool arm base relative to the laparoscopic arm base; S41: Obtain a set of data on the position of the surgical tool arm end-effector before movement, based on previously calibrated laparoscopic parameters. B) and a set of data after the move ( ,B'); S42: After repeating step S41 k times, examine the mean squared error (RMS) of the difference between the rotation angle of the laparoscopic external parameter and the rotation angle of the surgical tool arm tip. In the formula, express The rotation matrix, express The rotation matrix, express The rotation matrix, express rotation matrix; S43: If the RMS is within the preset accuracy range, it means that the laparoscopic parameters are accurate enough and the test is complete; otherwise, the intrinsic parameter calibration should be performed again.

2. The method according to claim 1, characterized in that, The method further includes: Fix the surgical tool arm and the endoscope arm so that the relative positions of the surgical tool arm and the endoscope arm base remain unchanged; Zhang's calibration method was used to correct lens image distortion and calibrate laparoscopic internal parameters to ensure that reprojection error was within a preset range.

3. The method according to claim 1, characterized in that, The method further includes: While ensuring the calibration plate is visible through the laparoscope, move the surgical tool arm and the end effector of the laparoscopic arm to record the position of the end flange of the surgical tool arm. The position of the end flange of the laparoscopic arm In addition, external reference B for laparoscopy was used to collect n sets of data.

4. The method according to claim 3, characterized in that, The method further includes: Calculated using the least squares method: Obtain the unknown standard quantity ; The LM algorithm is used to estimate the standardization quantitation. In other words: These are the 6-dimensional Lie algebraic small perturbation increments corresponding to the calibrated quantities Z, X1, and X2 in the LM optimization iteration. They are The transpose row vector They represent Lie group index mapping, Let Z, X1, and X2 represent the update transformation matrices after superimposing the corresponding Lie algebraic perturbations on the calibrated quantities Z, X1, and X2, respectively. Choose any set of initial values make Where I is the identity matrix Obtain a set of scaling factors that make L smaller, and iterate sufficiently to obtain the scaling factors that satisfy the conditions: 。 5. A calibration system for a split-type surgical robot, characterized in that, include: Surgical tool arm, used to hold surgical instruments; Laparoscopic arm, used to hold the laparoscope; The calibration plate, fixed to the surgical tool arm that holds the surgical instruments, is used to correct lens distortion and determine the spatial position of the surgical tool arm and the endoscope arm and the conversion relationship between pixels. Laparoscopy allows for multi-angle observation of the abdominal cavity during laparoscopic surgery. A computing device that performs calculations on calibration data to implement the method described in any one of claims 1-4.

Images