A method and system for calibrating the target area spatial position of a medical surgical robot

Abstract

A method and system for calibrating the spatial position of a target area for a medical surgical robot is disclosed. An infrared binocular camera uses infrared feature points to spatially register the robotic arm and target area. During robot operation, all infrared feature points are ensured to be within the field of view of the infrared binocular camera. Dynamic alignment is performed using the coordinates of the infrared feature points in the current camera coordinate system with the coordinates of the infrared feature points in the robotic arm coordinate system and the CT coordinate system recorded during spatial registration. The dynamic conversion relationship between the different coordinate systems is calculated, enabling real-time dynamic spatial registration of the robot during operation. The present invention possesses dynamic characteristics and can dynamically establish the relative positional relationship between the target area and the robot in real time during robot operation, enabling real-time spatial registration of the robot's target. Even if the relative positional relationship between the two in physical space changes, the operational positioning of the robot system is not affected.

Description

Technical Field

[0001] The present invention belongs to the field of medical equipment, and utilizes an infrared binocular camera and CT to realize the calibration of the spatial position information of a robot relative to a patient's target area. It is a method and system for calibrating the target area spatial position of a medical surgical robot. Background Art

[0002] With the advancement of science and technology, the application of robots in the medical field has rapidly expanded. Some traditional medical procedures no longer meet the stringent requirements of speed, accuracy, and minimal trauma, necessitating the assistance of robots. This invention utilizes CT technology, infrared recognition technology, and robotics to achieve precise spatial registration of the robot relative to the target area being operated on.

[0003] The current common method for registering the camera coordinates of a robotic arm's end-point is to place the tip of a registration pen at the end-point and illuminate the infrared feature points on the pen with a camera to calculate the camera coordinates of the pen tip, that is, the end-point. This method has several problems: 1. The registration pen itself has accuracy issues; 2. This manual registration method can reduce registration accuracy due to hand shake, external interference, and other issues, making it impossible to achieve automation requirements; 3. If the robotic arm's end-point is suspended in the air, this registration method is no longer applicable.

[0004] In the prior art, the relative positions of the camera, the robot, and the target area are all fixed in advance, and the camera performs joint spatial registration of the robot and the target area, and the robot and the target area establish a spatial registration relationship through the camera. During the operation of the robot, when the camera illuminates the infrared feature points of the target area and the base of the robotic arm, it is necessary to distinguish which infrared feature points identified by the camera are target feature points and which are robotic arm base feature points, and then pair the camera coordinate system coordinates of the distinguished infrared feature points with their robotic arm coordinate system coordinates or CT coordinate system coordinates to form a one-to-one correspondence. Currently, the commonly used means of distinction are occlusion and orientation recognition. The occlusion method will affect the robot's operation of the target area, and the commonly used manual occlusion method cannot meet the automation requirements. Once the position of the robot or the target area moves relative to the camera, the orientation recognition method will fail.

[0005] Furthermore, once the relative positions of the robot arm, camera, and target area have been registered, if the robot or target area moves during operation, causing the relative positions of the three to change, re-registration is required, otherwise deviations will occur. This approach clearly does not meet the requirements of convenient and fast use. Summary of the Invention

[0006] The technical problem of the present invention is: to provide a calibration method and system for establishing the dynamic relative position relationship between the operating area and the surgical robot in response to the use requirements of the medical robot.

[0007] The technical solution of the present invention is: a method for calibrating the spatial position of a target area of ​​a medical surgical robot, which uses a CT device and an infrared binocular camera to calibrate the spatial position of the robot, establishes a coordinate transformation relationship from the CT device coordinate system to the camera coordinate system and then to the robot's mechanical arm coordinate system, and dynamically updates the relative position relationship between the target area and the mechanical arm in real time during the operation of the robot, thereby realizing dynamic spatial position calibration of the robot for the operation target. The method includes the following steps:

[0008] 1) Space registration:

[0009] The infrared binocular camera performs spatial registration of the manipulator and the target area. First, the coordinates of the manipulator's end point in the manipulator coordinate system are obtained using the manipulator's forward kinematics principle. The coordinates of the manipulator's end point in the camera coordinate system are also obtained, and the initial transformation relationship between the manipulator coordinate system and the camera coordinate system is established. At least three infrared feature points are marked on the manipulator base. The infrared binocular camera is used to identify and record the camera coordinates of the base's infrared feature points. The coordinates of the base's infrared feature points in the manipulator coordinate system are then calculated from the initial transformation relationship.

[0010] At least three infrared characteristic points of the target area are marked in the robot's target operating area, i.e., the target area. The target area is photographed by a CT device and a three-dimensional model of the target area is reconstructed in the CT coordinate system. The coordinates of the infrared characteristic points of the target area in the CT coordinate system are identified and recorded, and the relative position relationship between the infrared characteristic points of the target area and the target area is established in the three-dimensional model.

[0011] 2) Real-time dynamic registration:

[0012] During the operation of the robot, it is ensured that the infrared feature points of the robot's manipulator and the target area are within the field of view of the infrared binocular camera. The infrared binocular camera obtains the coordinates of the infrared feature points of the base and the target area in the current camera coordinate system, and dynamically aligns them with the recorded coordinates of the infrared feature points of the base in the manipulator coordinate system and the target area in the CT coordinate system to obtain the dynamic conversion relationship from the camera coordinate system to the manipulator coordinate system and the dynamic conversion relationship from the CT coordinate system to the camera coordinate system. In this way, the dynamic conversion relationship from the CT coordinate system to the manipulator coordinate system is established, and the real-time dynamic spatial position calibration of the robot during operation is realized.

[0013] The present invention also provides a target area spatial position calibration system for a medical surgical robot, comprising a CT device, an infrared binocular camera, a robot, infrared feature points, and a controller. The controller is connected to the CT device, the infrared binocular camera, and the robot, respectively. The controller is configured with a computer program. The CT device, the infrared binocular camera, the robot, and the infrared feature points are configured according to the above-mentioned spatial position calibration method, and the computer program is executed simultaneously to implement the above-mentioned spatial position calibration method.

[0014] The beneficial effects of the present invention are as follows: by shooting CT and reconstructing a three-dimensional model, the relative position relationship between the infrared feature points and the target area is established, and the invisible and intangible target area is exposed to the camera and the robotic arm; the center of sphere fitting algorithm is used to obtain the coordinates of the camera coordinate system of the end point of the robotic arm to ensure the accuracy of the measurement; the registration algorithm is used to automatically distinguish and identify the corresponding relationship of the infrared feature points in different coordinate systems. In actual operation, it is only necessary to expose the robot's operating target area and the robotic arm to the field of view of the infrared binocular camera, and no additional operation is required, which is convenient and fast; the entire process only requires initial registration once. During the subsequent use of the robot, even if the camera, operating target or the robotic arm of the robot are moved and the relative position relationship between them is changed, there is no need to re-register, which ensures convenient, fast and dynamic characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] The present invention provides a method and system for spatial position calibration of a medical surgical robot based on a binocular infrared camera and CT equipment. Spatial position calibration, also known as spatial registration, utilizes an infrared binocular camera to identify the position information of infrared feature points. Using registration and coordinate transformation methods, the method establishes a coordinate transformation relationship from the CT coordinate system to the camera coordinate system, and then to the robot coordinate system during surgery. This facilitates positioning the robot for treatment based on the patient's target area. The present invention possesses dynamic characteristics and can dynamically establish the relative position relationship between the patient's target area and the robot in real time in clinical surgical applications. Even if the relative position relationship between the two changes in physical space, the use of the system remains unchanged.

[0017] like Figure 1As shown, the infrared binocular camera of the present invention performs spatial registration of the robot arm and the target area respectively. First, the coordinates of the end point of the robot arm in the robot arm coordinate system are obtained using the principle of positive kinematics of the robot arm. The coordinates of the end point of the robot arm in the camera coordinate system of the infrared binocular camera are also obtained, and the initial transformation relationship between the robot arm coordinate system and the camera coordinate system is established. At least three infrared feature points of the base of the robot arm are marked, and the camera coordinate system coordinates of the base infrared feature points are identified and recorded using the infrared binocular camera. The coordinates of the base infrared feature points in the robot arm coordinate system are obtained from the initial transformation relationship. At the same time, the infrared feature points of the target area (i.e., the target area) are marked. The target area is photographed by CT equipment and a three-dimensional model of the target area is reconstructed in the CT coordinate system. The coordinates of the infrared feature points of the target area in the CT coordinate system are identified and recorded, and the relative position relationship between the infrared feature points of the target area and the target area is established in the three-dimensional model. Finally, real-time dynamic registration is performed. During the operation of the robot, the infrared feature points of the robot arm and the target area are ensured to be within the field of view of the infrared binocular camera. The infrared binocular camera obtains the coordinates of the infrared feature points of the base and the target area in the current camera coordinate system, and dynamically aligns them with the recorded coordinates of the infrared feature points of the base in the robotic arm coordinate system and the target area in the CT coordinate system. The dynamic conversion relationship from the camera coordinate system to the robotic arm coordinate system and the dynamic conversion relationship from the CT coordinate system to the camera coordinate system are obtained. The dynamic conversion relationship from the CT coordinate system to the robotic arm coordinate system is thus established, realizing real-time dynamic spatial registration of the robot during operation.

[0018] In the present invention, the positional relationship between the CT device, the infrared binocular camera, and the robot only needs to ensure that the infrared feature points are within the infrared camera's line of sight, that is, to ensure that the infrared camera can see all infrared feature points. During actual robot operation, the positional relationship between the camera and the robot or target area does not need to be consistent with the position during spatial registration, because the robot arm coordinate system coordinates of the infrared feature points of the robot arm base and the CT coordinate system coordinates of the infrared feature points of the target area have been recorded. Even if the infrared feature points of the base and the target area in the camera image change during robot operation, the positions of these infrared feature points on the robot arm base and the target area remain unchanged. Therefore, the transformation matrix between the three coordinate systems can be dynamically updated in real time during robot operation. In other words, in the present invention, only one registration is required at the beginning, and subsequently, as long as the infrared camera can continue to see all infrared feature points, even if the position of the camera and the robot or CT device changes, it will not affect the robot's spatial registration, thereby ensuring the accuracy of the robot's operational control.

[0019] In order to better illustrate the technical solutions and features of the present invention, the implementation of the present invention is described in detail below with reference to specific examples.

[0020] Step one, move the end of the robotic arm to four points in space, and perform the following operations at each point: mark an infrared feature point at the end of the robotic arm to ensure that the relative distance between the infrared feature point and the end endpoint does not change during the movement of the robotic arm. This is because the end endpoint of the robotic arm is relatively fine, and the infrared feature point is difficult to accurately align with the end endpoint. The present invention adopts a spherical center fitting algorithm, controls the end endpoint to remain stationary through instructions, and randomly rotates the robotic arm. During the process, an infrared camera is used to continuously record the camera coordinate system coordinates of the infrared feature points. Finally, a set of infrared feature point coordinates recorded by the infrared camera is used. Since the distance between the infrared feature point and the end endpoint is certain, the spherical center coordinates can be calculated by the spherical center fitting method, that is, the camera coordinate system coordinates of the end endpoint of the robotic arm. The infrared feature points near the end of the robotic arm are used to calculate the camera coordinate system coordinates of the end endpoint of the robotic arm using the spherical center fitting algorithm. They are only used when establishing the initial conversion relationship and are no longer needed in the actual operation of the robot. The robotic arm coordinate system coordinates of the end endpoint of the robotic arm are provided by the robotic arm.

[0021] In step 2, the robot arm coordinate system and camera coordinate system coordinates of four pairs of non-collinear robot arm end point positions in different robot arm postures are collected using the same method as in step 1. Each pair of point coordinates is used as a group to calculate the transformation matrix. The remaining pair of point coordinates is used to verify the calculated transformation matrix error. The verification method is as follows: record the transformation matrix T from the camera to the robot arm coordinate system, the robot arm coordinate system coordinate A of the verification point, and the camera coordinate system coordinate B. Multiply T by A to calculate the camera coordinate system coordinate B′. Subtract B′ from B to calculate the error. Finally, select the transformation matrix with the smallest error as the initial transformation relationship.

[0022] Step three: mark at least three infrared feature points on the base of the robotic arm, use an infrared binocular camera to identify and record their camera coordinate system coordinates, and then use the initial conversion relationship established in step two to obtain the robotic arm coordinate system coordinates of the infrared feature points of the robotic arm base and record them.

[0023] Step 4: Determine the robot's operating area, i.e., the target area, and place at least three infrared feature points. A CT scan is taken with a CT device, and a 3D model is reconstructed based on the CT data. The CT coordinate system coordinates of the target area's infrared feature points are identified and recorded, and the relative positional relationship between the target area's infrared feature points and the target area is established in the 3D model.

[0024] Step 5: Place the robotic arm and target area within the monitoring range of the infrared binocular camera to ensure that the infrared feature points of the target area and the infrared feature points of the robotic arm base can be captured by the infrared binocular camera. During the operation of the robot, the infrared binocular camera identifies a set of infrared feature points and obtains the camera coordinate system coordinate set X of the infrared feature points in the current field of view. Suppose the total number of infrared feature points is N, with an arrangement H, and a corresponding relative distance set S between each point. The coordinates of these infrared feature points in the robotic arm coordinate system and the CT coordinate system coordinate set X' recorded during spatial registration are: Permutation H′ n , corresponding to the relative distance set S′ n , Separate S and S′ n Compare and find S′ with the smallest comprehensive error with S n , then the corresponding arrangement H′ n Each point in the arrangement H is matched one by one, and the conversion correspondence between the current camera coordinate system, the robotic arm coordinate system, and the CT coordinate system is solved by the two matched sets of coordinates; when the positional relationship between the infrared binocular camera, the robotic arm, and the target area changes, the coordinate system conversion correspondence is updated according to the dynamic registration to realize real-time dynamic spatial registration of the robot during operation.

[0025] The above-mentioned dynamic registration method of the present invention does not need to distinguish in advance which infrared feature points identified by the camera are target feature points and which are robot arm base feature points. The infrared feature points have been configured with their real-world positions during the first spatial registration. The coordinates of the robot arm coordinate system and the coordinate system of the infrared feature points recorded during spatial registration are unchanged. Based on this, the present invention uses the above-mentioned dynamic registration method to match the feature points in the camera coordinate system with the feature points in other coordinate systems, and is not affected by the changes in the relative position relationship between the robot arm, target area and camera. As long as the camera can always capture all infrared feature points, the conversion correspondence of the coordinate system can be updated in real time. This feature of the present invention is particularly beneficial for the use of medical robots. During the operation of the robot, even if the target area position is moved, it will not affect the operation.

[0026] To further reduce the computational complexity during registration, the present invention further preferably configures different arrangement features when setting the infrared feature points of the base and the target area. Before dynamic registration, the infrared feature points identified by the infrared binocular camera are first distinguished as base infrared feature points or target area infrared feature points based on the arrangement features. The dynamic matching relationship between the base infrared feature points and the target area infrared feature points is then calculated to obtain the corresponding relationship for the current coordinate system conversion. Compared with the occlusion or orientation recognition methods of the prior art, the configuration of different arrangement features of the infrared feature points based on the dynamic registration of the present invention is more convenient and faster, does not affect actual operation, and will not fail due to position shifting.

[0027] The following takes the infrared feature points of the robot base as an example to illustrate the dynamic alignment of the present invention. Assume that the camera coordinate system coordinate set of the infrared feature points recognized by the camera is [A(x,y,z),B(x,y,z),C(x,y,z)], whose arrangement is H(ABC), and the relative distance set between the points is S[AB,AC,BA,BC,CA,CB]. The robot coordinate system coordinate set of these infrared feature points is: [A′(x,y,z),B′(x,y,z),C′(x,y,z)], whose arrangement is There are 6 types of relative distance sets between points: H′1(A′B′C′), H′2(A′C′B′), H′3(B′A′C′), H′4(B′C′A′), H′5(C′A′B′), H′6(C′B′A′), and there are 6 types of relative distance sets between points: S′1[A′B′,A′C′,B′A′,B′C′,C′A′,C′B′], S′2[A′C′,A′B′,C′A′,C′B′,B′A′,B′C′], S′3[…], S′4[…], S′5[…], S′6[…]. Compare S with S′1, S′2, S′3, S′4, S′5, S′6 respectively, and find S′ with the smallest comprehensive error with S. n , and its corresponding robot arm coordinate system coordinate point arrangement H′ n Matching the points in the camera coordinate system with the points in the camera coordinate system coordinate array H is a correct one-to-one correspondence. The coordinates of the two matched sets of feature points are used to solve the transformation correspondence between the current camera coordinate system and the robotic arm coordinate system. The dynamic matching of the target area infrared feature points is similar.

[0028] Step 6: Based on the manipulator coordinate system coordinates of the manipulator base feature points and the camera coordinate system coordinates obtained in step 5, as well as their corresponding relationship, a dynamic transformation relationship between the manipulator coordinate system and the camera coordinate system is established. Based on the CT coordinate system coordinates of the target area feature points and the camera coordinate system coordinates obtained in step 5, as well as their corresponding relationship, a dynamic transformation relationship between the CT coordinate system and the camera coordinate system is established. Finally, a dynamic transformation relationship between the CT coordinate system and the manipulator coordinate system is established, enabling dynamic and precise registration of the robot's spatial position information relative to the patient's target area.

[0029] The present invention provides a target area spatial position calibration system for a medical surgical robot corresponding to the above method. A controller is configured to connect a CT device, an infrared binocular camera, and a robot respectively. The CT device, the infrared binocular camera, the robot, and the infrared feature points are configured according to the above method. The controller is configured with a computer program. When the computer program is executed, the above spatial position calibration method is implemented, thereby realizing dynamic spatial registration of the robot.

Claims

1. A method for calibrating the target space position of a medical surgical robot, characterized by The robot's spatial position is calibrated using a CT device and an infrared binocular camera. A coordinate transformation relationship is established from the CT device coordinate system to the camera coordinate system and then to the robot's mechanical arm coordinate system. During the robot's operation, the relative position relationship between the target area and the mechanical arm is dynamically updated in real time to achieve dynamic spatial position calibration of the robot for the operation target. The process includes the following steps: 1) Space registration: The infrared binocular camera performs spatial registration of the manipulator and the target area. First, the coordinates of the manipulator's end point in the manipulator coordinate system are obtained using the manipulator's forward kinematics principle. The coordinates of the manipulator's end point in the camera coordinate system are also obtained, and the initial transformation relationship between the manipulator coordinate system and the camera coordinate system is established. At least three infrared feature points are marked on the manipulator base. The infrared binocular camera is used to identify and record the camera coordinates of the base's infrared feature points. The coordinates of the base's infrared feature points in the manipulator coordinate system are then calculated from the initial transformation relationship. At least three infrared characteristic points of the target area are marked in the robot's target operating area, i.e., the target area. The target area is photographed by a CT device and a three-dimensional model of the target area is reconstructed in the CT coordinate system. The coordinates of the infrared characteristic points of the target area in the CT coordinate system are identified and recorded, and the relative position relationship between the infrared characteristic points of the target area and the target area is established in the three-dimensional model. 2) Real-time dynamic registration: During the operation of the robot, it is ensured that the infrared feature points of the robot's manipulator and the target area are within the field of view of the infrared binocular camera. The infrared binocular camera obtains the coordinates of the infrared feature points of the base and the target area in the current camera coordinate system, and dynamically aligns them with the recorded coordinates of the infrared feature points of the base in the manipulator coordinate system and the target area in the CT coordinate system to obtain the dynamic conversion relationship from the camera coordinate system to the manipulator coordinate system and the dynamic conversion relationship from the CT coordinate system to the camera coordinate system. In this way, the dynamic conversion relationship from the CT coordinate system to the manipulator coordinate system is established, and the real-time dynamic spatial position calibration of the robot during operation is realized.

2. The method for calibrating the target area spatial position of a medical surgical robot according to claim 1, characterized in that The method for obtaining the coordinates of the end point of the robotic arm in the camera coordinate system is as follows: mark an infrared feature point at the end of the robotic arm to ensure that the relative distance between the infrared feature point and the end point of the robotic arm does not change during the movement of the robotic arm, control the end point to remain stationary through instructions, and randomly rotate the robotic arm. During the process, the infrared camera is used to continuously record the camera coordinate system coordinates of the infrared feature point. Finally, the coordinates of the center of the sphere are calculated using a set of coordinates recorded by the infrared camera and the sphere center fitting method, that is, the coordinates of the end point of the robotic arm in the camera coordinate system.

3. The method for calibrating the target area spatial position of a medical surgical robot according to claim 1, characterized in that The method for obtaining the initial transformation relationship between the robot coordinate system and the camera coordinate system is as follows: operate the end point of the robot arm to move to four non-collinear points in space, collect the coordinates of the four points in the robot coordinate system and the camera coordinate system respectively, and calculate the transformation matrix according to the camera coordinate system coordinates and the robot coordinate system coordinates of each three of the four points. Then, calculate the ideal camera coordinate system coordinates according to the robot coordinate system coordinates of the fourth point and the transformation matrix. Finally, obtain the error between the ideal camera coordinate system coordinates and the actual camera coordinate system coordinates of the fourth point to evaluate the accuracy of the transformation matrix. Thus, the group with the smallest error among several groups of data is obtained, and its transformation relationship is used as the optimal initial transformation relationship.

4. The method for calibrating the target area spatial position of a medical surgical robot according to claim 1, wherein The dynamic registration is as follows: the infrared binocular camera recognizes and obtains the infrared feature point camera coordinate system coordinate set X in the current field of view. Assume that the total number of infrared feature points is N, with an arrangement H, and the corresponding relative distance set S between each point. The coordinates of the manipulator coordinate system and the CT coordinate system coordinate set X' recorded during spatial registration of these infrared feature points are: Arrangement H ′ n , corresponding to the relative distance set S ′ n , S and S ′ n Compare and find the S with the smallest comprehensive error with S ′ n , then the corresponding arrangement H ′ n The points in the arrangement H are matched one by one, and the conversion correspondence between the current camera coordinate system, the robotic arm coordinate system, and the CT coordinate system is solved by the two matched sets of coordinates. When the positional relationship between the infrared binocular camera, the robotic arm, and the target area changes, the coordinate system conversion correspondence is updated according to the dynamic registration to realize real-time dynamic spatial position calibration of the robot during operation.

5. The method for calibrating the target area spatial position of a medical surgical robot according to claim 4, characterized in that The infrared feature points of the base and the target area have different arrangement characteristics. Before dynamic registration, the infrared feature points recognized by the infrared binocular camera are first distinguished as the base infrared feature points or the target area infrared feature points according to the arrangement characteristics. Then, the dynamic matching relationship of the base infrared feature points or the target area infrared feature points is calculated respectively to obtain the corresponding relationship of the current coordinate system transformation.

6. The method for calibrating the target area spatial position of a medical surgical robot according to claim 1, wherein The infrared feature points refer to objects with geometric structures that can be captured and identified by an infrared binocular camera.

7. A target area spatial position calibration system for a medical surgical robot, characterized by The device comprises a CT device, an infrared binocular camera, a robot, infrared feature points, and a controller, wherein the controller is connected to the CT device, the infrared binocular camera, and the robot respectively, and the controller is configured with a computer program. The CT device, the infrared binocular camera, the robot, and the infrared feature points are configured according to the spatial position calibration method according to any one of claims 1 to 6, and the computer program is executed simultaneously to implement the spatial position calibration method according to any one of claims 1 to 6.

Images