A master-slave posture registration method for surgical robots
Through image recognition and multi-point contact force measurement based on frame difference method, combined with the closest point iteration algorithm, high-precision posture registration between the catheter and blood vessel of the robot intravascular interventional surgery is achieved, solving the problem of insufficient force tactile feedback in the prior art, and improving surgical effect and system safety.
Patent Information
- Application Number
- CN202210447243.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-24
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2042-04-24
AI Technical Summary
Existing vascular interventional surgery robots are difficult to accurately model and register the multi-point dynamic line contact between the flexible catheter and the blood vessel, resulting in insufficient force tactile feedback and affecting the surgical effect.
Image recognition based on frame difference method is used to measure catheter posture, combined with multi-point contact force measurement, and the nearest point iteration algorithm (ICP) is designed to minimize force tactile information errors and realize high-precision registration of catheter-vascular multi-point dynamic line contact.
The multi-point dynamic line contact model registration accuracy between catheter and blood vessel is improved, real-time and accurate force-tactile reproduction is achieved, and the transparency and safety of the surgical robot system is enhanced.
Smart Images

Figure CN114863417B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of catheter modeling and force tactile rendering, and relates to a high-precision master-slave posture registration method for a surgical robot. Background Art
[0002] As an assistant to doctors, surgical robots allow doctors to operate remotely, preventing the impact of X-rays on doctors, so that patients can receive timely treatment from expert doctors in special situations such as remote areas or battlefield surgeries. With the development of communication technologies such as 5G, telemedicine technology has been further improved, but the lack of force and tactile feedback of medical robots has become an important issue restricting its promotion.
[0003] The virtual reality of the main hand of the vascular interventional surgery robot establishes a three-dimensional spatial coordinate system to truly reflect the position of the catheter from the hand to the virtual end. The flexible catheter and the blood vessel are in multi-point dynamic line contact, and its bending and twisting are difficult to accurately model and align. And the force monitoring based on physical model calculation is difficult to avoid modeling errors, which affects the modeling control effect.
[0004] The model registration based on posture measurement in the current force tactile reproduction technology of vascular interventional robots is difficult to accurately simulate the dynamic process of catheter-vessel collision force. The catheter and the blood vessel are in dynamic multi-point line contact, and the model registration accuracy of catheter bending and torsional deformation is difficult to guarantee.
[0005] Therefore, it is extremely important to study a high-precision master-slave posture registration method for surgical robots. Summary of the invention
[0006] The purpose of the present invention is to improve the accuracy of the registration of the multi-point dynamic line contact model of the flexible catheter and the blood vessel, and to provide a high-precision master-slave posture registration method for a surgical robot. The present invention measures the posture of the soft section at the front end of the catheter based on the frame difference method image recognition to achieve preliminary model registration; based on the multi-point contact force measurement, a closest point iteration algorithm is designed with the goal of minimizing the force tactile information error to achieve accurate catheter-blood vessel multi-point dynamic line contact rendering dynamic model registration.
[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0008] A high-precision master-slave posture registration method for surgical robots, which first realizes the preliminary registration of catheter posture (i.e., position and posture) based on image recognition measurement, and then realizes dynamic and accurate registration of catheter posture based on multi-point contact force measurement;
[0009] The process of achieving dynamic and precise alignment of catheter posture based on multi-point contact force measurement is as follows: based on the measured force, the deformation displacement of the catheter contact point is solved, and through the relative distance between the catheters at the master hand and the slave hand, the rotation matrix and translation vector between the catheter model at the master hand and the catheter model at the slave hand are found based on the iterative closest point algorithm (ICP), that is, the optimal matching method based on the least squares method, so that the two matching data meet the optimal match under a certain metric.
[0010] As the preferred technical solution:
[0011] As described above, a high-precision surgical robot master-slave posture alignment method has the following preliminary alignment process: a catheter model is established at the master hand end using the Cosserat elastic rod theory, and the posture change of the soft segment at the front end of the catheter is measured at the slave hand end through image recognition. Based on the DH coordinate transformation method, the conversion matrix between the coordinate system of the catheter model at the master hand end and the coordinate system of the catheter model at the slave hand end is solved to achieve the association between the catheter model at the master hand end and the catheter model at the slave hand end, and the preliminary alignment of the catheter posture is achieved through the rotation matrix and the translation vector.
[0012] As described above, a high-precision master-slave registration method for surgical robots is often used in the initial registration stage. The ICP algorithm is used to match the catheter position data output by the monocular camera with the center line corresponding to the catheter path in the virtual three-dimensional image coordinate system to obtain the initial rotation matrix R when the catheter enters the aorta and selects the target bifurcation vessel to reach the target. k and the translation vector T k The essence of this is to find the exact corresponding points of the master and slave point sets, and to continuously iterate the rotation matrix and translation vector of the two point sets. During the initial registration, the rotation matrix and translation vector are obtained through the following process:
[0013] (i) Based on the preoperative three-dimensional virtual image coordinate space, the centerline data of the virtual three-dimensional image coordinate system of the master hand end, i.e., the master hand end catheter model point set Q, Q = {Q i |Q i ∈R 3 ,i=1,2,...,n},Q i represents the specific point in the master hand endpoint set, R 3 represents the three-dimensional space coordinates, n represents the number of points in the point set; the catheter path data in the actual coordinate system of the hand end is obtained in real time through the posture sensor, that is, from the catheter model point set P at the hand end, P = {P i |P i ∈R 3 ,i=1,2,...,n},P i represents the specific point from the hand endpoint set, R 3 represents the three-dimensional space coordinates, and n represents the number of points in the point set;
[0014] (ii) Indicates P i The kth iteration of Indicates P i k The nearest point in the corresponding Q, that is, the line connecting this point and Q1 is on the straight line Q1Q n The length of the projection line on P is the same as P in P i k The line connecting P1 is on the straight line P1P n The length of the projection line on is closest;
[0015] (iii) Calculate the rotation matrix R k and the translation vector T k , so that
[0016] (iv) Calculation P k+1 is the data of point set P at the k+1th iteration, P i k+1 is a specific point P in the point set i In the data of the k+1th iteration, R i k For R k The rotation matrix of the i-th point in , T i k T k The translation vector of the i-th point in ;
[0017] (v) Calculate the average position error of each point in the master hand catheter model point set and the slave hand catheter model point set
[0018] (vi) Determine || d k+1 -d k ||<τ whether it holds, d k and d k+1 are the average position errors of the kth and k+1th iterations, respectively. τ is the iteration threshold. If yes, the R at the last iteration is output. k and T k , as the rotation matrix and translation vector for preliminary registration; otherwise, return to step (iii).
[0019] The specific steps of the high-precision master-slave posture registration method of the surgical robot described above, in which the catheter path data in the actual coordinate system of the hand end is obtained in real time by the posture sensor, are as follows:
[0020] (i1) Place the entire transparent simulated blood vessel within the shooting range of the monocular camera;
[0021] (i2) Wrapping the front end of the catheter with a piece of paper marked with stripes A and B; before wrapping, stripes A and stripes B are connected to form a V shape; after wrapping, the plane where stripes A is located is perpendicular to the central axis of the catheter, and stripes A is located at the front end of stripes B;
[0022] (i3) placing the catheter in a simulated blood vessel and moving it in contact, while a monocular camera captures stripes A and B at a rate of 20 frames per second. After filtering, the image is converted into a grayscale image, and the pixel coordinates of the center points of stripes A and B are detected and read by a frame difference method;
[0023] (i4) calibrating the proportional relationship between the pixel coordinates and the coordinates in the actual scene by using the length and width of the blood vessels in the actual scene and the length and width of the blood vessels in the image, and then converting the pixel coordinates of the center points of stripe A and stripe B into actual coordinates by using the proportional relationship;
[0024] (i5) The translational degree of freedom displacement L of the front end of the catheter can be obtained by the actual coordinate of the center point of stripe A, and the rotational displacement θ can be obtained by the distance d between the actual coordinate of the center point of stripe A and the actual coordinate of the center point of stripe B.
[0025] (i6) The motion coordinates of the catheter are obtained through the translational freedom displacement L and the rotational displacement θ, that is, the catheter path data in the actual coordinate system of the hand end is obtained.
[0026] In the high-precision master-slave posture registration method for a surgical robot as described above, in step (i1), the shooting range of the monocular camera is 640×480 pixels, the resolution is 1920×1080, and the maximum frame rate is 30 FPS.
[0027] As described above, a high-precision master-slave registration method for surgical robots is similar to the ICP algorithm. Accurate registration also requires obtaining the transformation matrix. Unlike the preliminary registration, in the accurate registration stage, during the dynamic movement of the catheter, the influence of the catheter contact force on the effectiveness of the master-slave registration is taken into account, that is, the catheter movement is divided into free movement and contact movement. The error of the contact force between the master hand and the slave hand needs to be considered in the objective function of the accurate registration; during free movement, the master-slave registration error mainly exists between the actual posture of the slave hand and the virtual posture of the master hand. The error measurement function of the system is the ratio of the deviation between the actual posture of the slave hand and the virtual posture of the master hand divided by the actual posture of the slave hand; during accurate registration, the rotation matrix and the translation vector are obtained through the following process:
[0028] (a) Calculate the rotation matrix R i and the translation vector T i Make P i and Q iThey are the position of the catheter path from the hand end and the corresponding point of the main hand end blood vessel centerline, R i is the rotation matrix of the i-th point in R, T i is the translation vector of the ith point of T, n is the number of points in the model point set, Err(R,T) is the error between the catheter pose at the master end and the catheter pose at the slave end, where:
[0029] Δx i =F k / k;
[0030] k N1 Δz1+k N2 Δz2+…+k N(m-1) Δz m-1 +k Nm Δz m =F k ;
[0031] In the formula, Δx i is the deformation of the ith contact point; F k is the force on the spring mass, unit is N; k is the elastic coefficient, unit is N / m; k Ni is the elastic coefficient of the spring in the normal direction of the i-th particle, in N / m, i=1…m; Δz i is the deformation of the i-th particle in the normal direction, in units of m, i=1…m;
[0032] (b) Calculation in, and Represents the R of the kth iteration i and T i , P k+1 is the data of point set P at the k+1th iteration, P i k+1 is a specific point P in the point set i In the data of the k+1th iteration, For R k The rotation matrix of the i-th point in , T i k T k The translation vector of the i-th point in ;
[0033] (c) Calculate the average position error of each point in the master hand catheter model point set and the slave hand catheter model point set
[0034] (d) Judgement || d k+1 -d k ||<τ whether it holds, d k for and k+1are the average position errors of the kth and k+1th iterations, respectively. τ is the iteration threshold. If yes, the R at the last iteration is output. i and T i , as the rotation matrix and translation vector for accurate registration; otherwise, return to step (a).
[0035] Beneficial Effects
[0036] The present invention realizes real-time and accurate force and tactile reproduction of vascular intervention robot through model registration and posture prediction, improves system transparency, and has the following main features:
[0037] (1) Based on multi-point contact force measurement, a model alignment method is designed to minimize the force tactile reproduction error, and high-precision dynamic posture alignment of vascular interventional surgical robots is achieved; this invention can enrich and develop dynamic modeling theory and promote the clinical application of force tactile feedback technology for vascular interventional robots;
[0038] (2) Model registration is performed with the goal of minimizing the relative distance error between the master hand and the slave hand, thereby achieving accurate and dynamic model registration for multi-point dynamic line contact modeling; the present invention can perform safety monitoring from two aspects: force tactile rendering model calculation and contact force measurement, thereby improving the transparency and safety of the surgical robot system. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is the Cosserat elastic rod model;
[0040] Figure 2 Schematic diagram of catheter posture measurement based on frame difference method;
[0041] Figure 3 Schematic diagram of wire collision and contact force measurement based on optical fiber and conductive rubber sensors; wherein, 1-copper electrode, 2-conductive rubber, 3-catheter, 4-optical fiber pressure sensor;
[0042] Figure 4 Schematic diagram of the closest point selection strategy for the registration process;
[0043] Figure 5 It is the force analysis diagram based on the mass-spring model;
[0044] Figure 6 Schematic diagram of relative distances of corresponding points;
[0045] Figure 7 Comparison of the positions of path points of the traditional ICP method and the precise registration of the present invention; wherein (a) is the master-slave catheter path without registration, (b) is the master-slave catheter path after registration by the traditional ICP method, and (c) is the master-slave catheter path after precise registration;
[0046] Figure 8 Comparison of the registration errors of the traditional ICP and the precise registration of the present invention; wherein, (a) is the traditional ICP registration error, and (b) is the precise registration error of the present invention;
[0047] Fig. 9 Comparison between the catheter path data after registration and the actual path. DETAILED DESCRIPTION
[0048] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.
[0049] A high-precision master-slave posture registration method for a surgical robot, the specific steps are as follows:
[0050] (1) Preliminary alignment of the catheter position based on image recognition measurement;
[0051] At the master hand end, the catheter model was established using the Cosserat elastic rod theory (e.g. Figure 1 As shown in the figure, n represents the number of discrete nodes of the catheter model, and the discrete nodes x0, x1, x2...x n Divide the entire line into multiple line segments e0, e1, e2...e n-1 , where e i-1 =x i -x i-1 ), the position and posture change of the soft section at the front end of the catheter is measured by image recognition at the slave end, and the conversion matrix between the coordinate system of the catheter model at the master end and the coordinate system of the catheter model at the slave end is solved based on the DH coordinate transformation method to realize the association between the catheter model at the master end and the catheter model at the slave end, and the preliminary alignment of the catheter position and posture is realized through the rotation matrix and the translation vector;
[0052] The rotation matrix and translation vector are obtained through the following process:
[0053] (i) Based on the preoperative three-dimensional virtual image coordinate space, the centerline data of the virtual three-dimensional image coordinate system of the master hand end, i.e., the master hand end catheter model point set Q, Q = {Q i |Q i ∈R 3 ,i=1,2,...,n},Q i represents the specific point in the master hand endpoint set, R 3 represents the three-dimensional space coordinates, n represents the number of points in the point set; the catheter path data in the actual coordinate system of the hand end is obtained in real time through the posture sensor, that is, from the catheter model point set P at the hand end, P = {P i |Pi ∈R 3 ,i=1,2,...,n},P i represents the specific point from the hand endpoint set, R 3 represents the three-dimensional space coordinates, and n represents the number of points in the point set;
[0054] The specific steps for obtaining the catheter path data in the actual coordinate system of the hand end in real time through the posture sensor are as follows:
[0055] (i1) Place the entire transparent simulated blood vessel within the shooting range of a monocular camera; the shooting range of the monocular camera is 640×480 pixels, the resolution is 1920×1080, and the maximum frame rate is 30FPS;
[0056] (i2) Figure 2 As shown, a piece of paper marked with stripes A and B is used to wrap the front end of the catheter; before wrapping, stripes A and stripes B are connected to form a V shape; after wrapping, the plane where stripes A is located is perpendicular to the central axis of the catheter, and stripes A is located at the front end of stripes B;
[0057] (i3) placing the catheter in a simulated blood vessel and moving it in contact, while a monocular camera captures stripes A and B at a rate of 20 frames per second. After filtering, the image is converted into a grayscale image, and the pixel coordinates of the center points of stripes A and B are detected and read by a frame difference method;
[0058] (i4) calibrating the proportional relationship between the pixel coordinates and the coordinates in the actual scene by using the length and width of the blood vessels in the actual scene and the length and width of the blood vessels in the image, and then converting the pixel coordinates of the center points of stripe A and stripe B into actual coordinates by using the proportional relationship;
[0059] (i5) The translational degree of freedom displacement L of the front end of the catheter can be obtained by the actual coordinate of the center point of stripe A, and the rotational displacement θ can be obtained by the distance d between the actual coordinate of the center point of stripe A and the actual coordinate of the center point of stripe B.
[0060] (i6) obtaining the motion coordinates of the catheter through the translational degree of freedom displacement L and the rotational displacement θ, that is, obtaining the catheter path data in the actual coordinate system from the hand end;
[0061] (ii) P i k Indicates P i The kth iteration of Indicates P i k The nearest point in the corresponding Q, that is, the line connecting this point and Q1 is on the straight line Q1Q n The length of the projection line on P is the same as P in P i kThe line connecting P1 is on the straight line P1P n The length of the projection line on is closest; Figure 4 As shown;
[0062] (iii) Calculate the rotation matrix R k and the translation vector T k , so that
[0063] (iv) Calculation P k+1 is the data of point set P at the k+1th iteration, P i k+1 is a specific point P in the point set i In the data of the k+1th iteration, R i k For R k The rotation matrix of the i-th point in , T i k T k The translation vector of the i-th point in ;
[0064] (v) Calculate the average position error of each point in the master hand catheter model point set and the slave hand catheter model point set
[0065] (vi) Determine || d k+1 -d k ||<τ (τ is 1mm) whether it holds true, d k and d k+1 are the average position errors of the kth and k+1th iterations respectively. If yes, the R at the last iteration is output. k and T k , as the rotation matrix and translation vector for preliminary registration; otherwise, return to step (iii);
[0066] (2) Dynamic and precise alignment of catheter position based on multi-point contact force measurement;
[0067] Based on the Iterative Closest Point (ICP) algorithm, the rotation matrix and translation vector between the master hand catheter model and the slave hand catheter model are found so that the two matching data meet the optimal match under a certain metric.
[0068] Compared with the preliminary registration, the precise registration is based on the measured force to solve the deformation displacement Δx of the catheter contact point i , through the concept of relative distance, calculate the optimal transformation matrix, Figure 6Schematic diagram of the relative distance between the catheter path point and the corresponding point on the center line of the blood vessel, where point a and point b are the collision contact points between the catheter and the blood vessel wall, Δx1 and Δx2 are the deformations when the contact force is applied, and d1 and d2 are the distances between point a, point b and their corresponding points, respectively. To measure the registration accuracy, the influence of position offset caused by force deformation on the registration accuracy can be reduced; the precise registration objective function takes force deformation into consideration, and the relative distance between the corresponding points of the master hand and the slave hand is minimized through iteration to solve the transformation matrix.
[0069] like Figure 3 As shown, the contact force at the front end of the catheter 5 is measured by the optical fiber pressure sensor 6 installed at the front end of the catheter, and the contact force at the side wall of the catheter 5 is measured by the piezoresistive effect of the conductive rubber 4 coated on the side wall of the catheter; the "copper electrode 3" is powered; the probe diameter of the optical fiber pressure sensor is 0.25mm, the accuracy is ±2mmHg, the resolution is 0.5mmHg, the outer diameter is 1mm, the length is 4m, the surface is coated with PTFE (polytetrafluoroethylene), and the maximum sampling frequency is 250Hz (that is, the sampling frequency does not exceed 250Hz); the optical fiber pressure sensor adopts white light interference propagation, and will not be interfered by electromagnetic signals, radio frequency signals, magnetic resonance and other microwave signals in the application environment, and can well maintain its original accuracy, Repeatability, it can be easily measured, can be used repeatedly, and has electromagnetic compatibility, which can meet the general requirements of vascular intervention: ① It is soft enough, harmless to the human body, and can adapt to the complex and changeable shapes of human blood vessels; ② It is small in size and easy to integrate with the catheter; ③ It has high sensitivity; The conductive rubber is a silicone rubber with conductive particles evenly dispersed inside; Under the action of pressure, the conductive particles in the conductive rubber come into contact to produce a piezoresistive effect; The conductive rubber has a simple and reliable structure, which can fully guarantee the real-time and accuracy of force measurement, and can meet the general requirements of vascular intervention: ① It is soft enough, harmless to the human body, and can adapt to the complex and changeable shapes of human blood vessels; ② It is small in size and easy to integrate with the catheter; ③ It has high sensitivity;
[0070] The rotation matrix and translation vector are obtained through the following process:
[0071] (a) Calculate the rotation matrix R i and the translation vector T i Make P i and Q i They are the position of the catheter path from the hand end and the corresponding point of the main hand end blood vessel centerline, R i is the rotation matrix of the i-th point in R, T i is the translation vector of the ith point of T, n is the number of points in the model point set, Err(R,T) is the error between the catheter posture of the master end and the catheter posture of the slave end, where Δxi is the deformation of the ith contact point, Δx i It is calculated based on the force measurement at the hand end. Considering that the blood vessel wall deforms in the area centered on the contact point, its deformation behavior is related to the adjacent springs, that is, the total deformation displacement under the action of the adjacent springs needs to be solved. The force analysis is as follows: Figure 5 As shown;
[0072] F k =k N1 Δz1+k T1 Δr1sinθ1=f N1 +f T1 sinθ1;
[0073] Among them, f T1 represents the tangential force, f N1 represents the normal force, k N1 represents the elastic coefficient of the spring in the direction of the mass point normal, Δr1 represents the deformation of the spring between adjacent mass points in the tangential direction, Δz1 represents the deformation of the mass point in the normal direction, and θ1 is the angle between the line connecting the adjacent mass points and the horizontal direction; for i spring mass points, there are:
[0074] f N1 +f N2 +…+f N(i-1) +f N i=F k -f T isinθ i ;
[0075] Assume that when i = m, sinθ i =0, then
[0076] k N1 Δz1+k N2 Δz2+…+k N(m-1) Δz m-1 +k Nm Δz m =F k ;
[0077] Among them, k Ni is the elastic coefficient of the spring in the normal direction of the i-th mass point, in N / m, i=1…m; Δzi is the deformation of the i-th mass point in the normal direction, in m, i=1…m; the total deformation displacement Δx in the area of each contact point i The expression is as follows:
[0078] Δx i =F k / k;
[0079] Among them, F kis the force on the spring mass, unit is N; k is the elastic coefficient, unit is N / m;
[0080] (b) Calculation in, and T i k Represents the R of the kth iteration i and T i , P k+1 is the data of point set P at the k+1th iteration, P i k+1 is a specific point P in the point set i In the data of the k+1th iteration, For R k The rotation matrix of the i-th point in , T i k T k The translation vector of the i-th point in ;
[0081] (c) Calculate the average position error of each point in the master hand catheter model point set and the slave hand catheter model point set
[0082] (d) Judgement || d k+1 -d k ||<τ whether it holds, d k for and k+1 are the average position errors of the kth and k+1th iterations respectively. If yes, the R at the last iteration is output. i and T i , as the rotation matrix and translation vector for accurate registration; otherwise, return to step (a).
[0083] During the experiment, the operator held the Omega.7 handle at the master hand end and pushed, pulled, or twisted it to insert the catheter at the end of the UR5 robot clamped at the slave hand end into the blood vessel, driving the catheter from the aorta through the bifurcation to the target lesion. At the same time, the catheter front end position data collected by the monocular camera and force sensor at the slave hand end was identified and processed to obtain the catheter path, which was then aligned with the catheter path at the master hand end. Figure 7 The position data of each path point after using the traditional ICP registration method and the high-precision posture registration method of the present invention, the master-slave catheter path position accuracy obtained after the high-precision posture registration method of the present invention is significantly higher than the traditional ICP registration method.
[0084] Calculate the registration error corresponding to the catheter path point as Figure 8As shown, during the registration process, the error gradually decreases, the error of the first iteration is the largest, and the error reduction is also the largest. As the number of iterations increases, the amplitude finally stabilizes. The traditional ICP registration method does not consider the influence of force, and the final registration root mean square error reaches 6.72mm. The high-precision posture registration method of the present invention reduces the registration error. As the number of iterations in the registration process increases, the acquired transformation matrix becomes more and more accurate, the corresponding registration error becomes smaller and smaller, and finally stabilizes at about 1.02mm.
[0085] Fig. 9 The real path and the registered path are shown. It can be seen from the figure that according to the registration result, the master hand end completes the real-time and accurate display of the catheter position. The precise registration accuracy of the present invention can meet the requirements of interventional surgery.
Claims
1. A method for master-slave posture registration of a surgical robot, characterized by: First, the catheter position is preliminarily aligned based on image recognition measurement, and then the catheter position is dynamically and accurately aligned based on multi-point contact force measurement. The process of realizing dynamic and accurate registration of catheter position based on multi-point contact force measurement is as follows: based on the measurement force, the deformation displacement of the catheter contact point is solved, and based on the closest point iteration algorithm, the rotation matrix and translation vector between the catheter model of the master hand end and the catheter model of the slave hand end are found, so that the two matching data meet the optimal matching under a certain metric; The process of preliminary registration is as follows: the catheter model is established at the master hand end using the Cosserat elastic rod theory, the position and posture changes of the soft segment at the front end of the catheter are measured at the slave hand end through image recognition, and the conversion matrix between the coordinate system of the catheter model at the master hand end and the coordinate system of the catheter model at the slave hand end is solved based on the DH coordinate transformation method to achieve the association between the catheter model at the master hand end and the catheter model at the slave hand end, and the preliminary registration of the catheter position and posture is achieved through the rotation matrix and translation vector; During the initial registration, the rotation matrix and translation vector are obtained through the following process: (i) Based on the preoperative three-dimensional virtual image coordinate space, the centerline data of the virtual three-dimensional image coordinate system of the master hand end, i.e., the master hand end catheter model point set Q, Q = {Q i |Q i ∈R 3 ,i=1,2,...,n},Q i represents the specific point in the master hand endpoint set, R 3 represents the three-dimensional space coordinates, n represents the number of points in the point set; the catheter path data in the actual coordinate system of the hand end is obtained in real time through the posture sensor, that is, from the catheter model point set P at the hand end, P = {P i |P i ∈R 3 ,i=1,2,...,n},P i represents the specific point from the hand endpoint set, R 3 represents the three-dimensional space coordinates, and n represents the number of points in the point set; (ii) P i k Indicates P i The kth iteration of Indicates P i k The length of the projection line of the line connecting this point and Q1 on the straight line Q1Qn is equal to the length of the projection line of P in P. i k The line connecting P1 is on the straight line P1P n The length of the projection line on is closest; (iii) Calculate the rotation matrix R of point set Q and point set P k and the translation vector T k , so that (iv) Calculation P k+1 is the data of point set P at the k+1th iteration, P i k+1 is a specific point P in the point set i In the data of the k+1th iteration, For R k The rotation matrix of the i-th point in , T i k T k The translation vector of the i-th point in ; (v) Calculate the average position error of each point in the master hand catheter model point set and the slave hand catheter model point set (vi) Determine || d k+1 -d k ||<τ whether it holds, d k and d k+1 are the average position errors of the kth and k+1th iterations, respectively. τ is the iteration threshold. If yes, the R at the last iteration is output. k and T k , as the rotation matrix and translation vector for preliminary registration; otherwise, return to step (iii).
2. A surgical robot master-slave posture registration method according to claim 1, characterized in that: The specific steps for obtaining the catheter path data in the actual coordinate system of the hand end in real time through the posture sensor are as follows: (i1) Place the entire transparent simulated blood vessel within the shooting range of the monocular camera; (i2) Wrap the front end of the catheter with a piece of paper marked with stripes A and B; Before wrapping, stripe A and stripe B are connected to form a V shape; after wrapping, the plane where stripe A is located is perpendicular to the central axis of the catheter, and stripe A is located at the front end of stripe B; (i3) placing the catheter in a simulated blood vessel and moving it in contact, while a monocular camera captures stripes A and B at a rate of 20 frames per second. After filtering, the image is converted into a grayscale image, and the pixel coordinates of the center points of stripes A and B are detected and read by a frame difference method; (i4) calibrating the proportional relationship between the pixel coordinates and the coordinates in the actual scene by using the length and width of the blood vessels in the actual scene and the length and width of the blood vessels in the image, and then converting the pixel coordinates of the center points of stripe A and stripe B into actual coordinates by using the proportional relationship; (i5) The translational degree of freedom displacement L of the front end of the catheter can be obtained by the actual coordinate of the center point of stripe A, and the rotational displacement θ can be obtained by the distance d between the actual coordinate of the center point of stripe A and the actual coordinate of the center point of stripe B. (i6) The motion coordinates of the catheter are obtained through the translational freedom displacement L and the rotational displacement θ, that is, the catheter path data in the actual coordinate system of the hand end is obtained.
3. A surgical robot master-slave posture registration method according to claim 2, characterized in that: In step (i1), the shooting range of the monocular camera is 640×480 pixels, the resolution is 1920×1080, and the maximum frame rate is 30FPS.
4. A surgical robot master-slave posture registration method according to claim 2, characterized in that: During precise registration, the rotation matrix and translation vector are obtained through the following process: (a) Calculate the rotation matrix R i and the translation vector T i Make P i and Q i They are the position of the catheter path from the hand end and the corresponding point of the main hand end blood vessel centerline, R i is the rotation matrix of the i-th point in R, T i is the translation vector of the ith point of T, n is the number of points in the model point set, Err(R,T) is the error between the catheter pose at the master end and the catheter pose at the slave end, where: Δx i =F k / k; k N1 Δz1+k N2 Δz2+…+k N(m-1) Δz m-1 +k Nm Δz m =F k ; In the formula, Δx i is the deformation of the ith contact point; F k is the force on the spring mass, unit is N; k is the elastic coefficient, unit is N / m; k Ni is the elastic coefficient of the spring in the normal direction of the i-th particle, in N / m, i=1…m; Δz i is the deformation of the i-th particle in the normal direction, in units of m, i=1…m; (b) Calculation in, and T i k Represents the R of the kth iteration i and T i , P k+1 is the data of point set P at the k+1th iteration, P i k+1 is a specific point P in the point set i In the data of the k+1th iteration, For R k The rotation matrix of the i-th point in , T i k T k The translation vector of the i-th point in ; (c) Calculate the average position error of each point in the master catheter model point set and the slave catheter model point set (d) Judgement || d k+1 -d k ||<τ whether it holds, d k for and k+1 are the average position errors of the kth and k+1th iterations, respectively. τ is the iteration threshold. If yes, the R at the last iteration is output. i and T i , as the rotation matrix and translation vector for accurate registration; otherwise, return to step (a).
Citation Information
Patent Citations
Wearable three-dimensional image navigation device for vascular intervention operation and realizing man-machine interaction
CN105395252A
Method for augmenting a surgical with virtual guidance content
WO2018200767A1