Surgical instrument tip positioning method based on AprilTag code

Abstract

The invention relates to an AprilTag code-based surgical instrument tip positioning method, which comprises the following steps of: adding an octagonal prism with a plurality of AprilTag marks at the tail part of a surgical instrument, and calculating the accurate position of the tip of the surgical instrument in a camera coordinate system by combining image information captured by a camera. According to the method, high detection precision and stability of an AprilTag mark are utilized, a transformation relation between a tool and a camera coordinate system is determined through a calibration process, and high-precision and high-robustness surgical instrument positioning is realized by combining real-time pose detection and a nonlinear optimization algorithm. The problems that a traditional marker is prone to being shielded and low in operation flexibility are solved, and the positioning precision and reliability of a surgical navigation system are improved.

Description

Technical Field

[0001] The invention relates to a precise positioning method for medical instruments, in particular to a positioning method for the tip of a surgical instrument based on an AprilTag code. Background Art

[0002] Precise positioning of surgical instruments is crucial for surgical navigation systems. Visual tracking methods based on optical positioning have become the most mainstream means of surgical instrument positioning due to their advantages in application scenarios, accuracy, stability and cost. AprilTag code is a visual reference system first proposed by Professor Edwin Olson of the University of Michigan and his experimental team. It can be used for various tasks, including augmented reality, robotics and camera calibration. Targets can be created from ordinary printers, and the AprilTag detection software can calculate the precise 3D position, orientation and identification of the tag relative to the camera.

[0003] Generally speaking, the optical positioning method uses a camera to capture images of markers installed on surgical instruments. After detecting and identifying the markers on the image, the spatial position of the markers is calculated using a visual algorithm, and then the three-dimensional coordinates of the surgical instrument tip in the camera coordinate system are indirectly calculated.

[0004] Traditional markers usually select multiple spherical reflectors or black and white grid patterns pasted on a plane to form a certain plane structure, such as a triangle, quadrilateral, etc., to meet the positioning requirements. However, this type of marker has the following disadvantages: 1) It takes up a lot of space and affects the operation: installing this type of marker on the surgical instrument requires more space, which increases the volume and weight of the instrument, affecting the doctor's flexibility and accuracy during the operation. 2) It is easy to be blocked and has poor reliability: when the angle between the plane composed of the markers and the camera optical axis is small, it is easy for the markers or the surgical instrument and the marker to block each other. In addition, the marker pattern pasted on the plane needs to be approximately facing the camera to be accurately detected and identified, which limits the operating angle and range of the surgical instrument. 3) Lack of diversity and difficulty in improving accuracy: This type of marker can only provide a single pose estimation result, and it is impossible to further improve the positioning accuracy and system reliability by fusing multiple sets of estimation results. Therefore, designing a new surgical instrument tip positioning method to overcome the shortcomings of the traditional optical positioning method and meet the needs of the surgical navigation system for high-precision and high-robustness positioning has become a problem that needs to be solved by technicians in this field.

[0005] In order to solve the above problems, the present invention proposes a method for positioning the tip of a surgical instrument based on the AprilTag code, which has the following advantages: 1) The AprilTag code can be printed on a small-sized medium, which is convenient for attachment to different parts of the surgical instrument. Its unique design enables it to be accurately detected even when partially blocked, reducing the problem of positioning failure caused by occlusion. 2) The method of fusing the postures of multiple markers at different viewing angles is adopted, and the posture information provided by multiple AprilTag codes is fused and optimized through the Levenberg-Marquardt nonlinear optimization algorithm, thereby improving the accuracy and robustness of positioning the tip of the surgical instrument. 3) Real-time posture optimization calculation is used to ensure that the positioning accuracy can be quickly adjusted when the surgical instrument moves, reducing the lag effect. Summary of the invention

[0006] The present invention adds an octagonal prism structure to the tail of the surgical instrument, referred to as a tool, exposing nine faces, and affixing an AprilTag code to each face as a marker. Figure 3 As shown in the figure, an improved surgical instrument tip positioning method is proposed by using these markers distributed on different surfaces and establishing a surgical instrument coordinate system based on the tool. The positioning method is divided into a calibration process and a positioning process. In the calibration process, the transformation relationship between the tool coordinate system and the camera coordinate system is determined by detecting the markers on the tool. Since the markers are distributed on different surfaces of the tool, they provide rich spatial information and avoid the degradation of posture estimation. The relative position of the surgical instrument tip in the tool coordinate system is calculated as the calibration result.

[0007] During the positioning process, the calibration results and the real-time detected position of the marker on the tool in the camera coordinate system can be used to accurately calculate the position of the surgical instrument tip in the camera coordinate system in real time, thereby achieving high-precision positioning of the surgical instrument tip.

[0008] In order to achieve the purpose of the present invention, the following technical solution will be adopted: a surgical instrument tip positioning method based on AprilTag code. The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments:

[0009] S1. Prepare the environment and equipment

[0010] S1-1. Determine the camera position in the calibration environment and obtain the camera's intrinsic parameters and distortion parameters.

[0011] The camera intrinsic parameter matrix K is expressed as

[0012]

[0013] where f x and f y are the focal lengths of the camera in the x and y directions, cx and c y are the coordinates of the camera's principal point (usually located at the center of the image).

[0014] S1-2. Figure 1 As shown in the figure, a tool is attached to the tail of the surgical instrument, and a marker is printed on each visible surface, for a total of nine markers. Each marker has a different ID, and i is used to represent the coordinate system of a marker with an ID equal to i. The geometric center of the tool is used as the origin of the tool coordinate system. According to the geometric shape of the tool, the tool coordinate system is defined and represented by tool. The Z axis points to the tool tip along the axial direction of the tool, and the X and Y axes are in the cross-sectional plane of the tool.

[0015] Each marker on the tool has a known fixed position and orientation relative to the tool coordinate system. After defining the tool coordinate system, the 3×1 translation vector t of each marker in the tool coordinate system can be obtained. i→tool and the 3×3 rotation matrix R i→tool .

[0016] S1-3. Prepare the calibration plate, such as Figure 2 As shown, the calibration plate is a square plane with four markers located at the four corners of the calibration plate. The center point is defined as the origin of the calibration plate coordinate system O board A dot is printed at the origin for placing the tool tip during calibration. The calibration board coordinate system is represented by board.

[0017] S1-4. Arrange the markers according to the pre-designed plan to ensure that at least one marker can be detected by the camera in various postures of the tool.

[0018] S2. Calibration of surgical instrument tips

[0019] S2-1. Figure 3 As shown, place the tip of the surgical instrument accurately at O board superior.

[0020] S2-2. The camera coordinate system is represented by cam. The calibration plate is placed in the camera field of view. The four markers on the calibration plate are detected. The built-in pose estimation function detect in the apriltags library is used to obtain the rotation matrix and translation vector R of the markers on the calibration plate in the camera coordinate system. i→cam ,t i→cam .

[0021] S2-3. According to the translation vector t of the four markers on the calibration plate under the cam i→cam , calculate O board The translation vector O under the cam board→cam .

[0022]

[0023] O board→cam As the three-dimensional coordinates of the surgical instrument tip in the camera coordinate system.

[0024] S2-4. Fix the tool axis at the tail of the surgical instrument, fix the surgical instrument and rotate the tool slowly so that each marker on the tool is exposed to the camera field of view in turn. During the rotation process, use the camera to continuously capture images and detect the R of all markers appearing in each frame of the image. i→cam and t i→cam , use the built-in pose estimation function detect in the apriltags library to calculate the translation vector t of each marker on the tool under the cam i→cam and the rotation matrix R i→cam .

[0025] S2-5. For the markers on the tool, the number of detections and position data of each specific ID marker are accumulated during the acquisition process, t i→cam Make sure that each marker ID is detected n times (n>0) to obtain enough pose data for optimized calibration.

[0026] S2-6 Quaternion has advantages over rotation matrix in representing rotation and synthesizing multiple rotations, especially in avoiding cumulative errors. i→tool Alternative R i→tool Indicates that the quaternion q i→cam Alternative R i→cam express.

[0027] S2-7. The overall rotation of the tool in the camera coordinate system is expressed using the quaternion q tool→cam Indicates that the translation is t tool→cam The position and posture of each marker on the tool in the camera coordinate system are calculated by the following formula:

[0028] Rotation relationship:

[0029] q i→cam =q tool→cam ·q i→tool

[0030] Translation relationship:

[0031] t i→cam =R tool→cam ·t i→tool +t tool→cam

[0032] q tool→cam and t tool→cam is the pose information that needs to be solved.

[0033] S2-8. Using the quaternion inner product q i→cam ·(q tool→cam ·q i→tool ) measure q i→cam With q tool→cam ·q i→tool The rotation error is Indicates that the range is between [-1,1]. The rotation error is defined as

[0034]

[0035] Taking the absolute value ensures that the rotation error measure is non-negative

[0036] Use Euclidean distance to measure t i→cam and R tool→cam ·t i→tool +t tool→cam The translation error is expressed as E trans express

[0037]

[0038] The pose error of the marker on a single tool is expressed as express

[0039]

[0040] S2-9. The overall optimization goal is to minimize the E of all markers collected. total The sum of the tool's overall center position relative to the camera coordinate system is calculated, denoted by e:

[0041]

[0042] S2-10. Levenberg-Marquardt method, referred to as LM method, this method combines the gradient descent method and Gauss-Newton method, and the update steps are

[0043] Δx=-(J T J+λI) -1 J T E

[0044] Where Δx=[Δq,Δt] is the parameter vector to be optimized, including q tool→cam and t tool→cam , I is the identity matrix. Q tool→cam and t tool→cam The partial derivative matrices of are stacked to obtain J, which represents the overall Jacobian matrix. is the error vector, which contains the rotation error and translation error of all markers.

[0045] S2-11. Use the LM method to calculate the parameter update step size Δx = [Δq, Δt]. Each iteration uses this Δx to adjust q tool→cam and t tool→cam .

[0046] q tool→cam Update: Change the current q tool→cam It is added to its update component Δq and normalized by dividing it by its L2 norm.

[0047]

[0048] t tool→cam Update: Change the current t tool→cam Add to its update component Δt

[0049] t tool→cam ←t tool→cam +Δt

[0050] S2-12. Repeat S2-7 to 2-11 until the error converges to less than the threshold ε or the number of iterations reaches the preset value. Get the optimal q tool→cam and t tool→cam . Use the Rodrigues formula to convert the quaternion q tool→cam Convert to rotation matrix R tool→cam .

[0051] S2-13. When the tool tip is aligned with the center of the calibration plate, the translation vector of the surgical instrument tip in the camera coordinate system is t tip→cam Meaning: t tip→cam =O board→cam .

[0052] Establish the position and posture relationship of the surgical instrument tip in the camera coordinate system.

[0053] t tip→cam =R tool→cam ·t tip→tool +t tool→cam

[0054] Based on this relationship, the position of the surgical instrument tip in the tool coordinate system is deduced and the calibration result t is obtained. tip→tool

[0055] t tip→tool =(R tool→cam ) T (t tip→cam -t tool→cam )

[0056] S3. Real-time positioning of surgical instrument tip

[0057] S3-1. Use the built-in pose estimation function detect in the apriltags library to detect the markers on the tool and detect the R of multiple markers of the tool on a frame image. i→cam ,t i→cam . Keep the R of all the markers in the past k frames (k>0) i→cam ,t i→cam

[0058] S3-2. Using the t obtained by calibration tip→tool , calculate the preliminary position estimate of the surgical instrument tip in the camera coordinate system.

[0059] t tip→cam =R tool→cam ·t tip→tool +t tool→cam

[0060] S3-4. Using the known position of the marker in the tool coordinate system and orientation The R of all the markers in the past k frames stored i→cam ,t i→cam As input, the nonlinear optimization method from S2-7 to 2-12 and the position relationship of the surgical instrument tip in the camera coordinate system established by S2-13 are used to obtain the real-time translation vector t of the surgical instrument tip in the camera coordinate system. tip→cam .

[0061] Through the above steps, if Figure 4 As shown in the flow chart, the present invention provides a method for positioning the tip of a surgical instrument based on the AprilTag code, which calculates the position of the tool tip in the camera coordinate system in real time by printing multiple AprilTag codes on a calibration plate and a tool, combined with the conversion between the tool coordinate system and the camera coordinate system. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is an octagonal prism with markings.

[0063] Figure 2 It is a calibration plate with markings.

[0064] Figure 3 It is a schematic diagram of the structure of the surgical instrument tip positioning system based on AprilTag marking of the present invention.

[0065] Figure 4 This is a flow chart of the surgical instrument tip positioning method based on AprilTag code of the present invention.

[0066] Figure 5 It is a planar structure with markers.

[0067] Figure 6Experimental results for evaluating positioning accuracy of the experimental group.

[0068] Figure 7 Experimental results for evaluating positioning accuracy of the control group.

[0069] Figure 8 It is the CDF result of the experimental group and the control group relative to the ground truth. DETAILED DESCRIPTION

[0070] The present invention adds an octagonal prism structure to the tail of the surgical instrument, referred to as a tool, exposing nine faces, and affixing an AprilTag code to each face as a marker. Figure 3 As shown in the figure, an improved surgical instrument tip positioning method is proposed by using these markers distributed on different surfaces and establishing a surgical instrument coordinate system based on the tool. The positioning method is divided into a calibration process and a positioning process. In the calibration process, the transformation relationship between the tool coordinate system and the camera coordinate system is determined by detecting the markers on the tool. Since the markers are distributed on different surfaces of the tool, they provide rich spatial information and avoid the degradation of posture estimation. The relative position of the surgical instrument tip in the tool coordinate system is calculated as the calibration result.

[0071] During the positioning process, the calibration results and the real-time detected position of the marker on the tool in the camera coordinate system can be used to accurately calculate the position of the surgical instrument tip in the camera coordinate system in real time, thereby achieving high-precision positioning of the surgical instrument tip.

[0072] In order to achieve the purpose of the present invention, the following technical solution will be adopted: a surgical instrument tip positioning method based on AprilTag code. The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and embodiments:

[0073] S1. Prepare the environment and equipment

[0074] S1-1. Determine the camera position in the calibration environment and obtain the camera's intrinsic parameters and distortion parameters.

[0075] The camera intrinsic parameter matrix K is expressed as

[0076]

[0077] where f x and f y are the focal lengths of the camera in the x and y directions, c x and c y are the coordinates of the camera's principal point (usually located at the center of the image).

[0078] S1-2. Figure 1As shown in the figure, a tool is attached to the tail of the surgical instrument, and a marker is printed on each visible surface, for a total of nine markers. Each marker has a different ID, and i is used to represent the coordinate system of a marker with an ID equal to i. The geometric center of the tool is used as the origin of the tool coordinate system. According to the geometric shape of the tool, the tool coordinate system is defined and represented by tool. The Z axis points to the tool tip along the axial direction of the tool, and the X and Y axes are in the cross-sectional plane of the tool.

[0079] Each marker on the tool has a known fixed position and orientation relative to the tool coordinate system. After defining the tool coordinate system, the 3×1 translation vector t of each marker in the tool coordinate system can be obtained. i→tool and the 3×3 rotation matrix R i→tool .

[0080] S1-3. Prepare the calibration plate, such as Figure 2 As shown, the calibration plate is a square plane with four markers located at the four corners of the calibration plate. The center point is defined as the origin of the calibration plate coordinate system O board A dot is printed at the origin for placing the tool tip during calibration. The calibration board coordinate system is represented by board.

[0081] S1-4. Arrange the markers according to the pre-designed plan to ensure that at least one marker can be detected by the camera in various postures of the tool.

[0082] S2. Calibration of surgical instrument tips

[0083] S2-1. Figure 3 As shown, place the tip of the surgical instrument accurately at O board superior.

[0084] S2-2. The camera coordinate system is represented by cam. The calibration plate is placed in the camera field of view. The four markers on the calibration plate are detected. The built-in pose estimation function detect in the apriltags library is used to obtain the rotation matrix and translation vector R of the markers on the calibration plate in the camera coordinate system. i→cam ,t i→cam .

[0085] S2-3. According to the translation vector t of the four markers on the calibration plate under the cam i→cam , calculate O board The translation vector O under the cam board→cam .

[0086]

[0087] O board→cam As the three-dimensional coordinates of the surgical instrument tip in the camera coordinate system.

[0088] S2-4. Fix the tool axis at the tail of the surgical instrument, fix the surgical instrument and rotate the tool slowly so that each marker on the tool is exposed to the camera field of view in turn. During the rotation process, use the camera to continuously capture images and detect the R of all markers appearing in each frame of the image. i→cam and t i→cam , use the built-in pose estimation function detect in the apriltags library to calculate the translation vector t of each marker on the tool under the cam i→cam and the rotation matrix R i→cam .

[0089] S2-5. For the markers on the tool, the number of detections and position data of each specific ID marker are accumulated during the acquisition process, t i→cam Make sure that each marker ID is detected n times (n>0) to obtain enough pose data for optimized calibration.

[0090] S2-6 Quaternion has advantages over rotation matrix in representing rotation and synthesizing multiple rotations, especially in avoiding cumulative errors. i→tool Alternative R i→tool Indicates that the quaternion q i→cam Alternative R i→cam express.

[0091] S2-7. The overall rotation of the tool in the camera coordinate system is expressed using the quaternion q tool→cam Indicates that the translation is t tool→cam The position and posture of each marker on the tool in the camera coordinate system are calculated by the following formula:

[0092] Rotation relationship:

[0093] q i→cam =q tool→cam ·q i→tool

[0094] Translation relationship:

[0095] t i→cam =R tool→cam ·t i→tool +t tool→cam

[0096] q tool→cam and t tool→cam is the pose information that needs to be solved.

[0097] S2-8. Using the quaternion inner product q i→cam ·(q tool→cam ·q i→tool ) measure q i→cam With q tool→cam ·qi→tool The rotation error is Indicates that the range is between [-1,1]. The rotation error is defined as

[0098]

[0099] Taking the absolute value ensures that the rotation error is non-negative and using the Euclidean distance to measure t i→cam and R tool→cam ·t i→tool +t tool→cam The translation error is expressed as E trans express

[0100]

[0101] The pose error of the marker on a single tool is expressed as express

[0102]

[0103] S2-9. The overall optimization goal is to minimize the E of all markers collected. total The sum of the tool's overall center position relative to the camera coordinate system is calculated, denoted by e:

[0104]

[0105] S2-10. Levenberg-Marquardt method, referred to as LM method, this method combines the gradient descent method and Gauss-Newton method, and the update steps are

[0106] Δx=-(J T J+λI) -1 J T E

[0107] Where Δx=[Δq,Δt] is the parameter vector to be optimized, including q tool→cam and t tool→cam , I is the identity matrix. Q tool→cam and t tool→cam The partial derivative matrices of are stacked to obtain J, which represents the overall Jacobian matrix. is the error vector, which contains the rotation error and translation error of all markers.

[0108] S2-11. Use the LM method to calculate the parameter update step size Δx = [Δq, Δt]. Each iteration uses this Δx to adjust q tool→cam and t tool→cam .

[0109] q tool→cam Update: Change the current qtool→cam It is added to its update component Δq and normalized by dividing it by its L2 norm.

[0110]

[0111] t tool→cam Update: Change the current t tool→cam Add to its update component Δt

[0112] t tool→cam ←t tool→cam +Δt

[0113] S2-12. Repeat S2-7 to 2-11 until the error converges to less than the threshold ε or the number of iterations reaches the preset value. Get the optimal q tool→cam and t tool→cam . Use the Rodrigues formula to convert the quaternion q tool→cam Convert to rotation matrix R tool→cam .

[0114] S2-13. When the tool tip is aligned with the center of the calibration plate, the translation vector of the surgical instrument tip in the camera coordinate system is t tip→cam Meaning: t tip→cam =O board→cam .

[0115] Establish the position and posture relationship of the surgical instrument tip in the camera coordinate system.

[0116] t tip→cam =R tool→cam ·t tip→tool +t tool→cam

[0117] Based on this relationship, the position of the surgical instrument tip in the tool coordinate system is deduced and the calibration result t is obtained. tip→tool

[0118] t tip→tool =(R tool→cam ) T (t tip→cam -t tool→cam )

[0119] S3. Real-time positioning of surgical instrument tip

[0120] S3-1. Use the built-in pose estimation function detect in the apriltags library to detect the markers on the tool and detect the R of multiple markers of the tool on a frame image. i→cam ,t i→cam . Keep the R of all the markers in the past k frames (k>0) i→cam ,t i→cam S3-2. Using the t obtained by calibrationtip→tool , calculate the preliminary position estimate of the surgical instrument tip in the camera coordinate system.

[0121] t tip→cam =R tool→cam ·t tip→tool +t tool→cam

[0122] S3-4. Using the known position of the marker in the tool coordinate system and orientation The R of all the markers in the past k frames stored i→cam ,t i→cam As input, the nonlinear optimization method from S2-7 to 2-12 and the position relationship of the surgical instrument tip in the camera coordinate system established by S2-13 are used to obtain the real-time translation vector t of the surgical instrument tip in the camera coordinate system. tip→cam .

[0123] Through the above steps, if Figure 4 As shown in the flow chart, the present invention provides a method for positioning the tip of a surgical instrument based on the AprilTag code, which calculates the position of the tool tip in the camera coordinate system in real time by printing multiple AprilTag codes on a calibration plate and a tool, combined with the conversion between the tool coordinate system and the camera coordinate system.

[0124] Example

[0125] The experiment used a computer equipped with an Intel Core i5-12600KF CPU and 32GB memory, and the stereo camera used an Intel RealSense D405, using the internal parameters calibrated before leaving the factory. It worked in 30fps mode and was connected to the computer via a USB3.0 interface.

[0126] The present invention designs a set of experiments to verify the superiority of the proposed surgical tip positioning method

[0127] The tool and calibration plate structure of the experimental group were modeled using Sketchup and 3D printed. The markers were printed on a laminated adhesive and pasted to the corresponding position of the octagonal prism tool, and the tool was installed on the tail end axis of the surgical handpiece. The control group used the background technology method and used a planar structure as a tool, such as Figure 5 As shown, the markers are pasted on the structure in a 2*4 arrangement.

[0128] The experimental group set the minimum number of detections of each marker with different ID to 50, the number of iterations in the optimization algorithm to 100, and the convergence threshold to 1e -6. k is set to 20 during the positioning process. The present invention designs a positioning accuracy evaluation experiment to evaluate the positioning accuracy of the surgical instrument tip positioning method based on the AprilTag code, and fixes the tool with completed tool tip calibration on a 3D platform with a step resolution of 0.02 mm. The platform moves once every 5 mm along its X and Y axes and once every 5 mm along the Z axis. The position of the surgical tip in the camera coordinate system at each position is recorded to create a 3×3×3 spatial point array.

[0129] In the control group, the center of the plane was used as the origin of the tool coordinate system, and the global poses of all the detected markers on the tool were accumulated and averaged to obtain the pose of the tool center in the camera coordinate system. The translation vector of the surgical instrument tip in the tool coordinate system was obtained by the derivation formula of S2-13 to obtain the calibration result. During the positioning of the surgical instrument tip, the calibration result was used in combination with the average pose of the 8 markers on the tool in the camera coordinate system to calculate the real-time translation vector of the surgical instrument tip in the camera coordinate system to achieve positioning. Similarly, the tool with completed tool tip calibration was fixed on a 3D platform with a step resolution of 0.02 mm. The position of the tool tip in the camera coordinate system at each position was recorded as in the experimental group, and a 3×3×3 spatial point array was created.

[0130] FRE (Fiducial Registration Error) is an error metric used to evaluate registration algorithms.

[0131]

[0132] Where n represents the number of points in a set used for registration evaluation, and p i is the landmark point in the original coordinate system, p i ′ is the corresponding marker point position in the registered coordinate system. FRE mainly measures the error between the marker point (usually a known reference point) and its actual position after transformation in different coordinate systems during the registration process. It represents the deviation between the known reference marker position after registration and its ideal position. The 3×3×3 array is registered to the known actual physical position based on the point-based registration method. Figure 6 As shown in the figure, the FRE of the experimental group registration result is 0.25mm. This meets the requirements of the surgical navigation system for high-precision and high-robust positioning. Figure 7 , the FRE of the control group's registration result was 0.89mm.

[0133] The cumulative distribution function (CDF) is a function used in statistics to describe the cumulative probability of a random variable or data set. For a random variable X, its cumulative distribution function F(x) is defined as

[0134] F(x)=P(X≤x)

[0135] It represents the probability that the random variable X is less than or equal to a certain value x. The experiment compares the distance distribution of different groups of data (GroundTruth, experimental group, and control group), such as Figure 8 These distances describe the degree of deviation of each point from the origin. Through CDF, we can clearly see the error accumulation of different data sets. Compared with the control group, the CDF curve of the experimental group is steeper and accumulates faster at smaller error values. The surgical instrument positioning error of the experimental group method is smaller and the accuracy is higher. It is more suitable for the task of positioning the tip of the surgical instrument.

Claims

1. A surgical instrument tip positioning method based on AprilTag code, comprising the following steps: A tool is attached to the tail of the surgical instrument. The tool is an octagonal prism structure, and AprilTag markers with different IDs are attached to the nine faces. Place markers on the four corners of the calibration plate, and the center of the calibration plate is the origin of the calibration plate coordinate system; Use a camera to capture the marker image and obtain the camera's intrinsic parameters and distortion parameters; Detect and obtain the position and pose of the marker in the camera coordinate system; The pose information of multiple markers is fused through the Levenberg-Marquardt (LM) nonlinear optimization algorithm to calculate the overall pose of the tool; The calibration results are used to calculate the position of the surgical instrument tip in the camera coordinate system in real time.

2. The method according to claim 1, wherein: During the calibration process, the tool is rotated so that markers on different surfaces are exposed to the camera field of view in turn, and the pose data is accumulated to optimize the calibration results of the tool.

3. The method according to claim 1, wherein: During the calibration phase, the tip of the surgical instrument is placed at the center of the calibration plate, and the three-dimensional coordinates of the tip of the surgical instrument in the camera coordinate system are calculated using the markers on the calibration plate.

4. The method according to claim 1, wherein: During the calibration process, after the detection number of each marker reaches a predetermined threshold, the LM method is used to optimize the overall pose of the tool.

5. The method according to claim 1, wherein: During real-time positioning, the marker pose data within the most recent N frames are retained and fused to optimize the real-time position of the surgical instrument tip.

Images