Tool posture evaluation method based on hemispherical laser list method

Abstract

The present invention provides an apparatus posture evaluation method based on the hemispherical laser list method. The method comprises: pasting a transparent chessboard sticker on the outer surface of the hemisphere, using a binocular endoscope measurement system to capture a certain frame k, including two left and right pictures, and calculating a depth map D by a visual matching algorithm. k , obtain the three-dimensional coordinate values ​​of M points in the depth map; place the tip of the instrument at each marked point on the hemisphere, adjust the instrument posture, so that the real-time posture #imgabs0# coincides with the estimated normal vector posture #imgabs1#; project the laser spot onto the ECG, record the posture coordinate value #imgabs2#, and repeat the operations of placing the tip of the instrument, adjusting the posture, projecting the light spot, and recording the posture coordinate value #imgabs3# until a certain number of data samples are reached; calculate the size of the posture error angle each time according to the functional expression of the posture estimation. The instrument posture evaluation method based on the hemispherical laser list method provided by the present invention can increase the number of measurement point samples according to the experimenter's wishes and improve the credibility of the precision analysis.

Description

Technical Field

[0001] The present invention relates to the field of medical-engineering integration technology, and in particular to an instrument posture evaluation method based on a hemispherical laser list method. Background Art

[0002] In most cases, human bones (such as knee joints and the outer contour of the spine) are free-form surfaces with uneven surfaces. When surgery needs to be performed on the bone surface, it is difficult to obtain consistent and reliable accuracy based on the surgeon's personal feeling alone. Posture estimation and positioning based on visual measurement play an extremely important role in the field of medical and engineering integration (such as robot-assisted diagnosis in orthopedic surgery) because it can insert instruments or drill bones with fixed postures and feed amounts. At present, common posture estimation and instrument positioning methods are based on three-dimensional images or two-dimensional key frames. However, these evaluation methods do not set up error analysis based on the characteristics of the methods themselves, but are evaluated through comparative analysis, for example, by comparing with the results of manual operation by experienced surgeons, or by comparing with existing commercial optical positioning instruments as the gold standard. Such comparative analysis can only provide an accuracy reference standard, but cannot discover the source of errors in the posture estimation algorithm, nor can it provide suggestions for controlling or improving errors, so it cannot improve the accuracy of the posture estimation algorithm. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an instrument posture evaluation method based on the hemispherical laser list method, which does not require complex components and harsh experimental environment, and can increase the number of measurement point samples according to the experimenter's wishes to improve the credibility of precision analysis.

[0004] In order to solve the above technical problems, the present invention provides an apparatus posture evaluation method based on the hemispherical laser list method, the method comprising: placing a transparent standard hemispherical model on a medical electrocardiogram recording paper ECG, defining a spherical three-dimensional coordinate system; arranging posture tracking sensors S on the endoscope and the hemispherical model respectively; i (i=2,4); mark n arcs R with a length of πr / 2 on the hemisphere n , and each arc has equally spaced horizontal rotation angles around the z axis In R n Mark N (N ≥ 1) points on the n Equally divided into N+1 sub-arcs; rotate the hemisphere counterclockwise around the z axis Angle, ensure that the ECG paper remains stationary during rotation, and the hemisphere does not translate in the x and y directions; paste the transparent chessboard sticker on the outer surface of the hemisphere, use the binocular endoscope measurement system to capture a frame k, which contains the left and right pictures, and calculate a depth map D by the visual matching algorithm k , get the depth map The three-dimensional coordinate value of each point; place the tip of the instrument on each marked point on the hemisphere, adjust the instrument posture, so that the real-time posture With the estimated normal vector pose Overlap; laser projects a spot onto the ECG and records The attitude coordinate value Repeat the operations of placing the tool tip, adjusting the posture, projecting the light spot, and recording the posture coordinate values ​​until a certain number of data samples are reached; calculate the size of the posture error angle each time based on the functional expression of the posture estimation.

[0005] In some embodiments, a transparent standard hemisphere model is placed on a medical electrocardiogram recording paper ECG, and a spherical three-dimensional coordinate system is defined, including: fixing the medical electrocardiogram recording paper ECG on a non-metallic workbench, placing the transparent standard hemisphere model on the ECG paper, and defining the spherical three-dimensional coordinate system xoy, so that the x and y axes are located on the ECG paper, and the z axis passes through the upper vertex of the hemisphere.

[0006] In some embodiments, posture tracking sensors S are arranged on the endoscope and the hemispherical model respectively. i (i=2,4), including: arranging posture tracking sensors S2 and S4 on the endoscope and the hemispherical model respectively.

[0007] In some embodiments, the arrangement rules of the sensors include: each sensor is rigidly connected to the endoscope and the hemisphere; the transmitter is located within 1 square meter of the hemisphere model to ensure accuracy; during measurement, there must be no metal objects around to avoid affecting accuracy.

[0008] In some embodiments, n arcs R with arc length πr / 2 are marked on the hemisphere. n , and each arc has equally spaced horizontal rotation angles around the z axis Includes: Selecting the initial arc, dividing the horizontal angle equally on the edge of the hemisphere in the Oxy plane Every corner Corresponding to an arc with an arc length of πr / 2, there are n arcs with an arc length of πr / 2 marked on the hemisphere, and each arc is defined as R n (n=1,2,3,…,n).

[0009] In some embodiments, in R n Mark N (N ≥ 1) points on the n Equally divided into N+1 small arcs, including: in arc R n Mark N (N ≥ 1) points on the arc, such as the Nth marked point on the nth arc , it is equally divided into N+1 small arcs, and the central angle θ corresponding to each small arc is equal.

[0010] In some embodiments, after marking, vertex K to point The arc length corresponding to the central angle is .

[0011] In some embodiments, rotating the hemisphere counterclockwise about the z-axis Angle, ensure that the ECG paper remains stationary during rotation, and the hemisphere does not translate in the x and y directions, including: fixing the ECG paper and keeping the top vertex of the hemisphere on the z axis, limiting the freedom of the hemisphere in the x and y directions, and rotating the hemisphere counterclockwise around the z axis angle.

[0012] In some embodiments, a laser projects a spot onto the ECG, recording The attitude coordinate value Including: hemisphere radius r, marking point The central angle of Marking Points Arc R n Corner According to the spherical coordinate system, the arc R can be calculated using the following formula: n On point The three-dimensional coordinate values ​​of

[0013]

[0014] After adopting such a design, the present invention has at least the following advantages:

[0015] The present invention introduces the three-dimensional image acquired by the endoscope and the laser perspective measurement into the posture accuracy evaluation method, and realizes the following innovations: (1) The measurement of the instrument posture error angle is based on the analysis of the acquired three-dimensional image. Compared with other analysis methods, it can better reflect the consistency between the endoscope's own measurement and its own evaluation, and is convenient for finding the error source of the posture algorithm; (2) By utilizing the advantages of the standard transparent ball model, the problem that other geometric shapes (such as planes and cylinders) are difficult to obtain the theoretical normal vector of the measured point is avoided; (3) With the help of the embedded laser module, the current instrument posture is perspectively viewed on the horizontal cross-section of the hemisphere, thereby establishing the error expression between the estimated posture and the theoretical posture. Moreover, the number of samples of the measured points can be increased according to customization to improve the credibility of the evaluation. Combining the above innovations, this method realizes the unity of evaluation reliability and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0017] Figure 1 It is a schematic diagram of the process of the present invention;

[0018] Figure 2 Schematic diagram of sampling the contour of the target object for endoscope;

[0019] Figure 3 It is a schematic diagram of the optical design layout of a binocular endoscope;

[0020] Figure 4 is a schematic diagram of the structure of the apparatus;

[0021] Figure 5 Schematic diagram of device posture estimation and positioning;

[0022] Figure 6 Schematic diagram of the principle of posture evaluation method.

[0023] Description of reference numerals:

[0024] 21 Electromagnetic transmitter 22 Electromagnetic sensor 2

[0025] 23 Binocular endoscope 24 Electromagnetic sensor 4

[0026] 25 Hemisphere Model 26 ECG Recording Paper

[0027] 31 Binocular camera 32 Speckle emitter module

[0028] 33 Electromagnetic sensor 3 41 Electromagnetic sensor

[0029] 42 Through hole 43 Laser DETAILED DESCRIPTION

[0030] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0031] Step 1: Fix the medical electrocardiogram recording paper ECG26 on a non-metallic workbench, place the transparent standard hemisphere model 25 (radius r) on the ECG paper 26, and define the spherical three-dimensional coordinate system xoy so that the x and y axes are located on the ECG paper 26, and the z axis passes through the upper vertex of the hemisphere 25, as shown in Figure 2 shown.

[0032] Step 2: If Figure 3 As shown, posture tracking sensors S222 and S424 are arranged on the endoscope 23 and the hemispherical model 25, respectively. Sensor arrangement principles: (1) Each sensor is rigidly connected to the endoscope 23 and the hemispherical model 25, (2) the transmitter 21 is located at 1 square meter of the hemispherical model 25 to ensure accuracy, and (3) during measurement, there should be no metal objects around to avoid affecting the accuracy.

[0033] Step 3: Select the initial arc, and bisect the horizontal rotation angle (around the z-axis) at equal intervals on the edge of the hemisphere in the Oxy plane. Every corner Corresponding to an arc with an arc length of πr / 2, there are n arcs with an arc length of πr / 2 marked on the hemisphere 25, and each arc is defined as R n (n=1,2,3,…,n).

[0034] Step 4: In arc R n Mark N (N ≥ 1) points on the arc, such as the Nth marked point on the nth arc It is equally divided into N+1 small arcs, and the central angle θ corresponding to each small arc is equal. In addition, the arc length from the upper vertex K of the hemisphere 25 to the point The corresponding central angle is The arc length from vertex K to the point The corresponding central angle is By analogy, the arc length from vertex K to point The corresponding central angle is

[0035] Step 5: Fix the ECG paper 26 and keep the top vertex of the hemisphere 25 on the z-axis, limit the freedom of the hemisphere in the x and y directions, and rotate the hemisphere 25 counterclockwise around the z-axis. angle.

[0036] Step 6: Paste the transparent chessboard sticker on the outer surface of the hemisphere 25, use the binocular endoscope 23 measurement system to capture (require as many marker points as possible to be captured by the left and right cameras) a frame k, which contains the left and right pictures, and calculate a depth map D through the visual matching algorithm (Semiglobal Matching algorithm) k , the depth map has The three-dimensional coordinates of a point.

[0037] When the binocular endoscope 23 scans and calculates the surface of the hemisphere 25, the three-dimensionally reconstructed surface will gradually be displayed in the coordinate system of the electromagnetic transmitter 21. represents a point set in the kth frame of the scanned scene, c P i represents the point cloud coordinates generated in the camera coordinate system of the endoscope 23, and the 3D point cloud can be converted to the coordinate system of the electromagnetic transmitter 21 by formula 1:

[0038]

[0039] In formula (1), represents the orientation of the sensor 222 relative to the transmitter 21, represents the orientation of the camera relative to the sensor 222,t P s2 represents the position of the sensor 222 in the coordinate system of the transmitter 21, s2 d is the relative distance between the sensor 222 and the camera.

[0040] In step 6, the operations of pasting and removing the chessboard paper before and after scanning will cause displacement and rotation of the hemispherical model 25. If equation 1 is used to display the 3D reconstructed surface, the subsequent pose evaluation may be disturbed or interrupted because the hemispherical model 25 has been translated. Therefore, in this solution, we propose a local display method for 3D points. This method has good robustness and anti-motion interference capabilities.

[0041] A sensor 4 24 is added to the hemispherical surface, as described in step 2, and the sensor 4 24 replaces the display method centered on the electromagnetic transmitter 21 in equation 1. Therefore, the 3D point in the kth frame of the scanned scene can be displayed by equation 2:

[0042]

[0043] In formula (2), t P s4 It represents the position of the sensor 424 in the coordinate system of the transmitter 21. The display method based on Formula 2 can avoid the problem of asynchronous positioning and registration after the position and posture offset of the object to be measured after scanning, because Formula 2 is a display method based on the sensor 424, and the sensor 424 and the hemispherical model 25 of the object to be measured are rigidly fixed.

[0044] Step 7: The structure of the device used in this scheme is as follows Figure 4 As shown. The axial direction of the device has a through hole 42, and a micro laser module 43 is embedded in the through hole at the top of the device, and the module 43 can project a laser dot pattern. In addition, an electromagnetic sensor 41 is fixedly installed on the top of the device.

[0045] like Figure 5 As shown, touch the tip of the tool to the marked point on the hemisphere, such as the Nth marked point on the nth arc. At this time, two vectors will be displayed on the computer screen. Adjust the posture of the instrument so that the instrument can move in real time. The normal vector pose obtained by the pose estimation algorithm Project the laser spot, the laser is projected onto the ECG paper 26 through the hemisphere, and the point is recorded on the ECG paper 26 (Oxy plane) with its coordinate values.

[0046] Step 8: If Figure 6 As shown, the hemisphere radius r and the marked point The central angle of Marking Points Arc R n Corner According to the spherical coordinate system, the arc R can be calculated n On point The three-dimensional coordinate values ​​of

[0047]

[0048] Repeat steps 7 and 8 until a certain number of data samples are reached. The more measurements you make, the more meaningful your evaluation becomes.

[0049] Step 9: From step 8, we can know The three-dimensional coordinate values ​​of like Figure 6 As shown, the origin of the coordinate system O and the marked point Coordinates and points of light spot on ECG paper The coordinates of the line segment The length of the hemisphere, and then the radius r of the hemisphere, that is, the known triangle The lengths of the three sides are used to calculate the attitude error angle β of each marker point measurement according to the cosine theorem, as shown in Formula 4. The attitude error angles of multiple measurements are statistically analyzed, and the error accuracy is expressed in the form of average value and standard deviation.

[0050]

[0051] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Those skilled in the art may make some simple modifications, equivalent changes or modifications using the technical contents disclosed above, which all fall within the protection scope of the present invention.

Claims

1. A tool posture evaluation method based on hemispherical laser list method, characterized in that: include: The transparent standard hemisphere model is placed on the medical electrocardiogram recording paper ECG, and the spherical three-dimensional coordinate system is defined, including: Fix the medical electrocardiogram recording paper ECG on a non-metallic workbench, place the transparent standard hemisphere model on the ECG paper, and define the spherical three-dimensional coordinate system xoy so that the x and y axes are located on the ECG paper and the z axis passes through the upper vertex of the hemisphere; Arrange the posture tracking sensors S on the endoscope and the hemispherical model respectively. i (i=2,4); Mark n arcs R of length πr / 2 on the hemisphere n , and each arc has equally spaced horizontal rotation angles around the z axis In R n Mark N points on the n Equally divided into N+1 small arcs; Rotate the hemisphere counterclockwise around the z axis Angle, make sure the ECG paper remains stationary during rotation and that the hemisphere does not translate in the x and y directions; A transparent chessboard sticker is pasted on the outer surface of the hemisphere, and a binocular endoscope measurement system is used to capture a frame k, which contains two left and right pictures. At the same time, a depth map D is calculated by the visual matching algorithm. k , obtain the three-dimensional coordinate values ​​of M points in the depth map; Place the tip of the instrument at each marked point on the hemisphere and adjust the instrument posture so that the real-time posture With the estimated normal vector pose Overlap, project the laser spot, project the laser through the hemisphere onto the ECG paper, and record the points on the Oxy plane on the ECG paper and its coordinate values; The laser projects a spot onto the ECG and records The attitude coordinate value include: The hemisphere radius r and the marking point The central angle of Marking Points Arc R n Corner According to the spherical coordinate system, the arc R can be calculated using the following formula: n On point The three-dimensional coordinate values ​​of Repeat the operations of placing the tool tip, adjusting the posture, projecting the light spot, and recording the posture coordinate values ​​until a certain number of data samples are reached; According to the function expression of attitude estimation, the size of the attitude error angle is calculated each time, including: The origin of the coordinate system O and the marked point Coordinates and points of light spot on ECG paper The coordinates of the line segment The length of the hemisphere, and then the radius r of the hemisphere, that is, the known triangle The lengths of the three sides are used to calculate the attitude error angle β of each marker point measurement according to the cosine theorem. The attitude error angles of multiple measurements are statistically analyzed, and the error accuracy is calculated using the average value and standard deviation.

2. The tool posture evaluation method based on the hemispherical laser list method according to claim 1, characterized in that: Arrange the posture tracking sensors S on the endoscope and the hemispherical model respectively. i (i=2,4), including: Position tracking sensors S2 and S4 are arranged on the endoscope and the hemispherical model respectively.

3. The tool posture evaluation method based on the hemispherical laser list method according to claim 2, characterized in that: The sensor placement rules include: Each sensor is rigidly connected to the endoscope and hemisphere; The transmitter is located near 1 square meter of the hemispherical model; When measuring, there must be no metal objects around.

4. The tool posture evaluation method based on the hemispherical laser list method according to claim 1, characterized in that: Mark n arcs R of length πr / 2 on the hemisphere n , and each arc has equally spaced horizontal rotation angles around the z axis include: Select the initial arc and bisect the horizontal angle at equal intervals on the edge of the hemisphere in the Oxy plane. Every corner Corresponding to an arc with an arc length of πr / 2, there are n arcs with an arc length of πr / 2 marked on the hemisphere, and each arc is defined as R n (n=1,2,3,…,n).

5. The tool posture evaluation method based on the hemispherical laser list method according to claim 1, characterized in that: In R n Mark N points on the n It is divided into N+1 small arcs, including: In the arc R n Mark N points on the nth arc, such as the Nth marked point on the nth arc It is equally divided into N+1 small arcs, and the central angle θ corresponding to each small arc is equal.

6. The tool posture evaluation method based on the hemispherical laser list method according to claim 5, characterized in that: After marking, vertex K to point The arc length corresponding to the central angle is 7. The tool posture evaluation method based on the hemispherical laser list method according to claim 1, characterized in that: Rotate the hemisphere counterclockwise around the z axis Angle, ensure that the ECG paper remains stationary during rotation and that the hemisphere does not translate in the x and y directions, including: Fix the ECG paper and keep the top vertex of the hemisphere on the z-axis, limit the freedom of the hemisphere in the x and y directions, and make the hemisphere rotate counterclockwise around the z-axis. angle.

Images