A structured light-based surgical navigation surface registration method

Abstract

The application discloses a kind of based on structural light's surgical navigation surface registration method, comprising: 1) with projector projection coding structural light template to the surface of object to be measured, optical positioner obtains binocular image with coded light spot;2) processing binocular image obtains the subpixel coordinate point set of all light spot center;3) through decoding and polar line constraint processing point set in step 2), obtain matching point set;4) through triangulation principle, the point set of matching point set is calculated as the surface three-dimensional point set of object to be measured in actual operation space;5) through the CT image of object to be measured, its outer surface point set in image space is obtained, with the point set obtained in step 4) is registered in space, obtains the coordinate conversion relationship of operation space to image space.The application uses coding structural light to obtain the surface three-dimensional point set of object to be measured in actual operation space, not only reduce the probability of subpixel point mismatching, and one binocular image can obtain more three-dimensional points in shorter time, convenient operation.

Description

Technical Field

[0001] This invention relates to the technical field of optical surgical navigation, and in particular to a surgical navigation surface registration method based on structured light. Background Technology

[0002] Compared to traditional surgery, which heavily relies on surgeon experience, optical surgical navigation systems provide a more intuitive and real-time view of the relative positions of lesions and surgical instruments on a display device. This offers surgeons a better field of vision, reduces additional trauma, and promotes minimally invasive and precise surgical procedures. Spatial registration technology is the core technology for obtaining the conversion relationship between medical image space and actual surgical space during optical surgical navigation. The accuracy of spatial registration technology directly affects the application of the surgical navigation system.

[0003] Currently, the spatial registration method commonly used in optical surgical navigation involves attaching several markers to the surface of the subject. Spatial registration is then performed by acquiring the coordinates of the actual surgical space markers and the coordinates of the image space markers. The advantage is high registration accuracy; however, if the markers shift, fall off, or become obstructed, the intraoperative navigation accuracy will be affected. Markerless registration methods, which do not require wearing markers, primarily involve surface registration by acquiring the surface point set of the subject in the actual surgical space and the corresponding surface point set in the image space. However, the surgical tool scanning and single-point laser scanning methods commonly used to acquire the surface point set in the actual surgical space are not only inconvenient to operate but also time-consuming in the acquisition process. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies and propose a structured light-based surgical navigation surface registration method. This method effectively solves the problems of inconvenient operation and time-consuming acquisition processes of existing tools for acquiring actual surgical space point sets in optical surgical navigation. An coded structured light template is designed to address the problem of mismatches easily occurring when performing epipolar constraint matching on a large number of sub-pixel points during binocular stereo matching. This improves the speed and accuracy of acquiring actual surgical space point sets, thereby ensuring the accuracy of surgical navigation.

[0005] To achieve the above objectives, the technical solution provided by this invention is: a surgical navigation surface registration method based on structured light, comprising the following steps:

[0006] S1: Project the coded structured light template onto the surface of the object to be tested using a projector, and simultaneously acquire two images with coded light spots using the binocular camera of the optical positioning instrument.

[0007] S2: Process the two images obtained in step S1 to obtain the sub-pixel coordinate point set P of all light spot centers on the image. l and P r ;

[0008] S3: Based on the encoding information and epipolar constraints, process the sub-pixel coordinate point set P obtained in step S2. l and P r Perform matching to obtain the set of matching points P corresponding to each internal point. l 'and P r ';

[0009] S4: Using the triangulation principle, the matching point set P obtained in step S3 is... l 'and P r 'Calculated as a three-dimensional point set, which is the three-dimensional point set P on the surface of the object to be tested in the actual surgical space;

[0010] S5: Import the CT image sequence of the object to be tested, obtain the outer surface contour point set Q of the object to be tested, use Q as the target point set, and use the point set P obtained in step S4 as the source point set. Through coarse registration and fine registration, obtain the coordinate transformation relationship from surgical space to image space, and complete the surface registration for surgical navigation.

[0011] Furthermore, in step S1, the coded structured light template used is a speckle template, which consists of a dot matrix of circular light spots and a feature light spot that is evenly distributed and has a larger radius than other light spots. This feature light spot is called the anchor point light spot.

[0012] Further, in step S2, the two images obtained in step S1 are binarized by setting a grayscale threshold to obtain the pixel range of the light spots. Then, the grayscale values ​​of the pixels are used as weights to obtain the sub-pixel coordinate point set of all light spot centers in the image. Let the sub-pixel coordinate point sets of the left and right images be P. l and P r .

[0013] Furthermore, the specific steps of step S3 are as follows:

[0014] S31: The point set P obtained in step S2 based on the feature of the spot area size. l and P r Distinguish all anchor point spots in the middle and obtain the set of sub-pixel coordinates of the center of the anchor point spots in the left and right images, C. l and C r C l ∈P l C r ∈P r ;

[0015] S32: Based on the known intrinsic and extrinsic parameters of the binocular camera of the optical positioning instrument, adjust the point set C obtained in step S31. l and C r Perform epipolar constraint matching, specifically by: point set C lEach point C in li There exists a corresponding epipolar line on the right image for each point set C. r Each point C in ri The distance to the polar line is considered to be within a set distance threshold, and point C is considered to be within that threshold range. ri That is point C. li The matching point, if point C is within the threshold range li If no matching point is found, then start from point set C. l Delete point C li If point C li If the number of matching points within this threshold range is greater than 1, then point C will be temporarily set as the match point. li All matching points are calculated as 3D points using the principle of triangulation. Then, based on the rule that 3D points calculated from mismatched point pairs in 3D space are always outliers, some mismatched point pairs are deleted. Following the above operations, the final set of matching points C is obtained. l 'and C r ', point set C l Each point in the point set C r Each has a unique corresponding point;

[0016] S33: The set of matching points C obtained in step S32 l 'and C r ', these are point sets P l and P r A subset of the point set C, perform the following operations: First, set the point set C... l 'and C r 'From point set P respectively l and P r Delete, and you get point set M l and M r , point set C l 'and C r The points in ' are used as seed points for dividing the epipolar constraint matching region. The specific process is as follows: obtain the point set C. l Point C in ' li In M l The neighbor set of points in the set, and the point set C r Point C corresponding to ' ri 'In M r For the nearest neighbor set C, perform epipolar constraint matching within a small range; l 'and C r Perform the above process once for each point in the set N, and save the matching point pairs obtained from this process to the point set N. l and N r Then the point set N l and N r From point set M respectively l and Mr Delete, with point set N l and N r Repeat the above process using the seed point until the point set P is reached. l and P r The points in the set no longer have a matching relationship; merge the matching point set saved by repeatedly performing the above process and the center sub-pixel coordinate matching point set C of the anchor spot. l 'and C r 'For the final set of matching points P l 'and P r ', point set P l Each point in the point set P r Each has a unique corresponding point.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0018] 1. This invention designs an coded structured light template, which uses the area as a feature of the anchor point light spot to distinguish it from other light spots. The anchor point light spot is used as a seed point to divide the epipolar constraint matching region. The small-range epipolar constraint matching method reduces the number of mismatches of sub-pixel points and improves the accuracy of the acquired three-dimensional points.

[0019] 2. This invention acquires a set of actual surgical space surface points for surface registration in optical surgical navigation. It can obtain a larger set of actual surgical space surface points in the same amount of time than it can be obtained by surgical tools and single-point laser scanning, using only two images acquired in a single frame by a binocular camera. At the same time, it takes less time to acquire the point set compared to line structured light, and the operation process is more convenient. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the method of the present invention.

[0021] Figure 2 This is a schematic diagram of the coded structured light template for projection.

[0022] Figure 3 A schematic diagram of the left and right light spot images acquired by a binocular camera.

[0023] Figure 4 This is a schematic diagram illustrating the division of the epipolar constraint matching region using seed points.

[0024] Figure 5 This is a schematic diagram of the point set P on the surface of the object to be tested in the actual surgical space.

[0025] Figure 6 A schematic diagram of the surface registration results for the image spatial point set Q and the actual surgical spatial point set P. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0027] like Figure 1 As shown, this embodiment provides a surgical navigation surface registration method based on structured light, including the following steps:

[0028] Step S101, the coded structured light template used consists of a dot matrix of circular light spots and uniformly distributed feature light spots with a larger radius than other light spots. These feature light spots are called anchor point light spots, such as... Figure 2 As shown, the template is projected onto the surface of the object to be measured as comprehensively as possible to ensure that the images captured by the binocular camera are as complete as possible. The two images obtained are shown below. Figure 3 As shown.

[0029] Step S102: Binarize the two acquired images by setting a grayscale threshold to obtain the pixel range of the light spots. Then, use the grayscale values ​​of the pixels as weights to obtain the sub-pixel coordinate point set of all light spot centers in the image. Let P be the sub-pixel coordinate point set of the light spot centers of the two images. l and P r .

[0030] Area features can be used to distinguish anchor spot from other spots, thus obtaining the set of sub-pixel coordinates C of the anchor spot's center. l and C r C l ∈P l C r ∈P r First, for the point set C l and C r Perform epipolar constraint matching, such as Figure 4 As shown, since the point set is uniformly and dispersedly distributed on the image, the probability of mismatch is much lower than that of epipolar constraint matching using the sub-pixel coordinate point set of the entire image. The specific operation is as follows: Point set C l Each point C in li An epipolar line can be drawn on the corresponding right image, and the point set C can be calculated. r Each point C in ri The distance to the polar line is considered to be within a set distance threshold, and point C is considered to be within that threshold range. ri That is point C. li The matching point, if point C is within the threshold range li If no matching point is found, then start from point set C. l Delete point C li If point C li If the number of matching points within this threshold range is greater than 1, then point C will be temporarily set as the match point. liAll matching points are calculated as 3D points using the principle of triangulation. Based on the rule that 3D points calculated from mismatched point pairs in 3D space are always outliers, some mismatched point pairs are deleted. Following the above operations, the final set of matching points C is obtained. l 'and C r ', point set C l Each point in the point set C r Each has a unique corresponding point.

[0031] At this time, point set C l 'and C r 'Still, they are point sets P' l and P r A subset of C l '∈P l C r '∈P r Perform the following operations: First, set the point set C... l 'and C r 'From point set P respectively l and P r Delete, and you get point set M l and M r Point set C l 'and C r The points in ' are used as seed points for dividing the epipolar constraint matching region. The specific process is as follows: obtain the point set C. l Point C in ' li In M l The neighbor set of points in the set, and the point set C r Point C corresponding to ' ri 'In M r For the nearest neighbor set C, perform epipolar constraint matching within a small range; l 'and C r Perform the above process once for each point in the set N, and save the matching point pairs obtained from this process to the point set N. l and N r Then the point set N l and N r From point set M respectively l and M r Delete, with point set N l and N r Repeat the above process using the seed point until the point set P is reached. l and P r The points in the set no longer have a matching relationship; merge the matching point set saved by repeatedly performing the above process and the center sub-pixel coordinate matching point set C of the anchor spot. l 'and C r 'For the final set of matching points P l 'and P r ', point set Pl Each point in the point set P r Each has a unique corresponding point.

[0032] Step S103: According to the principle of triangulation, the matching point set P obtained in step S102 is... l 'and P r 'Calculated as a 3D point set P, this point set P is the surface point set of the object under test in the actual surgical space. It can be obtained by taking multiple pictures with a binocular camera, calculating the 3D point set using multiple pairs of images, merging all the obtained 3D point sets, and removing some outliers. The result is as follows.' Figure 5 As shown.

[0033] Step S104: The point set corresponding to the surface point set P of the test object in the actual surgical space obtained in step S103 is the outer surface point set Q in the CT image space. Typically, after importing the CT image sequence of the test object, the corresponding outer surface contour point set Q is obtained by binarizing the volume data slices and filling in the holes. Image segmentation is performed on point sets P and Q to obtain the corresponding feature parts of the two point sets. Then, Principal Component Analysis (PCA) algorithm is used to coarsely register point sets P and Q, roughly aligning the two point sets to obtain the transformation matrix S1 from image space to actual surgical space. Next, the nearest point iterative ICP algorithm is used for fine registration to obtain the optimal registration result of the two point sets, resulting in the fine registration transformation matrix S2. The final surface registration result, i.e., the transformation matrix between the actual surgical space and the CT image space, is S = S2 * S1. The registration results of the two point sets are as follows: Figure 6 As shown, the set of points Q on the outer surface of the CT image space is brighter and has more points, while the set of points P on the surface of the object under test in the actual surgical space is darker and has fewer points.

[0034] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A surgical navigation surface registration method based on structured light, characterized in that, Includes the following steps: S1: Project the coded structured light template onto the surface of the object to be measured using a projector, and simultaneously acquire two images with coded light spots using the binocular camera of the optical positioning instrument; the coded structured light template used is a speckle template, which consists of a dot matrix of circular light spots and a feature light spot that is evenly distributed and has a larger radius than other light spots. This feature light spot is called the anchor point light spot. S2: Process the two images obtained in step S1 to obtain the sub-pixel coordinate point set of all light spot centers on the image. and ; S3: Based on the encoding information and epipolar constraints, process the sub-pixel coordinate point set obtained in step S2. and Perform matching to obtain a set of matching points corresponding to each internal point. and ; S4: Using the triangulation principle, the matching point set obtained in step S3 is... and The calculation is performed as a three-dimensional point set, which is the three-dimensional point set of the surface of the object under test in the actual surgical space. ; S5: Import the CT image sequence of the object to be tested and obtain the set of contour points on the outer surface of the object to be tested. ,Will The point set obtained in step S4 serves as the target point set. As a source point set, the coordinate transformation relationship from surgical space to image space is obtained through coarse registration and fine registration, thus completing the surface registration for surgical navigation.

2. The surgical navigation surface registration method based on structured light according to claim 1, characterized in that, In step S2, the two images obtained in step S1 are binarized by setting a grayscale threshold to obtain the pixel range of the light spots. Then, the grayscale values ​​of the pixels are used as weights to obtain the sub-pixel coordinate point set of all light spot centers in the image. Let the sub-pixel coordinate point sets of the left and right images be respectively... and .

3. The surgical navigation surface registration method based on structured light according to claim 2, characterized in that, The specific steps of step S3 are as follows: S31: The point set obtained in step S2 based on the feature of the spot area size. and Distinguish all anchor point spots in the middle and obtain the set of sub-pixel coordinates of the center of the anchor point spots in the left and right images. and , , ; S32: Based on the known intrinsic and extrinsic parameters of the binocular camera of the optical positioning instrument, adjust the point set obtained in step S31. and Performing epipolar constraint matching involves the following steps: point set Each point in There exists a corresponding epipolar line on the right image for each point set. Each point in The distance to the polar line is considered to be within a set distance threshold, meaning points within that threshold range are considered to be within the range. That is the point The matching point, if the points are within the threshold range If no matching point is found, then start from the point set. Delete point If you click If the number of matching points within this threshold range is greater than 1, then temporarily set the points as... All matching points are calculated as 3D points using the principle of triangulation. Then, based on the rule that 3D points calculated from mismatched point pairs in 3D space must be outliers, some mismatched point pairs are deleted. Following the above operations, the final set of matching points is obtained. and dot set Each point in the point set Each has a unique corresponding point; S33: The set of matching points obtained in step S32 and They are point sets. and A subset of the point set, perform the following operations: First, set the point set... and From point set respectively and Delete to obtain the point set and , point set and The points in the set are used as seed points to divide the epipolar constraint matching region. The specific process is as follows: obtain the point set. Points in exist The neighbor set in the point set, and the point set The corresponding point in exist The nearest neighbor set is used for small-scale epipolar constraint matching; point set and Perform the above process once for each point in the dataset, and save the matching point pairs obtained from this process to the point set. and Then the point set and From point set respectively and Deleted, using point set and Repeat the above process using seed points until the point set is reached. and The points in the array no longer have a matching relationship; merge the matching point set saved by repeatedly performing the above process and the center sub-pixel coordinate matching point set of the anchor spot. and For the final set of matching points and dot set Each point in the point set Each has a unique corresponding point.

Images